2024-04-20T10:00:55.651 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-20T10:00:55.651 DEBUG:teuthology.run:Teuthology command: teuthology --name rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/xfstests-dev} --owner scheduled_rishabh@teuthology --archive /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664420 --verbose -- /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664420/orig.config.yaml 2024-04-20T10:00:55.696 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-20T10:00:55.784 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664420 branch: wip-rishabh-testing-20240416.193735 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/xfstests-dev} email: null first_in_suite: false job_id: '7664420' kernel: flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240416.193735 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: a8c84f5997950fd7d008204839253291d52e6eda ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: a8c84f5997950fd7d008204839253291d52e6eda selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240416.193735 sha1: a8c84f5997950fd7d008204839253291d52e6eda owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 7954 sha1: a8c84f5997950fd7d008204839253291d52e6eda sleep_before_teardown: 0 subset: 364/512 suite: fs suite_branch: wip-rishabh-testing-20240416.193735 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a8c84f5997950fd7d008204839253291d52e6eda syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi121.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDwe06AVl2/32bDUjOa4CZ3632mEG9zF+J0zWvWIBKREkq+95M61hZTmM7tTEDHkajzjNb/g3ICfn2BJO3iRSno= smithi123.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL4H76MxIbiuUrrg5fWvZxX2p/H8QEzrAnJWPuUM0ZDIde5fB3OeGVfnDBwnxJu/suKVEmGmC37T1rQ/TOw3cI8= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.tests_from_xfstests_dev teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-20_09:50:15 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-20T10:00:55.784 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa; will attempt to use it 2024-04-20T10:00:55.785 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks 2024-04-20T10:00:55.785 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-20T10:00:55.787 INFO:teuthology.task.internal:Checking packages... 2024-04-20T10:00:55.811 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a8c84f5997950fd7d008204839253291d52e6eda' 2024-04-20T10:00:55.811 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-20T10:00:55.811 INFO:teuthology.packaging:ref: None 2024-04-20T10:00:55.811 INFO:teuthology.packaging:tag: None 2024-04-20T10:00:55.811 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240416.193735 2024-04-20T10:00:55.811 INFO:teuthology.packaging:sha1: a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T10:00:55.811 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240416.193735 2024-04-20T10:00:55.953 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3133.ga8c84f59 2024-04-20T10:00:55.955 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-20T10:00:55.962 INFO:teuthology.task.internal:no buildpackages task found 2024-04-20T10:00:55.962 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-20T10:00:55.980 INFO:teuthology.task.internal:Saving configuration 2024-04-20T10:00:55.996 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-20T10:00:56.002 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-20T10:00:56.026 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi121.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664420', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 09:51:09.143236', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDwe06AVl2/32bDUjOa4CZ3632mEG9zF+J0zWvWIBKREkq+95M61hZTmM7tTEDHkajzjNb/g3ICfn2BJO3iRSno='} 2024-04-20T10:00:56.047 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi123.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664420', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 09:51:09.141926', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL4H76MxIbiuUrrg5fWvZxX2p/H8QEzrAnJWPuUM0ZDIde5fB3OeGVfnDBwnxJu/suKVEmGmC37T1rQ/TOw3cI8='} 2024-04-20T10:00:56.047 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-20T10:00:56.054 INFO:teuthology.task.internal:roles: ubuntu@smithi121.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-20T10:00:56.054 INFO:teuthology.task.internal:roles: ubuntu@smithi123.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-20T10:00:56.054 INFO:teuthology.run_tasks:Running task console_log... 2024-04-20T10:00:56.134 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fddec041160>, signals=[15]) 2024-04-20T10:00:56.135 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-20T10:00:56.142 INFO:teuthology.task.internal:Opening connections... 2024-04-20T10:00:56.142 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi121.front.sepia.ceph.com 2024-04-20T10:00:56.144 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:00:56.220 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi123.front.sepia.ceph.com 2024-04-20T10:00:56.221 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:00:56.295 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-20T10:00:56.302 DEBUG:teuthology.orchestra.run.smithi121:> uname -m 2024-04-20T10:00:56.320 INFO:teuthology.orchestra.run.smithi121.stdout:x86_64 2024-04-20T10:00:56.321 DEBUG:teuthology.orchestra.run.smithi121:> cat /etc/os-release 2024-04-20T10:00:56.378 INFO:teuthology.orchestra.run.smithi121.stdout:NAME="CentOS Stream" 2024-04-20T10:00:56.378 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION="9" 2024-04-20T10:00:56.378 INFO:teuthology.orchestra.run.smithi121.stdout:ID="centos" 2024-04-20T10:00:56.378 INFO:teuthology.orchestra.run.smithi121.stdout:ID_LIKE="rhel fedora" 2024-04-20T10:00:56.378 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION_ID="9" 2024-04-20T10:00:56.378 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM_ID="platform:el9" 2024-04-20T10:00:56.378 INFO:teuthology.orchestra.run.smithi121.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-20T10:00:56.378 INFO:teuthology.orchestra.run.smithi121.stdout:ANSI_COLOR="0;31" 2024-04-20T10:00:56.378 INFO:teuthology.orchestra.run.smithi121.stdout:LOGO="fedora-logo-icon" 2024-04-20T10:00:56.379 INFO:teuthology.orchestra.run.smithi121.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-20T10:00:56.379 INFO:teuthology.orchestra.run.smithi121.stdout:HOME_URL="https://centos.org/" 2024-04-20T10:00:56.379 INFO:teuthology.orchestra.run.smithi121.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-20T10:00:56.379 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-20T10:00:56.379 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-20T10:00:56.379 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server 2024-04-20T10:00:56.404 DEBUG:teuthology.orchestra.run.smithi123:> uname -m 2024-04-20T10:00:56.421 INFO:teuthology.orchestra.run.smithi123.stdout:x86_64 2024-04-20T10:00:56.422 DEBUG:teuthology.orchestra.run.smithi123:> cat /etc/os-release 2024-04-20T10:00:56.478 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="CentOS Stream" 2024-04-20T10:00:56.479 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="9" 2024-04-20T10:00:56.479 INFO:teuthology.orchestra.run.smithi123.stdout:ID="centos" 2024-04-20T10:00:56.479 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE="rhel fedora" 2024-04-20T10:00:56.479 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="9" 2024-04-20T10:00:56.479 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM_ID="platform:el9" 2024-04-20T10:00:56.479 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-20T10:00:56.479 INFO:teuthology.orchestra.run.smithi123.stdout:ANSI_COLOR="0;31" 2024-04-20T10:00:56.479 INFO:teuthology.orchestra.run.smithi123.stdout:LOGO="fedora-logo-icon" 2024-04-20T10:00:56.479 INFO:teuthology.orchestra.run.smithi123.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-20T10:00:56.479 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://centos.org/" 2024-04-20T10:00:56.479 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-20T10:00:56.479 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-20T10:00:56.479 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-20T10:00:56.480 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server 2024-04-20T10:00:56.504 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-20T10:00:56.513 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-20T10:00:56.520 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-20T10:00:56.520 DEBUG:teuthology.orchestra.run.smithi121:> test '!' -e /home/ubuntu/cephtest 2024-04-20T10:00:56.522 DEBUG:teuthology.orchestra.run.smithi123:> test '!' -e /home/ubuntu/cephtest 2024-04-20T10:00:56.538 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-20T10:00:56.545 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-20T10:00:56.545 DEBUG:teuthology.orchestra.run.smithi121:> test -z $(ls -A /var/lib/ceph) 2024-04-20T10:00:56.580 DEBUG:teuthology.orchestra.run.smithi123:> test -z $(ls -A /var/lib/ceph) 2024-04-20T10:00:56.674 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-20T10:00:56.731 INFO:teuthology.run_tasks:Running task kernel... 2024-04-20T10:00:56.746 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-20T10:00:56.746 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, timeout 300 2024-04-20T10:00:56.747 DEBUG:teuthology.orchestra.run.smithi121:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-20T10:00:56.748 DEBUG:teuthology.orchestra.run.smithi123:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-20T10:00:56.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:00:56.764 DEBUG:teuthology.task.kernel:sha1 for mon.a is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T10:00:56.764 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-20T10:00:56.764 DEBUG:teuthology.orchestra.run.smithi121:> uname -r 2024-04-20T10:00:56.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:00:56.766 DEBUG:teuthology.task.kernel:sha1 for mon.b is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T10:00:56.766 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-20T10:00:56.766 DEBUG:teuthology.orchestra.run.smithi123:> uname -r 2024-04-20T10:00:56.821 INFO:teuthology.orchestra.run.smithi121.stdout:5.14.0-437.el9.x86_64 2024-04-20T10:00:56.821 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T10:00:56.822 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-20T10:00:56.822 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T10:00:56.825 INFO:teuthology.orchestra.run.smithi123.stdout:5.14.0-437.el9.x86_64 2024-04-20T10:00:56.826 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T10:00:56.826 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-20T10:00:56.826 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T10:00:56.967 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum upgrade -y linux-firmware 2024-04-20T10:00:56.970 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum upgrade -y linux-firmware 2024-04-20T10:00:59.687 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - BaseOS 7.1 MB/s | 8.0 MB 00:01 2024-04-20T10:01:01.181 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - BaseOS 3.2 MB/s | 8.0 MB 00:02 2024-04-20T10:01:03.554 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - AppStream 6.7 MB/s | 19 MB 00:02 2024-04-20T10:01:05.012 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - AppStream 6.8 MB/s | 19 MB 00:02 2024-04-20T10:01:09.438 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - CRB 8.3 MB/s | 6.2 MB 00:00 2024-04-20T10:01:10.396 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - CRB 1.9 MB/s | 6.2 MB 00:03 2024-04-20T10:01:11.905 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 16 kB 00:00 2024-04-20T10:01:13.280 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - Extras packages 6.5 kB/s | 16 kB 00:02 2024-04-20T10:01:13.633 INFO:teuthology.orchestra.run.smithi121.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-20T10:01:18.642 INFO:teuthology.orchestra.run.smithi121.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-04-20T10:01:20.630 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-04-20T10:01:20.631 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2024-04-20T10:01:20.631 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-04-20T10:01:20.740 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.a... 2024-04-20T10:01:20.741 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T10:01:21.000 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 2.7 MB/s | 21 MB 00:07 2024-04-20T10:01:21.065 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-20T10:01:21.066 DEBUG:teuthology.orchestra.run.smithi121:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-20T10:01:21.068 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi121.front.sepia.ceph.com to complete... 2024-04-20T10:01:26.021 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-20T10:01:28.031 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-04-20T10:01:28.032 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-04-20T10:01:28.032 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-04-20T10:01:28.148 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.b... 2024-04-20T10:01:28.148 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T10:01:28.432 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-20T10:01:28.432 DEBUG:teuthology.orchestra.run.smithi123:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-20T10:01:28.435 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi123.front.sepia.ceph.com to complete... 2024-04-20T10:09:13.731 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20 10:09:13 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-20T10:09:13.732 INFO:teuthology.orchestra.run.smithi121.stderr:FINISHED --2024-04-20 10:09:13-- 2024-04-20T10:09:13.732 INFO:teuthology.orchestra.run.smithi121.stderr:Total wall clock time: 7m 53s 2024-04-20T10:09:13.732 INFO:teuthology.orchestra.run.smithi121.stderr:Downloaded: 1 files, 341M in 7m 52s (739 KB/s) 2024-04-20T10:09:13.735 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.a... 2024-04-20T10:09:13.736 DEBUG:teuthology.orchestra.run.smithi121:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-20T10:09:15.834 INFO:teuthology.orchestra.run.smithi121.stdout:Verifying... ######################################## 2024-04-20T10:09:15.964 INFO:teuthology.orchestra.run.smithi121.stdout:Preparing... ######################################## 2024-04-20T10:09:16.074 INFO:teuthology.orchestra.run.smithi121.stdout:Updating / installing... 2024-04-20T10:09:24.095 INFO:teuthology.orchestra.run.smithi121.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-20T10:09:43.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20 10:09:43 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-20T10:09:43.770 INFO:teuthology.orchestra.run.smithi123.stderr:FINISHED --2024-04-20 10:09:43-- 2024-04-20T10:09:43.770 INFO:teuthology.orchestra.run.smithi123.stderr:Total wall clock time: 8m 15s 2024-04-20T10:09:43.770 INFO:teuthology.orchestra.run.smithi123.stderr:Downloaded: 1 files, 341M in 8m 15s (706 KB/s) 2024-04-20T10:09:43.774 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.b... 2024-04-20T10:09:43.775 DEBUG:teuthology.orchestra.run.smithi123:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-20T10:09:45.929 INFO:teuthology.orchestra.run.smithi123.stdout:Verifying... ######################################## 2024-04-20T10:09:46.047 INFO:teuthology.orchestra.run.smithi123.stdout:Preparing... ######################################## 2024-04-20T10:09:46.158 INFO:teuthology.orchestra.run.smithi123.stdout:Updating / installing... 2024-04-20T10:09:54.140 INFO:teuthology.orchestra.run.smithi123.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-20T10:10:01.327 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi121.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-20T10:10:01.328 DEBUG:teuthology.orchestra.run.smithi121:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-20T10:10:01.408 INFO:teuthology.orchestra.run.smithi121.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-20T10:10:01.408 INFO:teuthology.orchestra.run.smithi121.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-20T10:10:01.408 INFO:teuthology.orchestra.run.smithi121.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-20T10:10:01.409 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-20T10:10:01.409 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-20T10:10:01.409 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-20T10:10:01.409 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-20T10:10:01.409 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-20T10:10:01.409 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-20T10:10:01.409 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-20T10:10:01.409 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-20T10:10:01.409 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-20T10:10:01.409 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-20T10:10:01.410 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-20T10:10:01.410 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-20T10:10:01.410 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-20T10:10:01.410 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-20T10:10:01.410 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-20T10:10:01.410 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-20T10:10:01.410 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-20T10:10:01.410 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-20T10:10:01.410 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-20T10:10:01.410 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-20T10:10:01.410 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-20T10:10:01.410 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-20T10:10:01.411 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-20T10:10:01.411 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-20T10:10:01.411 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-20T10:10:01.411 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-20T10:10:01.411 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-20T10:10:01.411 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-20T10:10:01.411 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-20T10:10:01.411 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-20T10:10:01.411 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-20T10:10:01.411 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-20T10:10:01.411 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-20T10:10:01.411 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-20T10:10:01.411 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-20T10:10:01.411 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-20T10:10:01.412 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-20T10:10:01.412 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-20T10:10:01.412 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-20T10:10:01.412 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-20T10:10:01.412 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-20T10:10:01.412 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-20T10:10:01.412 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-20T10:10:01.412 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-20T10:10:01.412 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-20T10:10:01.413 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-20T10:10:01.413 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-20T10:10:01.413 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-20T10:10:01.413 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-20T10:10:01.413 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-20T10:10:01.413 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-20T10:10:01.413 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-20T10:10:01.413 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-20T10:10:01.413 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-20T10:10:01.413 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-20T10:10:01.414 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-20T10:10:01.414 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-20T10:10:01.414 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-20T10:10:01.414 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-20T10:10:01.414 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-20T10:10:01.414 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-20T10:10:01.414 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-20T10:10:01.414 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-20T10:10:01.414 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-20T10:10:01.414 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-20T10:10:01.414 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-20T10:10:01.414 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-20T10:10:01.414 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-20T10:10:01.415 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-20T10:10:01.415 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-20T10:10:01.415 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-20T10:10:01.415 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-20T10:10:01.415 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-20T10:10:01.415 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-20T10:10:01.415 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-20T10:10:01.415 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-20T10:10:01.415 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-20T10:10:01.415 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-20T10:10:01.415 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-20T10:10:01.415 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-20T10:10:01.415 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-20T10:10:01.415 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-20T10:10:01.416 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-20T10:10:01.416 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-20T10:10:01.416 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-20T10:10:01.416 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-20T10:10:01.416 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-20T10:10:01.416 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-20T10:10:01.416 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-20T10:10:01.416 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-20T10:10:01.416 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-20T10:10:01.416 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-20T10:10:01.416 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-20T10:10:01.416 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-20T10:10:01.416 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-20T10:10:01.416 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-20T10:10:01.417 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-20T10:10:01.417 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-20T10:10:01.417 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-20T10:10:01.417 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-20T10:10:01.417 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-20T10:10:01.417 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-20T10:10:01.417 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-20T10:10:01.417 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-20T10:10:01.417 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-20T10:10:01.417 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-20T10:10:01.417 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-20T10:10:01.417 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-20T10:10:01.417 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-20T10:10:01.417 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-20T10:10:01.418 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-20T10:10:01.418 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-20T10:10:01.418 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-20T10:10:01.418 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-20T10:10:01.418 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-20T10:10:01.418 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-20T10:10:01.418 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-20T10:10:01.418 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-20T10:10:01.418 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-20T10:10:01.418 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-20T10:10:01.418 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-20T10:10:01.418 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-20T10:10:01.419 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-20T10:10:01.419 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-20T10:10:01.419 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-20T10:10:01.419 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-20T10:10:01.419 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-20T10:10:01.419 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-20T10:10:01.419 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-20T10:10:01.419 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-20T10:10:01.419 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-20T10:10:01.419 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-20T10:10:01.419 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-20T10:10:01.419 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-20T10:10:01.419 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-20T10:10:01.419 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-20T10:10:01.420 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-20T10:10:01.420 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-20T10:10:01.420 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-20T10:10:01.420 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-20T10:10:01.420 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-20T10:10:01.420 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-20T10:10:01.420 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-20T10:10:01.420 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-20T10:10:01.420 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-20T10:10:01.420 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-20T10:10:01.420 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-20T10:10:01.421 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-20T10:10:01.421 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-20T10:10:01.421 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-20T10:10:01.421 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-20T10:10:01.421 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-20T10:10:01.421 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-20T10:10:01.421 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-20T10:10:01.421 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-20T10:10:01.421 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-20T10:10:01.421 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-20T10:10:01.421 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-20T10:10:01.421 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-20T10:10:01.421 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-20T10:10:01.422 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-20T10:10:01.422 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-20T10:10:01.422 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-20T10:10:01.422 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-20T10:10:01.422 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-20T10:10:01.422 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-20T10:10:01.422 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-20T10:10:01.422 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-20T10:10:01.422 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-20T10:10:01.422 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-20T10:10:01.423 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-20T10:10:01.423 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-20T10:10:01.423 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-20T10:10:01.423 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-20T10:10:01.423 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-20T10:10:01.423 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-20T10:10:01.423 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-20T10:10:01.423 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-20T10:10:01.423 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-20T10:10:01.423 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-20T10:10:01.423 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-20T10:10:01.423 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-20T10:10:01.423 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-20T10:10:01.423 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-20T10:10:01.424 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-20T10:10:01.424 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-20T10:10:01.424 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-20T10:10:01.424 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-20T10:10:01.424 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-20T10:10:01.424 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-20T10:10:01.424 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-20T10:10:01.424 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-20T10:10:01.424 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-20T10:10:01.424 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-20T10:10:01.425 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-20T10:10:01.425 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-20T10:10:01.425 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-20T10:10:01.425 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-20T10:10:01.425 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-20T10:10:01.425 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-20T10:10:01.425 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-20T10:10:01.426 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-20T10:10:01.426 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-20T10:10:01.426 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-20T10:10:01.426 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-20T10:10:01.426 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-20T10:10:01.426 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-20T10:10:01.426 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-20T10:10:01.427 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-20T10:10:01.427 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-20T10:10:01.427 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-20T10:10:01.427 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-20T10:10:01.427 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-20T10:10:01.427 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-20T10:10:01.427 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-20T10:10:01.427 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-20T10:10:01.427 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-20T10:10:01.428 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-20T10:10:01.428 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-20T10:10:01.428 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-20T10:10:01.428 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-20T10:10:01.428 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-20T10:10:01.428 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-20T10:10:01.428 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-20T10:10:01.428 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-20T10:10:01.429 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-20T10:10:01.429 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-20T10:10:01.429 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-20T10:10:01.429 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-20T10:10:01.429 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-20T10:10:01.429 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-20T10:10:01.429 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-20T10:10:01.429 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-20T10:10:01.430 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-20T10:10:01.430 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-20T10:10:01.430 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-20T10:10:01.430 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-20T10:10:01.430 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-20T10:10:01.430 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-20T10:10:01.430 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-20T10:10:01.430 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-20T10:10:01.430 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-20T10:10:01.431 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-20T10:10:01.431 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-20T10:10:01.431 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-20T10:10:01.431 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-20T10:10:01.431 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-20T10:10:01.431 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-20T10:10:01.431 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-20T10:10:01.431 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-20T10:10:01.431 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-20T10:10:01.431 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-20T10:10:01.431 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-20T10:10:01.431 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-20T10:10:01.431 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-20T10:10:01.431 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-20T10:10:01.432 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-20T10:10:01.432 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-20T10:10:01.432 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-20T10:10:01.432 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-20T10:10:01.432 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-20T10:10:01.432 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-20T10:10:01.432 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-20T10:10:01.432 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-20T10:10:01.432 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-20T10:10:01.432 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-20T10:10:01.432 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-20T10:10:01.432 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-20T10:10:01.433 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-20T10:10:01.433 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-20T10:10:01.433 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-20T10:10:01.433 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-20T10:10:01.433 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-20T10:10:01.433 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-20T10:10:01.433 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-20T10:10:01.433 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-20T10:10:01.433 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-20T10:10:01.433 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-20T10:10:01.433 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-20T10:10:01.433 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-20T10:10:01.433 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-20T10:10:01.434 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-20T10:10:01.434 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-20T10:10:01.434 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-20T10:10:01.434 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-20T10:10:01.434 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-20T10:10:01.434 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-20T10:10:01.434 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-20T10:10:01.434 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-20T10:10:01.434 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-20T10:10:01.434 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-20T10:10:01.434 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-20T10:10:01.434 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-20T10:10:01.434 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-20T10:10:01.435 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-20T10:10:01.435 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-20T10:10:01.435 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-20T10:10:01.435 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-20T10:10:01.435 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-20T10:10:01.435 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-20T10:10:01.435 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-20T10:10:01.435 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-20T10:10:01.435 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-20T10:10:01.435 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-20T10:10:01.435 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-20T10:10:01.435 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-20T10:10:01.435 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-20T10:10:01.436 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-20T10:10:01.436 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-20T10:10:01.436 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-20T10:10:01.436 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-20T10:10:01.436 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-20T10:10:01.436 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-20T10:10:01.436 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-20T10:10:01.436 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-20T10:10:01.436 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-20T10:10:01.436 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-20T10:10:01.436 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-20T10:10:01.436 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-20T10:10:01.436 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-20T10:10:01.436 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-20T10:10:01.437 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-20T10:10:01.437 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-20T10:10:01.437 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-20T10:10:01.437 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-20T10:10:01.437 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-20T10:10:01.437 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-20T10:10:01.437 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-20T10:10:01.437 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-20T10:10:01.437 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-20T10:10:01.437 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-20T10:10:01.437 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-20T10:10:01.437 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-20T10:10:01.437 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-20T10:10:01.438 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-20T10:10:01.438 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-20T10:10:01.438 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-20T10:10:01.438 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-20T10:10:01.438 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-20T10:10:01.438 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-20T10:10:01.438 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-20T10:10:01.438 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-20T10:10:01.438 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-20T10:10:01.438 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-20T10:10:01.438 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-20T10:10:01.439 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-20T10:10:01.439 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-20T10:10:01.439 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-20T10:10:01.439 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-20T10:10:01.439 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-20T10:10:01.439 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-20T10:10:01.439 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-20T10:10:01.439 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-20T10:10:01.439 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-20T10:10:01.439 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-20T10:10:01.439 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-20T10:10:01.439 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-20T10:10:01.439 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-20T10:10:01.440 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-20T10:10:01.440 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-20T10:10:01.440 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-20T10:10:01.440 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-20T10:10:01.440 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-20T10:10:01.440 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-20T10:10:01.440 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-20T10:10:01.440 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-20T10:10:01.440 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-20T10:10:01.440 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-20T10:10:01.440 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-20T10:10:01.440 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-20T10:10:01.440 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-20T10:10:01.440 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-20T10:10:01.441 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-20T10:10:01.441 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-20T10:10:01.441 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-20T10:10:01.441 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-20T10:10:01.441 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-20T10:10:01.441 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-20T10:10:01.441 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-20T10:10:01.441 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-20T10:10:01.441 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-20T10:10:01.441 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-20T10:10:01.441 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-20T10:10:01.441 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-20T10:10:01.441 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-20T10:10:01.441 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-20T10:10:01.442 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-20T10:10:01.442 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-20T10:10:01.442 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-20T10:10:01.442 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-20T10:10:01.442 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-20T10:10:01.442 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-20T10:10:01.442 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-20T10:10:01.442 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-20T10:10:01.442 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-20T10:10:01.442 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-20T10:10:01.442 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-20T10:10:01.442 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-20T10:10:01.442 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-20T10:10:01.443 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-20T10:10:01.443 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-20T10:10:01.443 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-20T10:10:01.443 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-20T10:10:01.443 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-20T10:10:01.443 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-20T10:10:01.443 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-20T10:10:01.443 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-20T10:10:01.443 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-20T10:10:01.443 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-20T10:10:01.443 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-20T10:10:01.443 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-20T10:10:01.443 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-20T10:10:01.443 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-20T10:10:01.444 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-20T10:10:01.444 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-20T10:10:01.444 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-20T10:10:01.444 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-20T10:10:01.444 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-20T10:10:01.444 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-20T10:10:01.444 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-20T10:10:01.444 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-20T10:10:01.444 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-20T10:10:01.444 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-20T10:10:01.444 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-20T10:10:01.445 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-20T10:10:01.445 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-20T10:10:01.445 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-20T10:10:01.445 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-20T10:10:01.445 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-20T10:10:01.445 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-20T10:10:01.445 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-20T10:10:01.445 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-20T10:10:01.445 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-20T10:10:01.445 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-20T10:10:01.445 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-20T10:10:01.445 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-20T10:10:01.445 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-20T10:10:01.445 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-20T10:10:01.446 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-20T10:10:01.446 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-20T10:10:01.446 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-20T10:10:01.446 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-20T10:10:01.446 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-20T10:10:01.446 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-20T10:10:01.446 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-20T10:10:01.446 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-20T10:10:01.446 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-20T10:10:01.446 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-20T10:10:01.446 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-20T10:10:01.446 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-20T10:10:01.446 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-20T10:10:01.446 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-20T10:10:01.447 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-20T10:10:01.447 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-20T10:10:01.447 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-20T10:10:01.447 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-20T10:10:01.447 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-20T10:10:01.447 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-20T10:10:01.447 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-20T10:10:01.447 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-20T10:10:01.447 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-20T10:10:01.447 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-20T10:10:01.447 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-20T10:10:01.448 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-20T10:10:01.448 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-20T10:10:01.448 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-20T10:10:01.448 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-20T10:10:01.448 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-20T10:10:01.448 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-20T10:10:01.448 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-20T10:10:01.448 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-20T10:10:01.449 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-20T10:10:01.449 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-20T10:10:01.449 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-20T10:10:01.449 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-20T10:10:01.449 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-20T10:10:01.449 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-20T10:10:01.449 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-20T10:10:01.449 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-20T10:10:01.449 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-20T10:10:01.449 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-20T10:10:01.449 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-20T10:10:01.450 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-20T10:10:01.450 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-20T10:10:01.450 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-20T10:10:01.450 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-20T10:10:01.450 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-20T10:10:01.450 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-20T10:10:01.450 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-20T10:10:01.450 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-20T10:10:01.450 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-20T10:10:01.450 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-20T10:10:01.450 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-20T10:10:01.450 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-20T10:10:01.450 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-20T10:10:01.451 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-20T10:10:01.451 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-20T10:10:01.451 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-20T10:10:01.451 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-20T10:10:01.451 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-20T10:10:01.451 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-20T10:10:01.451 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-20T10:10:01.451 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-20T10:10:01.451 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-20T10:10:01.451 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-20T10:10:01.451 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-20T10:10:01.451 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-20T10:10:01.451 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-20T10:10:01.452 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-20T10:10:01.452 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-20T10:10:01.452 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-20T10:10:01.452 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-20T10:10:01.452 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-20T10:10:01.452 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-20T10:10:01.452 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-20T10:10:01.452 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-20T10:10:01.452 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-20T10:10:01.452 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-20T10:10:01.452 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-20T10:10:01.453 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-20T10:10:01.453 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-20T10:10:01.453 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-20T10:10:01.453 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-20T10:10:01.453 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-20T10:10:01.453 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-20T10:10:01.453 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-20T10:10:01.453 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-20T10:10:01.454 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-20T10:10:01.454 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-20T10:10:01.454 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-20T10:10:01.454 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-20T10:10:01.454 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-20T10:10:01.454 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-20T10:10:01.454 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-20T10:10:01.454 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-20T10:10:01.454 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-20T10:10:01.454 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-20T10:10:01.454 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-20T10:10:01.454 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-20T10:10:01.454 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-20T10:10:01.455 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-20T10:10:01.455 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-20T10:10:01.455 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-20T10:10:01.455 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-20T10:10:01.455 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-20T10:10:01.455 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-20T10:10:01.455 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-20T10:10:01.455 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-20T10:10:01.455 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-20T10:10:01.455 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-20T10:10:01.455 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-20T10:10:01.455 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-20T10:10:01.455 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-20T10:10:01.455 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-20T10:10:01.456 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-20T10:10:01.456 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-20T10:10:01.456 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-20T10:10:01.456 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-20T10:10:01.456 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-20T10:10:01.456 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-20T10:10:01.456 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-20T10:10:01.456 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-20T10:10:01.456 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-20T10:10:01.456 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-20T10:10:01.456 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-20T10:10:01.456 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-20T10:10:01.456 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-20T10:10:01.456 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-20T10:10:01.456 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-20T10:10:01.457 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-20T10:10:01.457 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-20T10:10:01.457 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-20T10:10:01.457 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-20T10:10:01.457 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-20T10:10:01.457 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-20T10:10:01.457 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-20T10:10:01.457 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-20T10:10:01.457 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-20T10:10:01.457 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-20T10:10:01.457 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-20T10:10:01.457 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-20T10:10:01.457 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-20T10:10:01.457 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-20T10:10:01.458 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-20T10:10:01.458 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-20T10:10:01.458 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-20T10:10:01.458 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-20T10:10:01.458 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-20T10:10:01.458 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-20T10:10:01.458 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-20T10:10:01.458 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-20T10:10:01.458 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-20T10:10:01.458 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-20T10:10:01.458 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-20T10:10:01.458 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-20T10:10:01.458 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-20T10:10:01.459 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-20T10:10:01.459 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-20T10:10:01.459 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-20T10:10:01.459 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-20T10:10:01.459 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-20T10:10:01.459 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-20T10:10:01.459 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-20T10:10:01.459 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-20T10:10:01.459 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-20T10:10:01.459 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-20T10:10:01.459 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-20T10:10:01.459 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-20T10:10:01.460 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-20T10:10:01.460 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-20T10:10:01.460 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-20T10:10:01.460 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-20T10:10:01.460 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-20T10:10:01.460 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-20T10:10:01.460 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-20T10:10:01.460 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-20T10:10:01.460 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-20T10:10:01.460 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-20T10:10:01.460 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-20T10:10:01.461 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-20T10:10:01.461 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-20T10:10:01.461 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-20T10:10:01.461 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-20T10:10:01.461 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-20T10:10:01.461 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-20T10:10:01.461 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-20T10:10:01.461 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-20T10:10:01.461 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-20T10:10:01.462 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-20T10:10:01.462 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-20T10:10:01.462 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-20T10:10:01.462 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-20T10:10:01.462 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-20T10:10:01.462 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-20T10:10:01.462 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-20T10:10:01.462 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-20T10:10:01.462 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-20T10:10:01.462 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-20T10:10:01.463 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-20T10:10:01.463 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-20T10:10:01.463 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-20T10:10:01.463 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-20T10:10:01.463 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-20T10:10:01.463 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-20T10:10:01.463 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-20T10:10:01.463 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-20T10:10:01.463 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-20T10:10:01.463 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-20T10:10:01.463 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-20T10:10:01.463 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-20T10:10:01.463 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-20T10:10:01.463 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-20T10:10:01.463 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-20T10:10:01.464 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-20T10:10:01.464 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-20T10:10:01.464 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-20T10:10:01.464 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-20T10:10:01.464 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-20T10:10:01.464 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-20T10:10:01.464 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-20T10:10:01.464 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-20T10:10:01.464 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-20T10:10:01.464 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-20T10:10:01.464 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-20T10:10:01.464 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-20T10:10:01.464 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-20T10:10:01.464 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-20T10:10:01.465 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-20T10:10:01.465 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-20T10:10:01.465 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-20T10:10:01.465 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-20T10:10:01.465 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-20T10:10:01.465 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-20T10:10:01.465 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-20T10:10:01.465 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-20T10:10:01.465 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-20T10:10:01.465 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-20T10:10:01.465 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-20T10:10:01.465 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-20T10:10:01.465 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-20T10:10:01.465 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-20T10:10:01.466 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-20T10:10:01.466 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-20T10:10:01.466 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-20T10:10:01.466 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-20T10:10:01.466 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-20T10:10:01.466 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-20T10:10:01.466 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-20T10:10:01.466 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-20T10:10:01.466 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-20T10:10:01.466 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-20T10:10:01.466 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-20T10:10:01.466 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-20T10:10:01.466 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-20T10:10:01.467 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-20T10:10:01.467 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-20T10:10:01.467 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-20T10:10:01.467 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-20T10:10:01.467 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-20T10:10:01.467 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-20T10:10:01.467 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-20T10:10:01.467 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-20T10:10:01.467 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-20T10:10:01.467 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-20T10:10:01.467 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-20T10:10:01.468 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-20T10:10:01.468 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-20T10:10:01.468 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-20T10:10:01.468 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-20T10:10:01.468 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-20T10:10:01.468 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-20T10:10:01.468 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-20T10:10:01.468 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-20T10:10:01.468 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-20T10:10:01.468 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-20T10:10:01.468 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-20T10:10:01.468 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-20T10:10:01.469 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-20T10:10:01.469 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-20T10:10:01.469 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-20T10:10:01.469 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-20T10:10:01.469 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-20T10:10:01.469 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-20T10:10:01.469 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-20T10:10:01.469 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-20T10:10:01.469 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-20T10:10:01.469 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-20T10:10:01.469 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-20T10:10:01.469 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-20T10:10:01.469 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-20T10:10:01.469 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-20T10:10:01.470 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-20T10:10:01.470 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-20T10:10:01.470 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-20T10:10:01.470 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-20T10:10:01.470 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-20T10:10:01.470 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-20T10:10:01.470 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-20T10:10:01.470 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-20T10:10:01.470 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-20T10:10:01.470 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-20T10:10:01.470 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-20T10:10:01.470 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-20T10:10:01.470 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-20T10:10:01.470 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-20T10:10:01.471 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-20T10:10:01.471 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-20T10:10:01.471 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-20T10:10:01.471 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-20T10:10:01.471 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-20T10:10:01.471 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-20T10:10:01.471 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-20T10:10:01.471 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-20T10:10:01.471 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-20T10:10:01.471 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-20T10:10:01.472 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-20T10:10:01.472 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-20T10:10:01.472 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-20T10:10:01.472 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-20T10:10:01.472 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-20T10:10:01.472 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-20T10:10:01.472 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-20T10:10:01.472 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-20T10:10:01.472 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-20T10:10:01.472 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-20T10:10:01.472 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-20T10:10:01.472 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-20T10:10:01.473 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-20T10:10:01.473 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-20T10:10:01.473 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-20T10:10:01.473 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-20T10:10:01.473 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-20T10:10:01.473 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-20T10:10:01.473 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-20T10:10:01.473 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-20T10:10:01.473 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-20T10:10:01.473 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-20T10:10:01.473 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-20T10:10:01.473 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-20T10:10:01.473 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-20T10:10:01.474 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-20T10:10:01.474 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-20T10:10:01.474 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-20T10:10:01.474 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-20T10:10:01.474 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-20T10:10:01.474 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-20T10:10:01.474 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-20T10:10:01.474 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-20T10:10:01.474 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-20T10:10:01.474 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-20T10:10:01.474 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-20T10:10:01.474 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-20T10:10:01.474 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-20T10:10:01.475 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-20T10:10:01.475 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-20T10:10:01.475 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-20T10:10:01.475 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-20T10:10:01.475 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-20T10:10:01.475 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-20T10:10:01.475 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-20T10:10:01.475 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-20T10:10:01.475 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-20T10:10:01.475 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-20T10:10:01.475 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-20T10:10:01.475 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-20T10:10:01.475 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-20T10:10:01.475 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-20T10:10:01.476 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-20T10:10:01.476 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-20T10:10:01.476 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-20T10:10:01.476 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-20T10:10:01.476 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-20T10:10:01.476 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-20T10:10:01.476 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-20T10:10:01.476 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-20T10:10:01.476 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-20T10:10:01.476 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-20T10:10:01.477 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-20T10:10:01.477 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-20T10:10:01.477 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-20T10:10:01.477 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-20T10:10:01.477 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-20T10:10:01.477 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-20T10:10:01.477 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-20T10:10:01.477 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-20T10:10:01.477 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-20T10:10:01.477 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-20T10:10:01.477 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-20T10:10:01.477 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-20T10:10:01.477 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-20T10:10:01.477 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-20T10:10:01.477 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-20T10:10:01.478 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-20T10:10:01.478 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-20T10:10:01.478 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-20T10:10:01.478 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-20T10:10:01.478 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-20T10:10:01.478 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-20T10:10:01.478 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-20T10:10:01.479 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-20T10:10:01.479 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-20T10:10:01.479 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-20T10:10:01.479 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-20T10:10:01.479 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-20T10:10:01.479 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-20T10:10:01.479 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-20T10:10:01.479 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-20T10:10:01.479 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-20T10:10:01.479 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-20T10:10:01.479 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-20T10:10:01.479 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-20T10:10:01.479 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-20T10:10:01.480 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-20T10:10:01.480 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-20T10:10:01.480 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-20T10:10:01.480 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-20T10:10:01.480 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-20T10:10:01.480 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-20T10:10:01.480 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-20T10:10:01.480 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-20T10:10:01.480 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-20T10:10:01.480 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-20T10:10:01.481 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-20T10:10:01.481 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-20T10:10:01.481 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-20T10:10:01.481 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-20T10:10:01.481 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-20T10:10:01.481 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-20T10:10:01.481 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-20T10:10:01.481 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-20T10:10:01.481 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-20T10:10:01.481 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-20T10:10:01.481 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-20T10:10:01.481 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-20T10:10:01.481 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-20T10:10:01.481 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-20T10:10:01.481 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-20T10:10:01.482 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-20T10:10:01.482 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-20T10:10:01.482 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-20T10:10:01.482 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-20T10:10:01.482 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-20T10:10:01.482 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-20T10:10:01.482 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-20T10:10:01.482 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-20T10:10:01.482 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-20T10:10:01.482 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-20T10:10:01.482 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-20T10:10:01.482 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-20T10:10:01.483 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-20T10:10:01.483 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-20T10:10:01.483 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-20T10:10:01.483 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-20T10:10:01.483 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-20T10:10:01.483 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-20T10:10:01.483 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-20T10:10:01.483 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-20T10:10:01.483 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-20T10:10:01.483 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-20T10:10:01.483 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-20T10:10:01.483 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-20T10:10:01.484 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-20T10:10:01.484 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-20T10:10:01.484 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-20T10:10:01.484 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-20T10:10:01.484 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-20T10:10:01.484 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-20T10:10:01.484 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-20T10:10:01.484 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-20T10:10:01.484 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-20T10:10:01.484 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-20T10:10:01.484 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-20T10:10:01.484 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-20T10:10:01.484 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-20T10:10:01.484 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-20T10:10:01.485 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-20T10:10:01.485 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-20T10:10:01.485 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-20T10:10:01.485 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-20T10:10:01.485 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-20T10:10:01.485 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-20T10:10:01.485 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-20T10:10:01.485 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-20T10:10:01.485 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-20T10:10:01.485 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-20T10:10:01.485 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-20T10:10:01.485 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-20T10:10:01.485 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-20T10:10:01.485 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-20T10:10:01.485 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-20T10:10:01.486 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-20T10:10:01.486 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-20T10:10:01.486 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-20T10:10:01.486 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-20T10:10:01.486 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-20T10:10:01.486 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-20T10:10:01.486 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-20T10:10:01.486 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-20T10:10:01.486 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-20T10:10:01.486 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-20T10:10:01.486 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-20T10:10:01.486 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-20T10:10:01.486 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-20T10:10:01.487 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-20T10:10:01.487 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-20T10:10:01.487 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-20T10:10:01.487 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-20T10:10:01.487 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-20T10:10:01.487 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-20T10:10:01.487 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-20T10:10:01.487 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-20T10:10:01.487 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-20T10:10:01.487 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-20T10:10:01.487 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-20T10:10:01.487 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-20T10:10:01.487 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-20T10:10:01.488 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-20T10:10:01.488 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-20T10:10:01.488 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-20T10:10:01.488 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-20T10:10:01.488 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-20T10:10:01.488 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-20T10:10:01.488 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-20T10:10:01.488 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-20T10:10:01.488 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-20T10:10:01.488 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-20T10:10:01.488 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-20T10:10:01.488 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-20T10:10:01.488 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-20T10:10:01.488 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-20T10:10:01.489 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-20T10:10:01.489 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-20T10:10:01.489 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-20T10:10:01.489 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-20T10:10:01.489 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-20T10:10:01.489 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-20T10:10:01.489 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-20T10:10:01.489 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-20T10:10:01.489 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-20T10:10:01.489 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-20T10:10:01.490 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-20T10:10:01.490 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-20T10:10:01.490 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-20T10:10:01.490 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-20T10:10:01.490 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-20T10:10:01.490 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-20T10:10:01.490 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-20T10:10:01.490 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-20T10:10:01.490 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-20T10:10:01.490 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-20T10:10:01.490 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-20T10:10:01.491 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-20T10:10:01.491 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-20T10:10:01.491 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-20T10:10:01.491 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-20T10:10:01.491 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-20T10:10:01.491 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-20T10:10:01.491 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-20T10:10:01.491 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-20T10:10:01.491 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-20T10:10:01.491 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-20T10:10:01.491 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-20T10:10:01.491 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-20T10:10:01.491 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-20T10:10:01.491 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-20T10:10:01.492 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-20T10:10:01.492 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-20T10:10:01.492 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-20T10:10:01.492 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-20T10:10:01.492 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-20T10:10:01.492 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-20T10:10:01.492 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-20T10:10:01.492 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-20T10:10:01.492 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-20T10:10:01.492 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-20T10:10:01.492 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-20T10:10:01.492 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-20T10:10:01.492 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-20T10:10:01.492 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-20T10:10:01.492 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-20T10:10:01.493 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-20T10:10:01.493 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-20T10:10:01.493 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-20T10:10:01.493 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-20T10:10:01.493 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-20T10:10:01.493 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-20T10:10:01.493 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-20T10:10:01.493 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-20T10:10:01.493 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-20T10:10:01.493 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-20T10:10:01.493 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-20T10:10:01.494 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-20T10:10:01.494 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-20T10:10:01.494 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-20T10:10:01.494 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-20T10:10:01.494 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-20T10:10:01.494 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-20T10:10:01.494 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-20T10:10:01.494 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-20T10:10:01.494 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-20T10:10:01.495 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-20T10:10:01.495 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-20T10:10:01.495 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-20T10:10:01.495 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-20T10:10:01.495 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-20T10:10:01.495 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-20T10:10:01.495 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-20T10:10:01.495 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-20T10:10:01.495 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-20T10:10:01.495 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-20T10:10:01.495 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-20T10:10:01.495 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-20T10:10:01.495 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-20T10:10:01.495 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-20T10:10:01.495 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-20T10:10:01.496 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-20T10:10:01.496 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-20T10:10:01.496 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-20T10:10:01.496 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-20T10:10:01.496 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-20T10:10:01.496 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-20T10:10:01.496 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-20T10:10:01.496 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-20T10:10:01.496 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-20T10:10:01.496 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-20T10:10:01.496 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-20T10:10:01.496 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-20T10:10:01.496 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-20T10:10:01.496 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-20T10:10:01.496 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-20T10:10:01.497 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-20T10:10:01.497 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-20T10:10:01.497 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-20T10:10:01.497 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-20T10:10:01.497 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-20T10:10:01.497 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-20T10:10:01.497 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-20T10:10:01.497 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-20T10:10:01.497 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-20T10:10:01.497 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-20T10:10:01.497 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-20T10:10:01.497 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-20T10:10:01.497 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-20T10:10:01.497 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-20T10:10:01.497 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-20T10:10:01.498 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-20T10:10:01.498 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-20T10:10:01.498 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-20T10:10:01.498 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-20T10:10:01.498 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-20T10:10:01.498 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-20T10:10:01.498 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-20T10:10:01.498 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-20T10:10:01.498 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-20T10:10:01.498 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-20T10:10:01.498 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-20T10:10:01.498 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-20T10:10:01.498 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-20T10:10:01.499 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-20T10:10:01.499 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-20T10:10:01.499 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-20T10:10:01.499 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-20T10:10:01.499 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-20T10:10:01.499 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-20T10:10:01.499 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-20T10:10:01.499 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-20T10:10:01.500 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-20T10:10:01.500 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-20T10:10:01.500 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-20T10:10:01.500 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-20T10:10:01.500 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-20T10:10:01.500 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-20T10:10:01.500 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-20T10:10:01.500 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-20T10:10:01.500 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-20T10:10:01.500 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-20T10:10:01.500 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-20T10:10:01.501 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-20T10:10:01.501 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-20T10:10:01.501 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-20T10:10:01.501 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-20T10:10:01.501 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-20T10:10:01.501 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-20T10:10:01.501 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-20T10:10:01.501 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-20T10:10:01.501 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-20T10:10:01.501 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-20T10:10:01.501 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-20T10:10:01.501 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-20T10:10:01.501 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-20T10:10:01.502 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-20T10:10:01.502 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-20T10:10:01.502 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-20T10:10:01.502 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-20T10:10:01.502 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-20T10:10:01.502 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-20T10:10:01.502 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-20T10:10:01.502 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-20T10:10:01.502 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-20T10:10:01.502 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-20T10:10:01.502 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-20T10:10:01.502 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-20T10:10:01.502 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-20T10:10:01.502 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-20T10:10:01.503 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-20T10:10:01.503 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-20T10:10:01.503 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-20T10:10:01.503 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-20T10:10:01.503 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-20T10:10:01.503 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-20T10:10:01.503 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-20T10:10:01.503 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-20T10:10:01.503 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-20T10:10:01.503 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-20T10:10:01.503 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-20T10:10:01.504 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-20T10:10:01.504 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-20T10:10:01.504 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-20T10:10:01.504 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-20T10:10:01.504 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-20T10:10:01.504 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-20T10:10:01.505 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-20T10:10:01.505 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-20T10:10:01.505 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-20T10:10:01.505 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-20T10:10:01.505 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-20T10:10:01.505 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-20T10:10:01.505 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-20T10:10:01.506 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-20T10:10:01.506 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-20T10:10:01.506 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-20T10:10:01.506 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-20T10:10:01.506 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-20T10:10:01.506 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-20T10:10:01.507 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-20T10:10:01.507 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-20T10:10:01.507 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-20T10:10:01.507 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-20T10:10:01.507 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-20T10:10:01.507 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-20T10:10:01.508 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-20T10:10:01.508 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-20T10:10:01.508 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-20T10:10:01.508 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-20T10:10:01.508 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-20T10:10:01.508 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-20T10:10:01.509 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-20T10:10:01.509 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-20T10:10:01.509 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-20T10:10:01.509 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-20T10:10:01.509 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-20T10:10:01.509 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-20T10:10:01.509 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-20T10:10:01.510 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-20T10:10:01.510 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-20T10:10:01.510 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-20T10:10:01.510 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-20T10:10:01.510 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-20T10:10:01.510 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-20T10:10:01.511 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-20T10:10:01.511 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-20T10:10:01.511 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-20T10:10:01.511 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-20T10:10:01.511 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-20T10:10:01.511 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-20T10:10:01.512 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-20T10:10:01.512 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-20T10:10:01.512 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-20T10:10:01.512 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-20T10:10:01.512 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-20T10:10:01.512 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-20T10:10:01.513 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-20T10:10:01.513 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-20T10:10:01.513 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-20T10:10:01.513 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-20T10:10:01.513 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-20T10:10:01.513 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-20T10:10:01.514 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-20T10:10:01.514 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-20T10:10:01.514 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-20T10:10:01.514 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-20T10:10:01.514 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-20T10:10:01.514 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-20T10:10:01.515 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-20T10:10:01.515 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-20T10:10:01.515 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-20T10:10:01.515 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-20T10:10:01.515 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-20T10:10:01.516 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-20T10:10:01.516 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-20T10:10:01.516 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-20T10:10:01.516 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-20T10:10:01.516 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-20T10:10:01.516 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-20T10:10:01.517 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-20T10:10:01.517 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-20T10:10:01.517 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-20T10:10:01.517 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-20T10:10:01.517 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-20T10:10:01.517 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-20T10:10:01.518 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-20T10:10:01.518 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-20T10:10:01.518 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-20T10:10:01.518 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-20T10:10:01.518 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-20T10:10:01.518 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-20T10:10:01.519 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-20T10:10:01.519 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-20T10:10:01.519 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-20T10:10:01.519 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-20T10:10:01.519 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-20T10:10:01.519 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-20T10:10:01.520 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-20T10:10:01.520 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-20T10:10:01.520 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-20T10:10:01.520 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-20T10:10:01.520 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-20T10:10:01.520 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-20T10:10:01.521 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-20T10:10:01.521 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-20T10:10:01.521 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-20T10:10:01.521 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-20T10:10:01.521 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-20T10:10:01.521 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-20T10:10:01.522 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-20T10:10:01.522 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-20T10:10:01.522 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-20T10:10:01.522 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-20T10:10:01.522 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-20T10:10:01.523 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-20T10:10:01.523 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-20T10:10:01.523 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-20T10:10:01.523 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-20T10:10:01.523 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-20T10:10:01.523 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-20T10:10:01.524 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-20T10:10:01.524 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-20T10:10:01.524 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-20T10:10:01.524 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-20T10:10:01.524 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-20T10:10:01.524 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-20T10:10:01.525 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-20T10:10:01.525 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-20T10:10:01.525 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-20T10:10:01.525 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-20T10:10:01.525 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-20T10:10:01.525 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-20T10:10:01.526 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-20T10:10:01.526 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-20T10:10:01.526 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-20T10:10:01.526 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-20T10:10:01.526 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-20T10:10:01.527 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-20T10:10:01.527 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-20T10:10:01.527 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-20T10:10:01.527 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-20T10:10:01.527 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-20T10:10:01.527 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-20T10:10:01.528 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-20T10:10:01.528 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-20T10:10:01.528 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-20T10:10:01.528 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-20T10:10:01.528 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-20T10:10:01.529 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-20T10:10:01.529 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-20T10:10:01.529 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-20T10:10:01.529 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-20T10:10:01.529 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-20T10:10:01.529 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-20T10:10:01.530 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-20T10:10:01.530 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-20T10:10:01.530 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-20T10:10:01.530 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-20T10:10:01.530 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-20T10:10:01.530 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-20T10:10:01.531 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-20T10:10:01.531 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-20T10:10:01.531 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-20T10:10:01.531 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-20T10:10:01.531 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-20T10:10:01.531 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-20T10:10:01.532 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-20T10:10:01.532 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-20T10:10:01.532 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-20T10:10:01.532 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-20T10:10:01.532 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-20T10:10:01.532 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-20T10:10:01.533 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-20T10:10:01.533 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-20T10:10:01.533 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-20T10:10:01.533 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-20T10:10:01.533 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-20T10:10:01.533 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-20T10:10:01.534 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-20T10:10:01.534 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-20T10:10:01.534 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-20T10:10:01.534 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-20T10:10:01.534 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-20T10:10:01.535 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-20T10:10:01.535 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-20T10:10:01.535 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-20T10:10:01.535 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-20T10:10:01.535 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-20T10:10:01.535 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-20T10:10:01.535 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-20T10:10:01.536 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-20T10:10:01.536 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-20T10:10:01.536 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-20T10:10:01.536 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-20T10:10:01.536 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-20T10:10:01.536 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-20T10:10:01.536 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-20T10:10:01.536 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-20T10:10:01.536 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-20T10:10:01.536 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-20T10:10:01.537 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-20T10:10:01.537 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-20T10:10:01.537 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-20T10:10:01.537 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-20T10:10:01.537 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-20T10:10:01.537 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-20T10:10:01.537 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-20T10:10:01.537 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-20T10:10:01.537 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-20T10:10:01.538 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-20T10:10:01.538 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-20T10:10:01.538 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-20T10:10:01.538 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-20T10:10:01.538 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-20T10:10:01.538 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-20T10:10:01.538 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-20T10:10:01.538 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-20T10:10:01.538 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-20T10:10:01.538 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-20T10:10:01.539 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-20T10:10:01.539 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-20T10:10:01.539 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-20T10:10:01.539 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-20T10:10:01.539 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-20T10:10:01.539 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-20T10:10:01.539 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-20T10:10:01.539 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-20T10:10:01.539 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-20T10:10:01.540 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-20T10:10:01.540 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-20T10:10:01.540 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-20T10:10:01.540 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-20T10:10:01.540 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-20T10:10:01.540 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-20T10:10:01.540 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-20T10:10:01.540 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-20T10:10:01.540 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-20T10:10:01.540 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-20T10:10:01.540 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-20T10:10:01.541 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-20T10:10:01.541 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-20T10:10:01.541 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-20T10:10:01.541 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-20T10:10:01.541 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-20T10:10:01.541 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-20T10:10:01.541 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-20T10:10:01.541 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-20T10:10:01.541 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-20T10:10:01.542 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-20T10:10:01.542 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-20T10:10:01.542 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-20T10:10:01.542 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-20T10:10:01.542 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-20T10:10:01.542 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-20T10:10:01.542 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-20T10:10:01.542 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-20T10:10:01.543 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-20T10:10:01.543 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-20T10:10:01.543 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-20T10:10:01.543 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-20T10:10:01.543 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-20T10:10:01.543 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-20T10:10:01.543 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-20T10:10:01.543 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-20T10:10:01.543 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-20T10:10:01.543 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-20T10:10:01.544 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-20T10:10:01.544 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-20T10:10:01.544 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-20T10:10:01.544 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-20T10:10:01.544 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-20T10:10:01.544 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-20T10:10:01.544 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-20T10:10:01.544 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-20T10:10:01.544 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-20T10:10:01.544 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-20T10:10:01.545 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-20T10:10:01.545 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-20T10:10:01.545 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-20T10:10:01.545 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-20T10:10:01.545 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-20T10:10:01.545 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-20T10:10:01.545 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-20T10:10:01.545 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-20T10:10:01.545 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-20T10:10:01.546 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-20T10:10:01.546 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-20T10:10:01.546 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-20T10:10:01.546 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-20T10:10:01.546 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-20T10:10:01.546 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-20T10:10:01.546 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-20T10:10:01.546 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-20T10:10:01.546 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-20T10:10:01.547 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-20T10:10:01.547 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-20T10:10:01.547 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-20T10:10:01.547 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-20T10:10:01.547 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-20T10:10:01.547 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-20T10:10:01.547 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-20T10:10:01.549 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-20T10:10:01.549 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-20T10:10:01.549 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-20T10:10:01.549 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-20T10:10:01.550 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-20T10:10:01.550 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-20T10:10:01.550 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-20T10:10:01.550 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-20T10:10:01.550 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-20T10:10:01.550 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-20T10:10:01.551 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-20T10:10:01.551 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-20T10:10:01.551 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-20T10:10:01.551 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-20T10:10:01.551 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-20T10:10:01.551 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-20T10:10:01.552 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-20T10:10:01.552 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-20T10:10:01.552 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-20T10:10:01.552 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-20T10:10:01.552 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-20T10:10:01.552 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-20T10:10:01.553 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-20T10:10:01.553 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-20T10:10:01.553 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-20T10:10:01.553 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-20T10:10:01.553 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-20T10:10:01.553 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-20T10:10:01.554 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-20T10:10:01.554 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-20T10:10:01.554 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-20T10:10:01.554 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-20T10:10:01.554 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-20T10:10:01.555 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-20T10:10:01.555 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-20T10:10:01.555 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-20T10:10:01.555 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-20T10:10:01.555 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-20T10:10:01.555 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-20T10:10:01.555 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-20T10:10:01.556 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-20T10:10:01.556 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-20T10:10:01.556 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-20T10:10:01.556 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-20T10:10:01.556 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-20T10:10:01.556 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-20T10:10:01.557 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-20T10:10:01.557 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-20T10:10:01.557 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-20T10:10:01.557 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-20T10:10:01.557 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-20T10:10:01.557 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-20T10:10:01.557 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-20T10:10:01.557 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-20T10:10:01.558 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-20T10:10:01.558 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-20T10:10:01.558 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-20T10:10:01.558 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-20T10:10:01.558 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-20T10:10:01.558 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-20T10:10:01.558 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-20T10:10:01.558 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-20T10:10:01.558 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-20T10:10:01.559 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-20T10:10:01.559 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-20T10:10:01.559 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-20T10:10:01.559 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-20T10:10:01.559 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-20T10:10:01.559 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-20T10:10:01.559 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-20T10:10:01.559 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-20T10:10:01.560 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-20T10:10:01.560 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-20T10:10:01.560 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-20T10:10:01.560 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-20T10:10:01.560 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-20T10:10:01.560 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-20T10:10:01.560 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-20T10:10:01.560 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-20T10:10:01.561 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-20T10:10:01.561 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-20T10:10:01.561 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-20T10:10:01.561 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-20T10:10:01.561 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-20T10:10:01.561 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-20T10:10:01.561 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-20T10:10:01.561 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-20T10:10:01.562 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-20T10:10:01.562 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-20T10:10:01.562 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-20T10:10:01.562 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-20T10:10:01.562 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-20T10:10:01.562 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-20T10:10:01.562 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-20T10:10:01.563 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-20T10:10:01.563 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-20T10:10:01.563 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-20T10:10:01.563 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-20T10:10:01.563 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-20T10:10:01.563 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-20T10:10:01.563 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-20T10:10:01.563 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-20T10:10:01.564 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-20T10:10:01.564 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-20T10:10:01.564 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-20T10:10:01.564 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-20T10:10:01.564 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-20T10:10:01.564 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-20T10:10:01.564 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-20T10:10:01.564 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-20T10:10:01.565 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-20T10:10:01.565 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-20T10:10:01.565 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-20T10:10:01.565 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-20T10:10:01.565 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-20T10:10:01.565 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-20T10:10:01.565 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-20T10:10:01.565 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-20T10:10:01.565 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-20T10:10:01.565 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-20T10:10:01.565 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-20T10:10:01.566 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-20T10:10:01.566 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-20T10:10:01.566 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-20T10:10:01.566 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-20T10:10:01.566 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-20T10:10:01.566 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-20T10:10:01.567 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-20T10:10:01.567 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-20T10:10:01.567 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-20T10:10:01.567 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-20T10:10:01.567 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-20T10:10:01.567 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-20T10:10:01.568 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-20T10:10:01.568 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-20T10:10:01.568 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-20T10:10:01.568 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-20T10:10:01.568 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-20T10:10:01.568 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-20T10:10:01.569 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-20T10:10:01.569 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-20T10:10:01.569 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-20T10:10:01.569 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-20T10:10:01.569 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-20T10:10:01.569 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-20T10:10:01.569 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-20T10:10:01.570 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-20T10:10:01.570 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-20T10:10:01.570 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-20T10:10:01.570 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-20T10:10:01.570 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-20T10:10:01.570 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-20T10:10:01.570 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-20T10:10:01.570 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-20T10:10:01.570 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-20T10:10:01.570 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-20T10:10:01.570 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-20T10:10:01.570 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-20T10:10:01.570 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-20T10:10:01.571 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-20T10:10:01.571 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-20T10:10:01.571 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-20T10:10:01.571 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-20T10:10:01.571 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-20T10:10:01.571 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-20T10:10:01.571 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-20T10:10:01.571 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-20T10:10:01.571 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-20T10:10:01.571 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-20T10:10:01.571 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-20T10:10:01.571 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-20T10:10:01.571 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-20T10:10:01.571 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-20T10:10:01.571 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-20T10:10:01.572 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-20T10:10:01.572 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-20T10:10:01.572 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-20T10:10:01.572 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-20T10:10:01.572 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-20T10:10:01.572 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-20T10:10:01.572 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-20T10:10:01.572 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-20T10:10:01.572 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-20T10:10:01.572 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-20T10:10:01.572 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-20T10:10:01.572 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-20T10:10:01.572 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-20T10:10:01.572 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-20T10:10:01.573 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-20T10:10:01.573 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-20T10:10:01.573 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-20T10:10:01.573 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-20T10:10:01.573 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-20T10:10:01.573 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-20T10:10:01.573 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-20T10:10:01.573 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-20T10:10:01.573 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-20T10:10:01.573 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-20T10:10:01.573 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-20T10:10:01.573 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-20T10:10:01.573 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-20T10:10:01.573 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-20T10:10:01.574 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-20T10:10:01.574 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-20T10:10:01.574 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-20T10:10:01.574 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-20T10:10:01.574 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-20T10:10:01.574 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-20T10:10:01.574 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-20T10:10:01.574 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-20T10:10:01.574 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-20T10:10:01.574 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-20T10:10:01.574 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-20T10:10:01.574 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-20T10:10:01.574 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-20T10:10:01.575 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-20T10:10:01.575 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-20T10:10:01.575 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-20T10:10:01.575 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-20T10:10:01.575 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-20T10:10:01.575 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-20T10:10:01.575 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-20T10:10:01.575 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-20T10:10:01.575 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-20T10:10:01.575 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-20T10:10:01.575 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-20T10:10:01.575 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-20T10:10:01.575 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-20T10:10:01.575 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-20T10:10:01.575 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-20T10:10:01.576 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-20T10:10:01.576 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-20T10:10:01.576 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-20T10:10:01.576 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-20T10:10:01.576 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-20T10:10:01.576 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-20T10:10:01.576 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-20T10:10:01.576 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-20T10:10:01.576 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-20T10:10:01.576 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-20T10:10:01.576 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-20T10:10:01.576 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-20T10:10:01.576 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-20T10:10:01.576 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-20T10:10:01.576 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-20T10:10:01.577 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-20T10:10:01.577 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-20T10:10:01.577 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-20T10:10:01.577 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-20T10:10:01.577 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-20T10:10:01.577 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-20T10:10:01.577 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-20T10:10:01.577 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-20T10:10:01.577 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-20T10:10:01.577 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-20T10:10:01.577 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-20T10:10:01.577 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-20T10:10:01.577 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-20T10:10:01.577 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-20T10:10:01.578 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-20T10:10:01.578 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-20T10:10:01.578 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-20T10:10:01.578 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-20T10:10:01.578 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-20T10:10:01.578 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-20T10:10:01.578 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-20T10:10:01.578 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-20T10:10:01.578 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-20T10:10:01.578 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-20T10:10:01.578 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-20T10:10:01.578 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-20T10:10:01.578 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-20T10:10:01.578 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-20T10:10:01.579 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-20T10:10:01.579 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-20T10:10:01.579 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-20T10:10:01.579 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-20T10:10:01.579 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-20T10:10:01.579 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-20T10:10:01.579 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-20T10:10:01.579 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-20T10:10:01.579 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-20T10:10:01.579 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-20T10:10:01.579 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-20T10:10:01.579 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-20T10:10:01.579 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-20T10:10:01.579 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-20T10:10:01.579 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-20T10:10:01.580 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-20T10:10:01.580 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-20T10:10:01.580 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-20T10:10:01.580 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-20T10:10:01.580 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-20T10:10:01.580 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-20T10:10:01.580 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-20T10:10:01.580 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-20T10:10:01.580 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-20T10:10:01.580 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-20T10:10:01.580 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-20T10:10:01.580 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-20T10:10:01.580 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-20T10:10:01.580 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-20T10:10:01.581 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-20T10:10:01.581 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-20T10:10:01.581 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-20T10:10:01.581 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-20T10:10:01.581 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-20T10:10:01.581 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-20T10:10:01.581 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-20T10:10:01.581 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-20T10:10:01.581 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-20T10:10:01.581 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-20T10:10:01.581 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-20T10:10:01.581 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-20T10:10:01.581 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-20T10:10:01.581 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-20T10:10:01.581 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-20T10:10:01.582 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-20T10:10:01.582 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-20T10:10:01.582 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-20T10:10:01.582 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-20T10:10:01.582 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-20T10:10:01.582 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-20T10:10:01.582 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-20T10:10:01.582 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-20T10:10:01.582 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-20T10:10:01.582 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-20T10:10:01.582 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-20T10:10:01.582 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-20T10:10:01.582 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-20T10:10:01.582 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-20T10:10:01.583 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-20T10:10:01.583 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-20T10:10:01.583 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-20T10:10:01.583 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-20T10:10:01.583 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-20T10:10:01.583 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-20T10:10:01.583 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-20T10:10:01.583 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-20T10:10:01.583 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-20T10:10:01.583 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-20T10:10:01.583 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-20T10:10:01.583 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-20T10:10:01.584 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-20T10:10:01.584 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-20T10:10:01.584 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-20T10:10:01.584 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-20T10:10:01.584 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-20T10:10:01.584 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-20T10:10:01.584 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-20T10:10:01.584 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-20T10:10:01.584 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-20T10:10:01.584 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-20T10:10:01.584 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-20T10:10:01.584 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-20T10:10:01.584 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-20T10:10:01.584 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-20T10:10:01.584 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-20T10:10:01.585 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-20T10:10:01.585 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-20T10:10:01.585 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-20T10:10:01.585 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-20T10:10:01.585 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-20T10:10:01.585 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-20T10:10:01.585 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-20T10:10:01.585 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-20T10:10:01.585 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-20T10:10:01.585 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-20T10:10:01.585 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-20T10:10:01.585 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-20T10:10:01.585 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-20T10:10:01.586 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-20T10:10:01.586 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-20T10:10:01.586 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-20T10:10:01.586 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-20T10:10:01.586 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-20T10:10:01.586 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-20T10:10:01.586 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-20T10:10:01.586 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-20T10:10:01.586 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-20T10:10:01.586 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-20T10:10:01.586 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-20T10:10:01.586 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-20T10:10:01.586 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-20T10:10:01.586 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-20T10:10:01.587 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-20T10:10:01.587 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-20T10:10:01.587 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-20T10:10:01.587 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-20T10:10:01.587 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-20T10:10:01.587 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-20T10:10:01.587 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-20T10:10:01.587 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-20T10:10:01.587 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-20T10:10:01.587 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-20T10:10:01.587 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-20T10:10:01.587 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-20T10:10:01.587 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-20T10:10:01.587 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-20T10:10:01.587 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-20T10:10:01.588 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-20T10:10:01.588 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-20T10:10:01.588 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-20T10:10:01.588 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-20T10:10:01.588 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-20T10:10:01.588 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-20T10:10:01.588 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-20T10:10:01.588 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-20T10:10:01.588 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-20T10:10:01.588 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-20T10:10:01.588 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-20T10:10:01.588 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-20T10:10:01.588 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-20T10:10:01.588 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-20T10:10:01.589 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-20T10:10:01.589 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-20T10:10:01.589 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-20T10:10:01.589 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-20T10:10:01.589 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-20T10:10:01.589 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-20T10:10:01.589 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-20T10:10:01.589 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-20T10:10:01.589 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-20T10:10:01.589 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-20T10:10:01.589 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-20T10:10:01.590 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-20T10:10:01.590 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-20T10:10:01.590 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-20T10:10:01.590 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-20T10:10:01.590 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-20T10:10:01.590 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-20T10:10:01.590 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-20T10:10:01.590 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-20T10:10:01.590 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-20T10:10:01.590 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-20T10:10:01.590 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-20T10:10:01.590 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-20T10:10:01.590 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-20T10:10:01.590 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-20T10:10:01.590 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-20T10:10:01.591 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-20T10:10:01.591 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-20T10:10:01.591 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-20T10:10:01.591 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-20T10:10:01.591 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-20T10:10:01.591 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-20T10:10:01.591 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-20T10:10:01.591 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-20T10:10:01.591 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-20T10:10:01.591 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-20T10:10:01.591 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-20T10:10:01.591 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-20T10:10:01.591 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-20T10:10:01.591 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-20T10:10:01.592 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-20T10:10:01.592 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-20T10:10:01.592 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-20T10:10:01.592 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-20T10:10:01.592 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-20T10:10:01.592 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-20T10:10:01.592 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-20T10:10:01.592 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-20T10:10:01.592 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-20T10:10:01.592 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-20T10:10:01.592 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-20T10:10:01.592 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-20T10:10:01.592 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-20T10:10:01.592 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-20T10:10:01.592 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-20T10:10:01.593 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-20T10:10:01.593 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-20T10:10:01.593 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-20T10:10:01.593 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-20T10:10:01.593 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-20T10:10:01.593 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-20T10:10:01.593 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-20T10:10:01.593 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-20T10:10:01.593 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-20T10:10:01.593 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-20T10:10:01.593 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-20T10:10:01.593 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-20T10:10:01.593 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-20T10:10:01.593 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-20T10:10:01.593 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-20T10:10:01.594 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-20T10:10:01.594 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-20T10:10:01.594 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-20T10:10:01.594 INFO:teuthology.orchestra.run.smithi121.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-20T10:10:01.598 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-20T10:10:01.598 DEBUG:teuthology.orchestra.run.smithi121:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-20T10:10:01.622 INFO:teuthology.orchestra.run.smithi121.stdout:postinstall scriptlet (using /bin/sh): 2024-04-20T10:10:01.622 INFO:teuthology.orchestra.run.smithi121.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-20T10:10:01.622 INFO:teuthology.orchestra.run.smithi121.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-20T10:10:01.622 INFO:teuthology.orchestra.run.smithi121.stdout:fi 2024-04-20T10:10:01.622 INFO:teuthology.orchestra.run.smithi121.stdout:for file in vmlinuz System.map config; do 2024-04-20T10:10:01.622 INFO:teuthology.orchestra.run.smithi121.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-20T10:10:01.622 INFO:teuthology.orchestra.run.smithi121.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-20T10:10:01.622 INFO:teuthology.orchestra.run.smithi121.stdout: fi 2024-04-20T10:10:01.622 INFO:teuthology.orchestra.run.smithi121.stdout:done 2024-04-20T10:10:01.623 INFO:teuthology.orchestra.run.smithi121.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-20T10:10:01.623 INFO:teuthology.orchestra.run.smithi121.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-20T10:10:01.623 INFO:teuthology.orchestra.run.smithi121.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-20T10:10:01.623 INFO:teuthology.orchestra.run.smithi121.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-20T10:10:01.623 INFO:teuthology.orchestra.run.smithi121.stdout:fi 2024-04-20T10:10:01.623 INFO:teuthology.orchestra.run.smithi121.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-20T10:10:01.623 INFO:teuthology.orchestra.run.smithi121.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-20T10:10:01.623 INFO:teuthology.orchestra.run.smithi121.stdout:fi 2024-04-20T10:10:01.623 INFO:teuthology.orchestra.run.smithi121.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-20T10:10:01.623 INFO:teuthology.orchestra.run.smithi121.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-20T10:10:01.623 INFO:teuthology.orchestra.run.smithi121.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-20T10:10:01.623 INFO:teuthology.orchestra.run.smithi121.stdout:fi 2024-04-20T10:10:01.624 DEBUG:teuthology.orchestra.run.smithi121:> sudo rpm -qi grub2-tools 2024-04-20T10:10:01.697 INFO:teuthology.orchestra.run.smithi121.stdout:Name : grub2-tools 2024-04-20T10:10:01.698 INFO:teuthology.orchestra.run.smithi121.stdout:Epoch : 1 2024-04-20T10:10:01.698 INFO:teuthology.orchestra.run.smithi121.stdout:Version : 2.06 2024-04-20T10:10:01.698 INFO:teuthology.orchestra.run.smithi121.stdout:Release : 68.el9 2024-04-20T10:10:01.698 INFO:teuthology.orchestra.run.smithi121.stdout:Architecture: x86_64 2024-04-20T10:10:01.698 INFO:teuthology.orchestra.run.smithi121.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-20T10:10:01.698 INFO:teuthology.orchestra.run.smithi121.stdout:Group : Unspecified 2024-04-20T10:10:01.698 INFO:teuthology.orchestra.run.smithi121.stdout:Size : 8264254 2024-04-20T10:10:01.698 INFO:teuthology.orchestra.run.smithi121.stdout:License : GPLv3+ 2024-04-20T10:10:01.698 INFO:teuthology.orchestra.run.smithi121.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-20T10:10:01.698 INFO:teuthology.orchestra.run.smithi121.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-20T10:10:01.698 INFO:teuthology.orchestra.run.smithi121.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-20T10:10:01.698 INFO:teuthology.orchestra.run.smithi121.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-20T10:10:01.698 INFO:teuthology.orchestra.run.smithi121.stdout:Packager : builder@centos.org 2024-04-20T10:10:01.698 INFO:teuthology.orchestra.run.smithi121.stdout:Vendor : CentOS 2024-04-20T10:10:01.699 INFO:teuthology.orchestra.run.smithi121.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-20T10:10:01.699 INFO:teuthology.orchestra.run.smithi121.stdout:Summary : Support tools for GRUB. 2024-04-20T10:10:01.699 INFO:teuthology.orchestra.run.smithi121.stdout:Description : 2024-04-20T10:10:01.699 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:10:01.699 INFO:teuthology.orchestra.run.smithi121.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-20T10:10:01.699 INFO:teuthology.orchestra.run.smithi121.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-20T10:10:01.699 INFO:teuthology.orchestra.run.smithi121.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-20T10:10:01.699 INFO:teuthology.orchestra.run.smithi121.stdout:hardware devices. 2024-04-20T10:10:01.699 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:10:01.699 INFO:teuthology.orchestra.run.smithi121.stdout:This subpackage provides tools for support of all platforms. 2024-04-20T10:10:01.700 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-20T10:10:01.700 INFO:teuthology.task.kernel:Updating grub on smithi121 to boot 6.9.0-rc3-g806ec36b2112 2024-04-20T10:10:01.700 DEBUG:teuthology.orchestra.run.smithi121:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-20T10:10:02.305 INFO:teuthology.orchestra.run.smithi121.stderr:Generating grub configuration file ... 2024-04-20T10:10:03.757 INFO:teuthology.orchestra.run.smithi121.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-20T10:10:03.783 INFO:teuthology.orchestra.run.smithi121.stderr:done 2024-04-20T10:10:03.785 DEBUG:teuthology.orchestra.run.smithi121:> mktemp 2024-04-20T10:10:03.800 INFO:teuthology.orchestra.run.smithi121.stdout:/tmp/tmp.grbTLX3cLG 2024-04-20T10:10:03.800 DEBUG:teuthology.orchestra.run.smithi121:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.grbTLX3cLG 2024-04-20T10:10:03.866 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 0666 /tmp/tmp.grbTLX3cLG 2024-04-20T10:10:04.008 DEBUG:teuthology.orchestra.remote:smithi121:/tmp/tmp.grbTLX3cLG is 6KB 2024-04-20T10:10:04.028 DEBUG:teuthology.orchestra.run.smithi121:> rm -fr /tmp/tmp.grbTLX3cLG 2024-04-20T10:10:04.042 DEBUG:teuthology.orchestra.run.smithi121:> sudo /bin/ls /boot/loader/entries || true 2024-04-20T10:10:04.102 INFO:teuthology.orchestra.run.smithi121.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-20T10:10:04.103 INFO:teuthology.orchestra.run.smithi121.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-20T10:10:04.103 INFO:teuthology.orchestra.run.smithi121.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-20T10:10:04.103 INFO:teuthology.orchestra.run.smithi121.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-20T10:10:04.104 DEBUG:teuthology.orchestra.run.smithi121:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-20T10:10:04.269 DEBUG:teuthology.orchestra.run.smithi121:> sudo shutdown -r now 2024-04-20T10:10:30.986 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi123.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-20T10:10:30.986 DEBUG:teuthology.orchestra.run.smithi123:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-20T10:10:31.067 INFO:teuthology.orchestra.run.smithi123.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-20T10:10:31.067 INFO:teuthology.orchestra.run.smithi123.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-20T10:10:31.067 INFO:teuthology.orchestra.run.smithi123.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-20T10:10:31.067 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-20T10:10:31.067 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-20T10:10:31.067 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-20T10:10:31.067 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-20T10:10:31.067 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-20T10:10:31.068 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-20T10:10:31.068 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-20T10:10:31.068 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-20T10:10:31.068 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-20T10:10:31.068 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-20T10:10:31.068 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-20T10:10:31.068 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-20T10:10:31.068 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-20T10:10:31.068 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-20T10:10:31.068 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-20T10:10:31.068 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-20T10:10:31.068 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-20T10:10:31.068 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-20T10:10:31.068 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-20T10:10:31.068 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-20T10:10:31.069 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-20T10:10:31.069 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-20T10:10:31.069 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-20T10:10:31.069 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-20T10:10:31.069 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-20T10:10:31.069 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-20T10:10:31.069 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-20T10:10:31.069 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-20T10:10:31.069 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-20T10:10:31.069 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-20T10:10:31.069 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-20T10:10:31.069 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-20T10:10:31.069 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-20T10:10:31.069 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-20T10:10:31.070 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-20T10:10:31.070 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-20T10:10:31.070 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-20T10:10:31.070 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-20T10:10:31.070 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-20T10:10:31.070 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-20T10:10:31.070 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-20T10:10:31.070 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-20T10:10:31.070 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-20T10:10:31.070 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-20T10:10:31.070 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-20T10:10:31.070 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-20T10:10:31.071 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-20T10:10:31.071 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-20T10:10:31.071 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-20T10:10:31.071 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-20T10:10:31.071 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-20T10:10:31.071 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-20T10:10:31.071 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-20T10:10:31.071 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-20T10:10:31.071 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-20T10:10:31.071 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-20T10:10:31.071 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-20T10:10:31.071 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-20T10:10:31.071 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-20T10:10:31.071 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-20T10:10:31.071 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-20T10:10:31.072 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-20T10:10:31.072 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-20T10:10:31.072 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-20T10:10:31.072 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-20T10:10:31.072 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-20T10:10:31.072 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-20T10:10:31.072 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-20T10:10:31.072 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-20T10:10:31.072 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-20T10:10:31.072 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-20T10:10:31.072 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-20T10:10:31.072 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-20T10:10:31.072 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-20T10:10:31.072 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-20T10:10:31.072 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-20T10:10:31.073 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-20T10:10:31.073 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-20T10:10:31.073 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-20T10:10:31.073 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-20T10:10:31.073 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-20T10:10:31.073 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-20T10:10:31.073 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-20T10:10:31.073 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-20T10:10:31.073 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-20T10:10:31.073 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-20T10:10:31.073 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-20T10:10:31.073 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-20T10:10:31.073 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-20T10:10:31.073 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-20T10:10:31.074 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-20T10:10:31.074 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-20T10:10:31.074 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-20T10:10:31.074 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-20T10:10:31.074 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-20T10:10:31.074 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-20T10:10:31.074 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-20T10:10:31.074 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-20T10:10:31.074 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-20T10:10:31.074 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-20T10:10:31.074 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-20T10:10:31.074 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-20T10:10:31.074 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-20T10:10:31.074 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-20T10:10:31.075 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-20T10:10:31.075 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-20T10:10:31.075 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-20T10:10:31.075 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-20T10:10:31.075 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-20T10:10:31.075 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-20T10:10:31.075 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-20T10:10:31.075 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-20T10:10:31.075 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-20T10:10:31.075 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-20T10:10:31.075 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-20T10:10:31.075 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-20T10:10:31.075 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-20T10:10:31.075 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-20T10:10:31.076 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-20T10:10:31.076 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-20T10:10:31.076 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-20T10:10:31.076 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-20T10:10:31.076 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-20T10:10:31.076 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-20T10:10:31.076 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-20T10:10:31.076 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-20T10:10:31.076 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-20T10:10:31.076 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-20T10:10:31.076 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-20T10:10:31.076 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-20T10:10:31.076 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-20T10:10:31.077 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-20T10:10:31.077 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-20T10:10:31.077 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-20T10:10:31.077 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-20T10:10:31.077 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-20T10:10:31.077 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-20T10:10:31.077 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-20T10:10:31.077 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-20T10:10:31.077 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-20T10:10:31.077 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-20T10:10:31.077 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-20T10:10:31.077 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-20T10:10:31.077 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-20T10:10:31.077 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-20T10:10:31.078 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-20T10:10:31.078 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-20T10:10:31.078 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-20T10:10:31.078 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-20T10:10:31.078 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-20T10:10:31.078 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-20T10:10:31.078 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-20T10:10:31.078 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-20T10:10:31.078 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-20T10:10:31.078 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-20T10:10:31.078 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-20T10:10:31.078 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-20T10:10:31.078 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-20T10:10:31.078 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-20T10:10:31.079 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-20T10:10:31.079 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-20T10:10:31.079 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-20T10:10:31.079 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-20T10:10:31.079 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-20T10:10:31.079 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-20T10:10:31.079 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-20T10:10:31.079 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-20T10:10:31.079 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-20T10:10:31.079 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-20T10:10:31.079 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-20T10:10:31.079 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-20T10:10:31.079 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-20T10:10:31.080 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-20T10:10:31.080 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-20T10:10:31.080 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-20T10:10:31.080 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-20T10:10:31.080 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-20T10:10:31.080 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-20T10:10:31.080 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-20T10:10:31.080 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-20T10:10:31.080 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-20T10:10:31.080 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-20T10:10:31.080 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-20T10:10:31.080 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-20T10:10:31.080 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-20T10:10:31.080 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-20T10:10:31.080 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-20T10:10:31.081 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-20T10:10:31.081 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-20T10:10:31.081 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-20T10:10:31.081 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-20T10:10:31.081 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-20T10:10:31.081 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-20T10:10:31.081 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-20T10:10:31.081 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-20T10:10:31.081 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-20T10:10:31.081 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-20T10:10:31.081 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-20T10:10:31.081 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-20T10:10:31.081 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-20T10:10:31.081 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-20T10:10:31.081 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-20T10:10:31.082 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-20T10:10:31.082 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-20T10:10:31.082 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-20T10:10:31.082 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-20T10:10:31.082 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-20T10:10:31.082 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-20T10:10:31.082 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-20T10:10:31.082 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-20T10:10:31.082 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-20T10:10:31.082 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-20T10:10:31.082 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-20T10:10:31.082 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-20T10:10:31.082 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-20T10:10:31.083 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-20T10:10:31.083 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-20T10:10:31.083 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-20T10:10:31.083 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-20T10:10:31.083 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-20T10:10:31.083 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-20T10:10:31.083 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-20T10:10:31.083 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-20T10:10:31.083 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-20T10:10:31.083 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-20T10:10:31.083 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-20T10:10:31.083 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-20T10:10:31.083 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-20T10:10:31.083 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-20T10:10:31.084 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-20T10:10:31.084 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-20T10:10:31.084 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-20T10:10:31.084 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-20T10:10:31.084 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-20T10:10:31.084 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-20T10:10:31.084 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-20T10:10:31.084 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-20T10:10:31.084 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-20T10:10:31.084 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-20T10:10:31.084 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-20T10:10:31.084 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-20T10:10:31.084 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-20T10:10:31.084 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-20T10:10:31.084 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-20T10:10:31.085 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-20T10:10:31.085 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-20T10:10:31.085 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-20T10:10:31.085 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-20T10:10:31.085 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-20T10:10:31.085 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-20T10:10:31.085 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-20T10:10:31.085 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-20T10:10:31.085 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-20T10:10:31.085 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-20T10:10:31.085 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-20T10:10:31.085 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-20T10:10:31.085 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-20T10:10:31.085 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-20T10:10:31.086 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-20T10:10:31.086 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-20T10:10:31.086 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-20T10:10:31.086 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-20T10:10:31.086 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-20T10:10:31.086 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-20T10:10:31.086 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-20T10:10:31.086 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-20T10:10:31.086 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-20T10:10:31.086 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-20T10:10:31.086 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-20T10:10:31.086 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-20T10:10:31.086 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-20T10:10:31.086 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-20T10:10:31.087 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-20T10:10:31.087 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-20T10:10:31.087 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-20T10:10:31.087 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-20T10:10:31.087 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-20T10:10:31.087 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-20T10:10:31.087 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-20T10:10:31.087 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-20T10:10:31.087 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-20T10:10:31.087 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-20T10:10:31.087 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-20T10:10:31.087 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-20T10:10:31.087 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-20T10:10:31.087 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-20T10:10:31.088 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-20T10:10:31.088 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-20T10:10:31.088 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-20T10:10:31.088 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-20T10:10:31.088 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-20T10:10:31.088 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-20T10:10:31.088 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-20T10:10:31.088 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-20T10:10:31.088 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-20T10:10:31.088 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-20T10:10:31.088 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-20T10:10:31.088 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-20T10:10:31.088 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-20T10:10:31.088 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-20T10:10:31.088 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-20T10:10:31.089 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-20T10:10:31.089 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-20T10:10:31.089 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-20T10:10:31.089 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-20T10:10:31.089 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-20T10:10:31.089 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-20T10:10:31.089 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-20T10:10:31.089 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-20T10:10:31.089 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-20T10:10:31.089 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-20T10:10:31.089 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-20T10:10:31.089 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-20T10:10:31.089 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-20T10:10:31.089 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-20T10:10:31.090 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-20T10:10:31.090 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-20T10:10:31.090 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-20T10:10:31.090 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-20T10:10:31.090 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-20T10:10:31.090 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-20T10:10:31.090 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-20T10:10:31.090 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-20T10:10:31.090 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-20T10:10:31.090 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-20T10:10:31.090 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-20T10:10:31.090 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-20T10:10:31.090 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-20T10:10:31.090 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-20T10:10:31.091 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-20T10:10:31.091 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-20T10:10:31.091 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-20T10:10:31.091 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-20T10:10:31.091 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-20T10:10:31.091 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-20T10:10:31.091 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-20T10:10:31.091 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-20T10:10:31.091 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-20T10:10:31.091 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-20T10:10:31.091 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-20T10:10:31.091 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-20T10:10:31.091 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-20T10:10:31.091 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-20T10:10:31.092 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-20T10:10:31.092 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-20T10:10:31.092 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-20T10:10:31.092 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-20T10:10:31.092 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-20T10:10:31.092 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-20T10:10:31.092 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-20T10:10:31.092 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-20T10:10:31.092 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-20T10:10:31.092 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-20T10:10:31.092 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-20T10:10:31.092 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-20T10:10:31.092 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-20T10:10:31.093 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-20T10:10:31.093 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-20T10:10:31.093 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-20T10:10:31.093 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-20T10:10:31.093 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-20T10:10:31.093 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-20T10:10:31.093 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-20T10:10:31.093 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-20T10:10:31.093 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-20T10:10:31.093 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-20T10:10:31.093 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-20T10:10:31.093 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-20T10:10:31.093 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-20T10:10:31.093 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-20T10:10:31.094 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-20T10:10:31.094 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-20T10:10:31.094 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-20T10:10:31.094 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-20T10:10:31.094 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-20T10:10:31.094 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-20T10:10:31.094 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-20T10:10:31.094 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-20T10:10:31.094 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-20T10:10:31.094 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-20T10:10:31.094 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-20T10:10:31.094 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-20T10:10:31.094 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-20T10:10:31.094 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-20T10:10:31.095 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-20T10:10:31.095 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-20T10:10:31.095 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-20T10:10:31.095 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-20T10:10:31.095 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-20T10:10:31.095 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-20T10:10:31.095 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-20T10:10:31.095 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-20T10:10:31.095 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-20T10:10:31.095 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-20T10:10:31.095 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-20T10:10:31.095 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-20T10:10:31.095 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-20T10:10:31.095 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-20T10:10:31.096 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-20T10:10:31.096 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-20T10:10:31.096 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-20T10:10:31.096 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-20T10:10:31.096 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-20T10:10:31.096 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-20T10:10:31.096 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-20T10:10:31.096 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-20T10:10:31.096 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-20T10:10:31.096 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-20T10:10:31.096 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-20T10:10:31.096 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-20T10:10:31.096 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-20T10:10:31.096 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-20T10:10:31.097 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-20T10:10:31.097 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-20T10:10:31.097 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-20T10:10:31.097 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-20T10:10:31.097 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-20T10:10:31.097 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-20T10:10:31.097 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-20T10:10:31.097 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-20T10:10:31.097 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-20T10:10:31.097 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-20T10:10:31.097 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-20T10:10:31.097 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-20T10:10:31.097 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-20T10:10:31.097 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-20T10:10:31.098 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-20T10:10:31.098 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-20T10:10:31.098 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-20T10:10:31.098 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-20T10:10:31.098 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-20T10:10:31.098 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-20T10:10:31.098 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-20T10:10:31.098 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-20T10:10:31.098 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-20T10:10:31.098 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-20T10:10:31.098 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-20T10:10:31.098 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-20T10:10:31.098 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-20T10:10:31.098 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-20T10:10:31.099 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-20T10:10:31.099 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-20T10:10:31.099 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-20T10:10:31.099 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-20T10:10:31.099 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-20T10:10:31.099 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-20T10:10:31.099 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-20T10:10:31.099 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-20T10:10:31.099 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-20T10:10:31.099 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-20T10:10:31.099 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-20T10:10:31.099 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-20T10:10:31.099 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-20T10:10:31.099 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-20T10:10:31.100 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-20T10:10:31.100 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-20T10:10:31.100 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-20T10:10:31.100 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-20T10:10:31.100 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-20T10:10:31.100 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-20T10:10:31.100 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-20T10:10:31.100 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-20T10:10:31.100 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-20T10:10:31.100 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-20T10:10:31.100 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-20T10:10:31.100 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-20T10:10:31.100 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-20T10:10:31.100 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-20T10:10:31.100 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-20T10:10:31.101 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-20T10:10:31.101 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-20T10:10:31.101 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-20T10:10:31.101 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-20T10:10:31.101 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-20T10:10:31.101 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-20T10:10:31.101 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-20T10:10:31.101 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-20T10:10:31.101 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-20T10:10:31.101 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-20T10:10:31.101 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-20T10:10:31.101 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-20T10:10:31.101 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-20T10:10:31.101 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-20T10:10:31.102 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-20T10:10:31.102 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-20T10:10:31.102 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-20T10:10:31.102 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-20T10:10:31.102 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-20T10:10:31.102 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-20T10:10:31.102 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-20T10:10:31.102 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-20T10:10:31.102 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-20T10:10:31.102 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-20T10:10:31.102 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-20T10:10:31.102 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-20T10:10:31.102 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-20T10:10:31.102 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-20T10:10:31.103 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-20T10:10:31.103 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-20T10:10:31.103 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-20T10:10:31.103 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-20T10:10:31.103 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-20T10:10:31.103 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-20T10:10:31.103 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-20T10:10:31.103 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-20T10:10:31.103 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-20T10:10:31.103 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-20T10:10:31.103 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-20T10:10:31.103 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-20T10:10:31.103 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-20T10:10:31.104 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-20T10:10:31.104 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-20T10:10:31.104 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-20T10:10:31.104 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-20T10:10:31.104 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-20T10:10:31.104 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-20T10:10:31.104 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-20T10:10:31.104 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-20T10:10:31.104 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-20T10:10:31.104 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-20T10:10:31.104 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-20T10:10:31.104 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-20T10:10:31.104 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-20T10:10:31.104 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-20T10:10:31.104 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-20T10:10:31.105 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-20T10:10:31.105 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-20T10:10:31.105 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-20T10:10:31.105 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-20T10:10:31.105 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-20T10:10:31.105 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-20T10:10:31.105 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-20T10:10:31.105 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-20T10:10:31.105 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-20T10:10:31.105 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-20T10:10:31.105 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-20T10:10:31.105 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-20T10:10:31.105 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-20T10:10:31.105 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-20T10:10:31.106 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-20T10:10:31.106 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-20T10:10:31.106 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-20T10:10:31.106 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-20T10:10:31.106 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-20T10:10:31.106 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-20T10:10:31.106 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-20T10:10:31.106 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-20T10:10:31.106 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-20T10:10:31.106 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-20T10:10:31.106 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-20T10:10:31.106 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-20T10:10:31.106 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-20T10:10:31.106 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-20T10:10:31.107 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-20T10:10:31.107 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-20T10:10:31.107 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-20T10:10:31.107 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-20T10:10:31.107 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-20T10:10:31.107 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-20T10:10:31.107 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-20T10:10:31.107 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-20T10:10:31.107 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-20T10:10:31.107 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-20T10:10:31.107 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-20T10:10:31.107 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-20T10:10:31.107 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-20T10:10:31.107 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-20T10:10:31.107 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-20T10:10:31.108 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-20T10:10:31.108 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-20T10:10:31.108 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-20T10:10:31.108 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-20T10:10:31.108 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-20T10:10:31.108 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-20T10:10:31.108 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-20T10:10:31.108 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-20T10:10:31.108 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-20T10:10:31.108 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-20T10:10:31.108 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-20T10:10:31.108 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-20T10:10:31.108 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-20T10:10:31.108 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-20T10:10:31.109 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-20T10:10:31.109 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-20T10:10:31.109 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-20T10:10:31.109 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-20T10:10:31.109 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-20T10:10:31.109 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-20T10:10:31.109 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-20T10:10:31.109 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-20T10:10:31.109 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-20T10:10:31.109 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-20T10:10:31.109 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-20T10:10:31.109 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-20T10:10:31.109 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-20T10:10:31.109 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-20T10:10:31.110 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-20T10:10:31.110 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-20T10:10:31.110 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-20T10:10:31.110 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-20T10:10:31.110 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-20T10:10:31.110 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-20T10:10:31.110 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-20T10:10:31.110 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-20T10:10:31.110 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-20T10:10:31.110 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-20T10:10:31.110 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-20T10:10:31.110 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-20T10:10:31.110 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-20T10:10:31.110 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-20T10:10:31.111 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-20T10:10:31.111 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-20T10:10:31.111 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-20T10:10:31.111 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-20T10:10:31.111 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-20T10:10:31.111 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-20T10:10:31.111 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-20T10:10:31.111 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-20T10:10:31.111 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-20T10:10:31.111 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-20T10:10:31.111 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-20T10:10:31.111 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-20T10:10:31.111 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-20T10:10:31.111 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-20T10:10:31.112 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-20T10:10:31.112 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-20T10:10:31.112 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-20T10:10:31.112 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-20T10:10:31.112 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-20T10:10:31.112 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-20T10:10:31.112 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-20T10:10:31.112 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-20T10:10:31.112 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-20T10:10:31.112 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-20T10:10:31.112 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-20T10:10:31.112 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-20T10:10:31.112 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-20T10:10:31.112 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-20T10:10:31.113 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-20T10:10:31.113 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-20T10:10:31.113 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-20T10:10:31.113 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-20T10:10:31.113 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-20T10:10:31.113 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-20T10:10:31.113 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-20T10:10:31.113 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-20T10:10:31.113 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-20T10:10:31.113 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-20T10:10:31.113 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-20T10:10:31.113 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-20T10:10:31.113 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-20T10:10:31.113 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-20T10:10:31.113 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-20T10:10:31.114 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-20T10:10:31.114 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-20T10:10:31.114 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-20T10:10:31.114 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-20T10:10:31.114 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-20T10:10:31.114 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-20T10:10:31.114 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-20T10:10:31.114 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-20T10:10:31.114 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-20T10:10:31.114 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-20T10:10:31.114 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-20T10:10:31.114 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-20T10:10:31.114 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-20T10:10:31.115 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-20T10:10:31.115 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-20T10:10:31.115 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-20T10:10:31.115 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-20T10:10:31.115 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-20T10:10:31.115 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-20T10:10:31.115 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-20T10:10:31.115 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-20T10:10:31.115 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-20T10:10:31.115 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-20T10:10:31.115 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-20T10:10:31.115 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-20T10:10:31.115 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-20T10:10:31.115 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-20T10:10:31.116 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-20T10:10:31.116 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-20T10:10:31.116 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-20T10:10:31.116 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-20T10:10:31.116 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-20T10:10:31.116 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-20T10:10:31.116 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-20T10:10:31.116 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-20T10:10:31.116 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-20T10:10:31.116 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-20T10:10:31.116 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-20T10:10:31.116 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-20T10:10:31.116 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-20T10:10:31.116 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-20T10:10:31.117 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-20T10:10:31.117 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-20T10:10:31.117 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-20T10:10:31.117 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-20T10:10:31.117 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-20T10:10:31.117 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-20T10:10:31.117 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-20T10:10:31.117 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-20T10:10:31.117 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-20T10:10:31.117 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-20T10:10:31.117 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-20T10:10:31.117 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-20T10:10:31.117 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-20T10:10:31.118 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-20T10:10:31.118 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-20T10:10:31.118 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-20T10:10:31.118 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-20T10:10:31.118 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-20T10:10:31.118 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-20T10:10:31.118 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-20T10:10:31.118 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-20T10:10:31.118 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-20T10:10:31.118 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-20T10:10:31.118 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-20T10:10:31.118 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-20T10:10:31.118 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-20T10:10:31.119 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-20T10:10:31.119 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-20T10:10:31.119 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-20T10:10:31.119 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-20T10:10:31.119 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-20T10:10:31.119 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-20T10:10:31.119 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-20T10:10:31.119 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-20T10:10:31.119 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-20T10:10:31.119 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-20T10:10:31.119 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-20T10:10:31.119 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-20T10:10:31.119 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-20T10:10:31.119 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-20T10:10:31.119 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-20T10:10:31.120 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-20T10:10:31.120 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-20T10:10:31.120 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-20T10:10:31.120 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-20T10:10:31.120 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-20T10:10:31.120 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-20T10:10:31.120 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-20T10:10:31.120 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-20T10:10:31.120 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-20T10:10:31.120 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-20T10:10:31.120 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-20T10:10:31.120 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-20T10:10:31.120 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-20T10:10:31.120 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-20T10:10:31.120 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-20T10:10:31.121 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-20T10:10:31.121 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-20T10:10:31.121 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-20T10:10:31.121 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-20T10:10:31.121 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-20T10:10:31.121 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-20T10:10:31.121 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-20T10:10:31.121 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-20T10:10:31.121 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-20T10:10:31.121 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-20T10:10:31.121 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-20T10:10:31.121 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-20T10:10:31.121 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-20T10:10:31.121 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-20T10:10:31.122 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-20T10:10:31.122 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-20T10:10:31.122 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-20T10:10:31.122 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-20T10:10:31.122 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-20T10:10:31.122 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-20T10:10:31.122 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-20T10:10:31.122 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-20T10:10:31.122 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-20T10:10:31.122 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-20T10:10:31.122 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-20T10:10:31.122 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-20T10:10:31.122 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-20T10:10:31.122 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-20T10:10:31.123 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-20T10:10:31.123 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-20T10:10:31.123 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-20T10:10:31.123 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-20T10:10:31.123 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-20T10:10:31.123 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-20T10:10:31.123 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-20T10:10:31.123 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-20T10:10:31.123 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-20T10:10:31.123 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-20T10:10:31.123 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-20T10:10:31.123 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-20T10:10:31.123 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-20T10:10:31.123 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-20T10:10:31.124 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-20T10:10:31.124 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-20T10:10:31.124 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-20T10:10:31.124 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-20T10:10:31.124 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-20T10:10:31.124 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-20T10:10:31.124 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-20T10:10:31.124 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-20T10:10:31.124 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-20T10:10:31.124 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-20T10:10:31.124 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-20T10:10:31.124 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-20T10:10:31.124 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-20T10:10:31.124 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-20T10:10:31.125 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-20T10:10:31.125 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-20T10:10:31.125 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-20T10:10:31.125 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-20T10:10:31.125 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-20T10:10:31.125 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-20T10:10:31.125 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-20T10:10:31.125 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-20T10:10:31.125 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-20T10:10:31.125 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-20T10:10:31.125 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-20T10:10:31.125 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-20T10:10:31.125 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-20T10:10:31.125 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-20T10:10:31.125 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-20T10:10:31.126 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-20T10:10:31.126 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-20T10:10:31.126 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-20T10:10:31.126 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-20T10:10:31.126 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-20T10:10:31.126 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-20T10:10:31.126 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-20T10:10:31.126 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-20T10:10:31.126 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-20T10:10:31.126 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-20T10:10:31.126 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-20T10:10:31.126 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-20T10:10:31.126 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-20T10:10:31.126 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-20T10:10:31.127 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-20T10:10:31.127 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-20T10:10:31.127 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-20T10:10:31.127 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-20T10:10:31.127 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-20T10:10:31.127 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-20T10:10:31.127 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-20T10:10:31.127 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-20T10:10:31.127 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-20T10:10:31.127 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-20T10:10:31.127 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-20T10:10:31.127 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-20T10:10:31.127 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-20T10:10:31.127 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-20T10:10:31.127 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-20T10:10:31.128 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-20T10:10:31.128 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-20T10:10:31.128 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-20T10:10:31.128 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-20T10:10:31.128 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-20T10:10:31.128 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-20T10:10:31.128 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-20T10:10:31.128 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-20T10:10:31.128 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-20T10:10:31.128 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-20T10:10:31.128 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-20T10:10:31.128 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-20T10:10:31.128 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-20T10:10:31.128 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-20T10:10:31.128 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-20T10:10:31.129 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-20T10:10:31.129 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-20T10:10:31.129 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-20T10:10:31.129 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-20T10:10:31.129 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-20T10:10:31.129 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-20T10:10:31.129 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-20T10:10:31.129 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-20T10:10:31.129 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-20T10:10:31.129 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-20T10:10:31.129 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-20T10:10:31.129 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-20T10:10:31.129 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-20T10:10:31.129 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-20T10:10:31.129 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-20T10:10:31.130 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-20T10:10:31.130 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-20T10:10:31.130 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-20T10:10:31.130 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-20T10:10:31.130 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-20T10:10:31.130 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-20T10:10:31.130 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-20T10:10:31.130 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-20T10:10:31.130 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-20T10:10:31.130 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-20T10:10:31.130 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-20T10:10:31.130 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-20T10:10:31.130 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-20T10:10:31.131 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-20T10:10:31.131 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-20T10:10:31.131 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-20T10:10:31.131 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-20T10:10:31.131 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-20T10:10:31.131 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-20T10:10:31.131 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-20T10:10:31.131 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-20T10:10:31.131 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-20T10:10:31.131 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-20T10:10:31.131 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-20T10:10:31.131 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-20T10:10:31.131 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-20T10:10:31.131 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-20T10:10:31.131 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-20T10:10:31.132 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-20T10:10:31.132 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-20T10:10:31.132 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-20T10:10:31.132 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-20T10:10:31.132 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-20T10:10:31.132 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-20T10:10:31.132 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-20T10:10:31.132 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-20T10:10:31.132 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-20T10:10:31.132 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-20T10:10:31.132 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-20T10:10:31.132 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-20T10:10:31.132 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-20T10:10:31.132 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-20T10:10:31.133 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-20T10:10:31.133 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-20T10:10:31.133 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-20T10:10:31.133 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-20T10:10:31.133 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-20T10:10:31.133 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-20T10:10:31.133 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-20T10:10:31.133 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-20T10:10:31.133 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-20T10:10:31.133 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-20T10:10:31.133 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-20T10:10:31.133 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-20T10:10:31.133 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-20T10:10:31.133 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-20T10:10:31.134 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-20T10:10:31.134 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-20T10:10:31.134 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-20T10:10:31.134 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-20T10:10:31.134 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-20T10:10:31.134 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-20T10:10:31.134 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-20T10:10:31.134 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-20T10:10:31.134 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-20T10:10:31.134 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-20T10:10:31.134 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-20T10:10:31.134 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-20T10:10:31.134 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-20T10:10:31.134 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-20T10:10:31.135 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-20T10:10:31.135 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-20T10:10:31.135 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-20T10:10:31.135 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-20T10:10:31.135 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-20T10:10:31.135 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-20T10:10:31.135 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-20T10:10:31.135 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-20T10:10:31.135 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-20T10:10:31.135 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-20T10:10:31.135 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-20T10:10:31.135 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-20T10:10:31.135 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-20T10:10:31.135 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-20T10:10:31.135 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-20T10:10:31.136 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-20T10:10:31.136 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-20T10:10:31.136 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-20T10:10:31.136 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-20T10:10:31.136 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-20T10:10:31.136 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-20T10:10:31.136 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-20T10:10:31.136 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-20T10:10:31.136 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-20T10:10:31.136 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-20T10:10:31.136 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-20T10:10:31.136 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-20T10:10:31.136 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-20T10:10:31.136 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-20T10:10:31.136 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-20T10:10:31.137 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-20T10:10:31.137 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-20T10:10:31.137 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-20T10:10:31.137 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-20T10:10:31.137 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-20T10:10:31.137 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-20T10:10:31.137 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-20T10:10:31.137 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-20T10:10:31.137 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-20T10:10:31.137 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-20T10:10:31.137 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-20T10:10:31.137 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-20T10:10:31.137 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-20T10:10:31.137 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-20T10:10:31.137 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-20T10:10:31.137 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-20T10:10:31.138 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-20T10:10:31.138 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-20T10:10:31.138 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-20T10:10:31.138 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-20T10:10:31.138 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-20T10:10:31.138 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-20T10:10:31.138 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-20T10:10:31.138 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-20T10:10:31.138 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-20T10:10:31.138 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-20T10:10:31.138 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-20T10:10:31.138 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-20T10:10:31.138 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-20T10:10:31.138 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-20T10:10:31.139 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-20T10:10:31.139 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-20T10:10:31.139 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-20T10:10:31.139 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-20T10:10:31.139 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-20T10:10:31.139 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-20T10:10:31.139 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-20T10:10:31.139 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-20T10:10:31.139 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-20T10:10:31.139 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-20T10:10:31.139 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-20T10:10:31.139 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-20T10:10:31.139 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-20T10:10:31.139 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-20T10:10:31.140 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-20T10:10:31.140 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-20T10:10:31.140 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-20T10:10:31.140 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-20T10:10:31.140 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-20T10:10:31.140 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-20T10:10:31.140 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-20T10:10:31.140 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-20T10:10:31.140 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-20T10:10:31.140 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-20T10:10:31.140 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-20T10:10:31.140 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-20T10:10:31.140 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-20T10:10:31.140 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-20T10:10:31.140 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-20T10:10:31.141 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-20T10:10:31.141 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-20T10:10:31.141 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-20T10:10:31.141 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-20T10:10:31.141 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-20T10:10:31.141 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-20T10:10:31.141 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-20T10:10:31.141 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-20T10:10:31.141 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-20T10:10:31.141 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-20T10:10:31.141 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-20T10:10:31.141 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-20T10:10:31.141 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-20T10:10:31.141 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-20T10:10:31.141 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-20T10:10:31.141 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-20T10:10:31.142 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-20T10:10:31.142 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-20T10:10:31.142 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-20T10:10:31.142 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-20T10:10:31.142 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-20T10:10:31.142 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-20T10:10:31.142 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-20T10:10:31.142 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-20T10:10:31.142 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-20T10:10:31.142 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-20T10:10:31.142 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-20T10:10:31.142 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-20T10:10:31.142 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-20T10:10:31.142 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-20T10:10:31.143 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-20T10:10:31.143 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-20T10:10:31.143 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-20T10:10:31.143 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-20T10:10:31.143 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-20T10:10:31.143 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-20T10:10:31.143 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-20T10:10:31.143 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-20T10:10:31.143 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-20T10:10:31.143 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-20T10:10:31.143 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-20T10:10:31.143 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-20T10:10:31.143 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-20T10:10:31.143 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-20T10:10:31.143 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-20T10:10:31.144 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-20T10:10:31.144 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-20T10:10:31.144 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-20T10:10:31.144 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-20T10:10:31.144 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-20T10:10:31.144 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-20T10:10:31.144 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-20T10:10:31.144 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-20T10:10:31.144 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-20T10:10:31.144 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-20T10:10:31.144 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-20T10:10:31.144 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-20T10:10:31.144 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-20T10:10:31.144 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-20T10:10:31.144 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-20T10:10:31.145 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-20T10:10:31.145 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-20T10:10:31.145 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-20T10:10:31.145 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-20T10:10:31.145 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-20T10:10:31.145 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-20T10:10:31.145 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-20T10:10:31.145 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-20T10:10:31.145 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-20T10:10:31.145 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-20T10:10:31.145 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-20T10:10:31.145 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-20T10:10:31.145 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-20T10:10:31.145 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-20T10:10:31.145 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-20T10:10:31.146 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-20T10:10:31.146 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-20T10:10:31.146 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-20T10:10:31.146 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-20T10:10:31.146 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-20T10:10:31.146 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-20T10:10:31.146 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-20T10:10:31.146 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-20T10:10:31.146 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-20T10:10:31.146 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-20T10:10:31.146 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-20T10:10:31.146 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-20T10:10:31.146 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-20T10:10:31.146 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-20T10:10:31.147 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-20T10:10:31.147 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-20T10:10:31.147 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-20T10:10:31.147 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-20T10:10:31.147 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-20T10:10:31.147 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-20T10:10:31.147 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-20T10:10:31.147 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-20T10:10:31.147 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-20T10:10:31.147 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-20T10:10:31.147 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-20T10:10:31.147 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-20T10:10:31.147 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-20T10:10:31.147 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-20T10:10:31.147 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-20T10:10:31.148 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-20T10:10:31.148 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-20T10:10:31.148 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-20T10:10:31.148 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-20T10:10:31.148 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-20T10:10:31.148 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-20T10:10:31.148 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-20T10:10:31.148 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-20T10:10:31.148 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-20T10:10:31.148 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-20T10:10:31.148 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-20T10:10:31.148 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-20T10:10:31.148 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-20T10:10:31.149 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-20T10:10:31.149 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-20T10:10:31.149 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-20T10:10:31.149 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-20T10:10:31.149 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-20T10:10:31.149 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-20T10:10:31.149 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-20T10:10:31.149 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-20T10:10:31.149 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-20T10:10:31.149 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-20T10:10:31.149 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-20T10:10:31.149 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-20T10:10:31.149 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-20T10:10:31.149 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-20T10:10:31.149 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-20T10:10:31.150 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-20T10:10:31.150 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-20T10:10:31.150 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-20T10:10:31.150 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-20T10:10:31.150 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-20T10:10:31.150 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-20T10:10:31.150 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-20T10:10:31.150 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-20T10:10:31.150 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-20T10:10:31.150 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-20T10:10:31.150 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-20T10:10:31.150 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-20T10:10:31.150 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-20T10:10:31.150 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-20T10:10:31.151 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-20T10:10:31.151 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-20T10:10:31.151 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-20T10:10:31.151 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-20T10:10:31.151 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-20T10:10:31.151 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-20T10:10:31.151 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-20T10:10:31.151 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-20T10:10:31.151 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-20T10:10:31.151 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-20T10:10:31.151 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-20T10:10:31.151 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-20T10:10:31.151 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-20T10:10:31.151 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-20T10:10:31.151 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-20T10:10:31.152 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-20T10:10:31.152 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-20T10:10:31.152 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-20T10:10:31.152 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-20T10:10:31.152 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-20T10:10:31.152 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-20T10:10:31.152 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-20T10:10:31.152 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-20T10:10:31.152 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-20T10:10:31.152 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-20T10:10:31.152 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-20T10:10:31.152 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-20T10:10:31.152 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-20T10:10:31.152 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-20T10:10:31.152 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-20T10:10:31.153 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-20T10:10:31.153 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-20T10:10:31.153 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-20T10:10:31.153 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-20T10:10:31.153 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-20T10:10:31.153 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-20T10:10:31.153 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-20T10:10:31.153 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-20T10:10:31.153 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-20T10:10:31.153 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-20T10:10:31.153 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-20T10:10:31.153 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-20T10:10:31.153 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-20T10:10:31.153 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-20T10:10:31.153 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-20T10:10:31.154 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-20T10:10:31.154 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-20T10:10:31.154 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-20T10:10:31.154 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-20T10:10:31.154 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-20T10:10:31.154 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-20T10:10:31.154 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-20T10:10:31.154 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-20T10:10:31.154 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-20T10:10:31.154 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-20T10:10:31.154 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-20T10:10:31.154 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-20T10:10:31.154 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-20T10:10:31.154 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-20T10:10:31.155 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-20T10:10:31.155 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-20T10:10:31.155 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-20T10:10:31.155 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-20T10:10:31.155 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-20T10:10:31.155 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-20T10:10:31.155 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-20T10:10:31.155 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-20T10:10:31.155 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-20T10:10:31.155 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-20T10:10:31.155 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-20T10:10:31.155 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-20T10:10:31.155 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-20T10:10:31.155 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-20T10:10:31.156 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-20T10:10:31.156 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-20T10:10:31.156 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-20T10:10:31.156 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-20T10:10:31.156 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-20T10:10:31.156 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-20T10:10:31.156 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-20T10:10:31.156 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-20T10:10:31.156 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-20T10:10:31.156 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-20T10:10:31.156 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-20T10:10:31.156 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-20T10:10:31.156 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-20T10:10:31.156 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-20T10:10:31.156 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-20T10:10:31.157 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-20T10:10:31.157 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-20T10:10:31.157 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-20T10:10:31.157 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-20T10:10:31.157 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-20T10:10:31.157 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-20T10:10:31.157 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-20T10:10:31.157 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-20T10:10:31.157 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-20T10:10:31.157 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-20T10:10:31.157 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-20T10:10:31.157 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-20T10:10:31.157 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-20T10:10:31.158 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-20T10:10:31.158 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-20T10:10:31.158 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-20T10:10:31.158 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-20T10:10:31.158 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-20T10:10:31.158 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-20T10:10:31.158 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-20T10:10:31.158 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-20T10:10:31.158 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-20T10:10:31.158 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-20T10:10:31.158 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-20T10:10:31.158 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-20T10:10:31.158 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-20T10:10:31.158 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-20T10:10:31.159 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-20T10:10:31.159 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-20T10:10:31.159 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-20T10:10:31.159 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-20T10:10:31.159 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-20T10:10:31.159 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-20T10:10:31.159 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-20T10:10:31.159 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-20T10:10:31.159 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-20T10:10:31.159 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-20T10:10:31.159 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-20T10:10:31.159 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-20T10:10:31.159 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-20T10:10:31.159 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-20T10:10:31.159 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-20T10:10:31.159 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-20T10:10:31.160 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-20T10:10:31.160 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-20T10:10:31.160 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-20T10:10:31.160 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-20T10:10:31.160 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-20T10:10:31.160 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-20T10:10:31.160 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-20T10:10:31.160 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-20T10:10:31.160 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-20T10:10:31.160 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-20T10:10:31.160 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-20T10:10:31.160 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-20T10:10:31.160 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-20T10:10:31.160 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-20T10:10:31.160 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-20T10:10:31.160 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-20T10:10:31.161 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-20T10:10:31.161 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-20T10:10:31.161 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-20T10:10:31.161 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-20T10:10:31.161 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-20T10:10:31.161 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-20T10:10:31.161 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-20T10:10:31.161 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-20T10:10:31.161 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-20T10:10:31.161 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-20T10:10:31.161 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-20T10:10:31.161 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-20T10:10:31.161 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-20T10:10:31.161 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-20T10:10:31.161 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-20T10:10:31.161 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-20T10:10:31.162 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-20T10:10:31.162 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-20T10:10:31.162 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-20T10:10:31.162 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-20T10:10:31.162 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-20T10:10:31.162 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-20T10:10:31.162 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-20T10:10:31.162 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-20T10:10:31.162 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-20T10:10:31.162 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-20T10:10:31.162 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-20T10:10:31.162 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-20T10:10:31.162 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-20T10:10:31.162 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-20T10:10:31.163 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-20T10:10:31.163 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-20T10:10:31.163 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-20T10:10:31.163 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-20T10:10:31.163 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-20T10:10:31.163 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-20T10:10:31.163 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-20T10:10:31.163 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-20T10:10:31.163 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-20T10:10:31.163 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-20T10:10:31.163 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-20T10:10:31.163 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-20T10:10:31.163 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-20T10:10:31.163 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-20T10:10:31.163 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-20T10:10:31.163 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-20T10:10:31.164 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-20T10:10:31.164 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-20T10:10:31.164 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-20T10:10:31.164 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-20T10:10:31.164 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-20T10:10:31.164 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-20T10:10:31.164 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-20T10:10:31.164 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-20T10:10:31.164 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-20T10:10:31.164 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-20T10:10:31.164 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-20T10:10:31.164 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-20T10:10:31.164 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-20T10:10:31.164 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-20T10:10:31.165 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-20T10:10:31.165 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-20T10:10:31.165 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-20T10:10:31.165 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-20T10:10:31.165 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-20T10:10:31.165 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-20T10:10:31.165 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-20T10:10:31.165 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-20T10:10:31.165 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-20T10:10:31.165 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-20T10:10:31.165 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-20T10:10:31.165 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-20T10:10:31.165 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-20T10:10:31.165 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-20T10:10:31.165 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-20T10:10:31.165 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-20T10:10:31.166 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-20T10:10:31.166 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-20T10:10:31.166 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-20T10:10:31.166 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-20T10:10:31.166 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-20T10:10:31.166 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-20T10:10:31.166 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-20T10:10:31.166 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-20T10:10:31.166 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-20T10:10:31.166 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-20T10:10:31.166 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-20T10:10:31.166 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-20T10:10:31.166 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-20T10:10:31.167 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-20T10:10:31.167 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-20T10:10:31.167 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-20T10:10:31.167 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-20T10:10:31.167 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-20T10:10:31.167 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-20T10:10:31.167 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-20T10:10:31.167 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-20T10:10:31.167 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-20T10:10:31.167 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-20T10:10:31.167 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-20T10:10:31.167 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-20T10:10:31.167 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-20T10:10:31.167 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-20T10:10:31.167 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-20T10:10:31.167 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-20T10:10:31.168 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-20T10:10:31.168 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-20T10:10:31.168 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-20T10:10:31.168 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-20T10:10:31.168 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-20T10:10:31.168 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-20T10:10:31.168 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-20T10:10:31.168 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-20T10:10:31.168 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-20T10:10:31.168 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-20T10:10:31.168 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-20T10:10:31.168 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-20T10:10:31.168 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-20T10:10:31.168 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-20T10:10:31.168 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-20T10:10:31.168 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-20T10:10:31.169 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-20T10:10:31.169 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-20T10:10:31.169 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-20T10:10:31.169 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-20T10:10:31.169 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-20T10:10:31.169 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-20T10:10:31.169 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-20T10:10:31.169 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-20T10:10:31.169 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-20T10:10:31.169 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-20T10:10:31.169 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-20T10:10:31.169 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-20T10:10:31.169 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-20T10:10:31.169 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-20T10:10:31.169 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-20T10:10:31.170 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-20T10:10:31.170 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-20T10:10:31.170 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-20T10:10:31.170 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-20T10:10:31.170 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-20T10:10:31.170 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-20T10:10:31.170 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-20T10:10:31.170 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-20T10:10:31.170 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-20T10:10:31.170 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-20T10:10:31.170 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-20T10:10:31.170 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-20T10:10:31.170 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-20T10:10:31.171 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-20T10:10:31.171 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-20T10:10:31.171 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-20T10:10:31.171 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-20T10:10:31.171 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-20T10:10:31.171 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-20T10:10:31.171 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-20T10:10:31.171 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-20T10:10:31.171 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-20T10:10:31.171 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-20T10:10:31.171 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-20T10:10:31.171 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-20T10:10:31.171 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-20T10:10:31.171 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-20T10:10:31.171 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-20T10:10:31.172 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-20T10:10:31.172 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-20T10:10:31.172 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-20T10:10:31.172 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-20T10:10:31.172 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-20T10:10:31.172 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-20T10:10:31.172 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-20T10:10:31.172 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-20T10:10:31.172 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-20T10:10:31.172 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-20T10:10:31.172 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-20T10:10:31.172 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-20T10:10:31.172 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-20T10:10:31.172 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-20T10:10:31.173 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-20T10:10:31.173 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-20T10:10:31.173 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-20T10:10:31.173 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-20T10:10:31.173 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-20T10:10:31.173 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-20T10:10:31.173 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-20T10:10:31.173 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-20T10:10:31.173 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-20T10:10:31.173 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-20T10:10:31.173 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-20T10:10:31.173 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-20T10:10:31.173 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-20T10:10:31.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-20T10:10:31.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-20T10:10:31.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-20T10:10:31.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-20T10:10:31.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-20T10:10:31.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-20T10:10:31.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-20T10:10:31.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-20T10:10:31.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-20T10:10:31.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-20T10:10:31.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-20T10:10:31.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-20T10:10:31.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-20T10:10:31.174 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-20T10:10:31.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-20T10:10:31.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-20T10:10:31.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-20T10:10:31.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-20T10:10:31.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-20T10:10:31.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-20T10:10:31.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-20T10:10:31.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-20T10:10:31.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-20T10:10:31.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-20T10:10:31.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-20T10:10:31.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-20T10:10:31.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-20T10:10:31.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-20T10:10:31.175 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-20T10:10:31.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-20T10:10:31.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-20T10:10:31.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-20T10:10:31.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-20T10:10:31.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-20T10:10:31.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-20T10:10:31.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-20T10:10:31.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-20T10:10:31.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-20T10:10:31.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-20T10:10:31.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-20T10:10:31.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-20T10:10:31.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-20T10:10:31.176 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-20T10:10:31.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-20T10:10:31.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-20T10:10:31.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-20T10:10:31.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-20T10:10:31.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-20T10:10:31.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-20T10:10:31.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-20T10:10:31.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-20T10:10:31.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-20T10:10:31.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-20T10:10:31.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-20T10:10:31.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-20T10:10:31.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-20T10:10:31.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-20T10:10:31.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-20T10:10:31.177 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-20T10:10:31.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-20T10:10:31.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-20T10:10:31.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-20T10:10:31.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-20T10:10:31.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-20T10:10:31.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-20T10:10:31.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-20T10:10:31.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-20T10:10:31.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-20T10:10:31.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-20T10:10:31.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-20T10:10:31.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-20T10:10:31.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-20T10:10:31.178 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-20T10:10:31.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-20T10:10:31.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-20T10:10:31.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-20T10:10:31.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-20T10:10:31.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-20T10:10:31.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-20T10:10:31.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-20T10:10:31.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-20T10:10:31.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-20T10:10:31.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-20T10:10:31.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-20T10:10:31.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-20T10:10:31.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-20T10:10:31.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-20T10:10:31.179 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-20T10:10:31.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-20T10:10:31.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-20T10:10:31.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-20T10:10:31.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-20T10:10:31.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-20T10:10:31.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-20T10:10:31.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-20T10:10:31.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-20T10:10:31.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-20T10:10:31.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-20T10:10:31.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-20T10:10:31.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-20T10:10:31.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-20T10:10:31.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-20T10:10:31.180 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-20T10:10:31.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-20T10:10:31.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-20T10:10:31.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-20T10:10:31.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-20T10:10:31.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-20T10:10:31.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-20T10:10:31.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-20T10:10:31.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-20T10:10:31.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-20T10:10:31.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-20T10:10:31.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-20T10:10:31.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-20T10:10:31.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-20T10:10:31.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-20T10:10:31.181 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-20T10:10:31.182 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-20T10:10:31.182 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-20T10:10:31.182 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-20T10:10:31.182 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-20T10:10:31.182 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-20T10:10:31.182 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-20T10:10:31.182 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-20T10:10:31.182 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-20T10:10:31.182 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-20T10:10:31.182 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-20T10:10:31.182 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-20T10:10:31.182 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-20T10:10:31.182 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-20T10:10:31.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-20T10:10:31.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-20T10:10:31.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-20T10:10:31.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-20T10:10:31.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-20T10:10:31.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-20T10:10:31.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-20T10:10:31.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-20T10:10:31.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-20T10:10:31.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-20T10:10:31.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-20T10:10:31.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-20T10:10:31.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-20T10:10:31.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-20T10:10:31.183 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-20T10:10:31.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-20T10:10:31.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-20T10:10:31.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-20T10:10:31.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-20T10:10:31.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-20T10:10:31.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-20T10:10:31.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-20T10:10:31.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-20T10:10:31.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-20T10:10:31.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-20T10:10:31.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-20T10:10:31.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-20T10:10:31.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-20T10:10:31.184 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-20T10:10:31.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-20T10:10:31.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-20T10:10:31.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-20T10:10:31.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-20T10:10:31.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-20T10:10:31.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-20T10:10:31.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-20T10:10:31.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-20T10:10:31.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-20T10:10:31.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-20T10:10:31.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-20T10:10:31.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-20T10:10:31.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-20T10:10:31.185 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-20T10:10:31.186 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-20T10:10:31.186 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-20T10:10:31.186 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-20T10:10:31.186 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-20T10:10:31.186 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-20T10:10:31.186 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-20T10:10:31.186 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-20T10:10:31.186 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-20T10:10:31.186 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-20T10:10:31.186 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-20T10:10:31.186 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-20T10:10:31.186 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-20T10:10:31.186 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-20T10:10:31.186 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-20T10:10:31.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-20T10:10:31.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-20T10:10:31.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-20T10:10:31.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-20T10:10:31.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-20T10:10:31.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-20T10:10:31.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-20T10:10:31.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-20T10:10:31.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-20T10:10:31.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-20T10:10:31.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-20T10:10:31.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-20T10:10:31.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-20T10:10:31.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-20T10:10:31.187 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-20T10:10:31.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-20T10:10:31.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-20T10:10:31.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-20T10:10:31.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-20T10:10:31.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-20T10:10:31.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-20T10:10:31.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-20T10:10:31.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-20T10:10:31.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-20T10:10:31.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-20T10:10:31.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-20T10:10:31.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-20T10:10:31.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-20T10:10:31.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-20T10:10:31.188 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-20T10:10:31.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-20T10:10:31.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-20T10:10:31.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-20T10:10:31.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-20T10:10:31.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-20T10:10:31.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-20T10:10:31.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-20T10:10:31.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-20T10:10:31.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-20T10:10:31.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-20T10:10:31.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-20T10:10:31.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-20T10:10:31.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-20T10:10:31.189 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-20T10:10:31.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-20T10:10:31.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-20T10:10:31.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-20T10:10:31.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-20T10:10:31.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-20T10:10:31.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-20T10:10:31.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-20T10:10:31.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-20T10:10:31.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-20T10:10:31.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-20T10:10:31.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-20T10:10:31.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-20T10:10:31.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-20T10:10:31.190 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-20T10:10:31.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-20T10:10:31.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-20T10:10:31.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-20T10:10:31.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-20T10:10:31.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-20T10:10:31.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-20T10:10:31.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-20T10:10:31.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-20T10:10:31.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-20T10:10:31.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-20T10:10:31.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-20T10:10:31.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-20T10:10:31.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-20T10:10:31.191 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-20T10:10:31.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-20T10:10:31.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-20T10:10:31.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-20T10:10:31.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-20T10:10:31.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-20T10:10:31.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-20T10:10:31.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-20T10:10:31.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-20T10:10:31.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-20T10:10:31.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-20T10:10:31.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-20T10:10:31.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-20T10:10:31.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-20T10:10:31.192 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-20T10:10:31.193 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-20T10:10:31.193 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-20T10:10:31.193 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-20T10:10:31.193 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-20T10:10:31.193 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-20T10:10:31.193 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-20T10:10:31.193 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-20T10:10:31.193 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-20T10:10:31.193 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-20T10:10:31.193 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-20T10:10:31.193 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-20T10:10:31.193 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-20T10:10:31.193 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-20T10:10:31.193 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-20T10:10:31.193 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-20T10:10:31.194 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-20T10:10:31.194 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-20T10:10:31.194 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-20T10:10:31.194 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-20T10:10:31.194 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-20T10:10:31.194 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-20T10:10:31.194 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-20T10:10:31.194 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-20T10:10:31.194 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-20T10:10:31.194 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-20T10:10:31.194 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-20T10:10:31.194 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-20T10:10:31.194 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-20T10:10:31.195 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-20T10:10:31.195 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-20T10:10:31.195 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-20T10:10:31.195 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-20T10:10:31.195 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-20T10:10:31.195 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-20T10:10:31.195 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-20T10:10:31.195 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-20T10:10:31.195 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-20T10:10:31.195 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-20T10:10:31.195 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-20T10:10:31.195 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-20T10:10:31.195 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-20T10:10:31.195 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-20T10:10:31.196 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-20T10:10:31.196 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-20T10:10:31.196 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-20T10:10:31.196 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-20T10:10:31.196 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-20T10:10:31.196 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-20T10:10:31.196 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-20T10:10:31.196 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-20T10:10:31.196 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-20T10:10:31.196 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-20T10:10:31.196 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-20T10:10:31.196 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-20T10:10:31.196 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-20T10:10:31.196 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-20T10:10:31.196 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-20T10:10:31.197 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-20T10:10:31.197 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-20T10:10:31.197 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-20T10:10:31.197 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-20T10:10:31.197 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-20T10:10:31.197 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-20T10:10:31.197 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-20T10:10:31.197 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-20T10:10:31.197 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-20T10:10:31.197 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-20T10:10:31.197 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-20T10:10:31.197 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-20T10:10:31.197 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-20T10:10:31.197 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-20T10:10:31.198 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-20T10:10:31.198 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-20T10:10:31.198 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-20T10:10:31.198 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-20T10:10:31.198 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-20T10:10:31.198 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-20T10:10:31.198 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-20T10:10:31.198 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-20T10:10:31.198 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-20T10:10:31.198 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-20T10:10:31.198 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-20T10:10:31.198 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-20T10:10:31.198 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-20T10:10:31.198 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-20T10:10:31.199 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-20T10:10:31.199 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-20T10:10:31.199 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-20T10:10:31.199 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-20T10:10:31.199 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-20T10:10:31.199 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-20T10:10:31.199 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-20T10:10:31.199 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-20T10:10:31.199 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-20T10:10:31.199 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-20T10:10:31.199 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-20T10:10:31.199 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-20T10:10:31.199 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-20T10:10:31.199 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-20T10:10:31.200 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-20T10:10:31.200 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-20T10:10:31.200 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-20T10:10:31.200 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-20T10:10:31.200 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-20T10:10:31.200 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-20T10:10:31.200 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-20T10:10:31.200 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-20T10:10:31.200 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-20T10:10:31.200 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-20T10:10:31.200 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-20T10:10:31.200 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-20T10:10:31.200 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-20T10:10:31.200 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-20T10:10:31.201 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-20T10:10:31.201 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-20T10:10:31.201 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-20T10:10:31.201 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-20T10:10:31.201 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-20T10:10:31.201 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-20T10:10:31.201 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-20T10:10:31.201 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-20T10:10:31.201 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-20T10:10:31.201 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-20T10:10:31.201 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-20T10:10:31.201 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-20T10:10:31.201 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-20T10:10:31.201 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-20T10:10:31.201 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-20T10:10:31.202 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-20T10:10:31.202 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-20T10:10:31.202 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-20T10:10:31.202 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-20T10:10:31.202 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-20T10:10:31.202 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-20T10:10:31.202 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-20T10:10:31.202 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-20T10:10:31.202 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-20T10:10:31.202 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-20T10:10:31.202 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-20T10:10:31.202 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-20T10:10:31.202 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-20T10:10:31.203 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-20T10:10:31.203 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-20T10:10:31.203 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-20T10:10:31.203 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-20T10:10:31.203 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-20T10:10:31.203 INFO:teuthology.orchestra.run.smithi123.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-20T10:10:31.207 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-20T10:10:31.207 DEBUG:teuthology.orchestra.run.smithi123:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-20T10:10:31.230 INFO:teuthology.orchestra.run.smithi123.stdout:postinstall scriptlet (using /bin/sh): 2024-04-20T10:10:31.230 INFO:teuthology.orchestra.run.smithi123.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-20T10:10:31.230 INFO:teuthology.orchestra.run.smithi123.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-20T10:10:31.230 INFO:teuthology.orchestra.run.smithi123.stdout:fi 2024-04-20T10:10:31.230 INFO:teuthology.orchestra.run.smithi123.stdout:for file in vmlinuz System.map config; do 2024-04-20T10:10:31.230 INFO:teuthology.orchestra.run.smithi123.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-20T10:10:31.230 INFO:teuthology.orchestra.run.smithi123.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-20T10:10:31.230 INFO:teuthology.orchestra.run.smithi123.stdout: fi 2024-04-20T10:10:31.230 INFO:teuthology.orchestra.run.smithi123.stdout:done 2024-04-20T10:10:31.230 INFO:teuthology.orchestra.run.smithi123.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-20T10:10:31.231 INFO:teuthology.orchestra.run.smithi123.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-20T10:10:31.231 INFO:teuthology.orchestra.run.smithi123.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-20T10:10:31.231 INFO:teuthology.orchestra.run.smithi123.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-20T10:10:31.231 INFO:teuthology.orchestra.run.smithi123.stdout:fi 2024-04-20T10:10:31.231 INFO:teuthology.orchestra.run.smithi123.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-20T10:10:31.231 INFO:teuthology.orchestra.run.smithi123.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-20T10:10:31.231 INFO:teuthology.orchestra.run.smithi123.stdout:fi 2024-04-20T10:10:31.231 INFO:teuthology.orchestra.run.smithi123.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-20T10:10:31.231 INFO:teuthology.orchestra.run.smithi123.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-20T10:10:31.231 INFO:teuthology.orchestra.run.smithi123.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-20T10:10:31.231 INFO:teuthology.orchestra.run.smithi123.stdout:fi 2024-04-20T10:10:31.232 DEBUG:teuthology.orchestra.run.smithi123:> sudo rpm -qi grub2-tools 2024-04-20T10:10:31.307 INFO:teuthology.orchestra.run.smithi123.stdout:Name : grub2-tools 2024-04-20T10:10:31.307 INFO:teuthology.orchestra.run.smithi123.stdout:Epoch : 1 2024-04-20T10:10:31.307 INFO:teuthology.orchestra.run.smithi123.stdout:Version : 2.06 2024-04-20T10:10:31.307 INFO:teuthology.orchestra.run.smithi123.stdout:Release : 68.el9 2024-04-20T10:10:31.307 INFO:teuthology.orchestra.run.smithi123.stdout:Architecture: x86_64 2024-04-20T10:10:31.307 INFO:teuthology.orchestra.run.smithi123.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-20T10:10:31.307 INFO:teuthology.orchestra.run.smithi123.stdout:Group : Unspecified 2024-04-20T10:10:31.307 INFO:teuthology.orchestra.run.smithi123.stdout:Size : 8264254 2024-04-20T10:10:31.307 INFO:teuthology.orchestra.run.smithi123.stdout:License : GPLv3+ 2024-04-20T10:10:31.308 INFO:teuthology.orchestra.run.smithi123.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-20T10:10:31.308 INFO:teuthology.orchestra.run.smithi123.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-20T10:10:31.308 INFO:teuthology.orchestra.run.smithi123.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-20T10:10:31.308 INFO:teuthology.orchestra.run.smithi123.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-20T10:10:31.308 INFO:teuthology.orchestra.run.smithi123.stdout:Packager : builder@centos.org 2024-04-20T10:10:31.308 INFO:teuthology.orchestra.run.smithi123.stdout:Vendor : CentOS 2024-04-20T10:10:31.308 INFO:teuthology.orchestra.run.smithi123.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-20T10:10:31.308 INFO:teuthology.orchestra.run.smithi123.stdout:Summary : Support tools for GRUB. 2024-04-20T10:10:31.308 INFO:teuthology.orchestra.run.smithi123.stdout:Description : 2024-04-20T10:10:31.308 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:10:31.308 INFO:teuthology.orchestra.run.smithi123.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-20T10:10:31.308 INFO:teuthology.orchestra.run.smithi123.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-20T10:10:31.308 INFO:teuthology.orchestra.run.smithi123.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-20T10:10:31.308 INFO:teuthology.orchestra.run.smithi123.stdout:hardware devices. 2024-04-20T10:10:31.308 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:10:31.309 INFO:teuthology.orchestra.run.smithi123.stdout:This subpackage provides tools for support of all platforms. 2024-04-20T10:10:31.309 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-20T10:10:31.309 INFO:teuthology.task.kernel:Updating grub on smithi123 to boot 6.9.0-rc3-g806ec36b2112 2024-04-20T10:10:31.310 DEBUG:teuthology.orchestra.run.smithi123:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-20T10:10:31.833 INFO:teuthology.orchestra.run.smithi123.stderr:Generating grub configuration file ... 2024-04-20T10:10:33.146 INFO:teuthology.orchestra.run.smithi123.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-20T10:10:33.175 INFO:teuthology.orchestra.run.smithi123.stderr:done 2024-04-20T10:10:33.178 DEBUG:teuthology.orchestra.run.smithi123:> mktemp 2024-04-20T10:10:33.193 INFO:teuthology.orchestra.run.smithi123.stdout:/tmp/tmp.QAQOgSMN5M 2024-04-20T10:10:33.193 DEBUG:teuthology.orchestra.run.smithi123:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.QAQOgSMN5M 2024-04-20T10:10:33.268 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0666 /tmp/tmp.QAQOgSMN5M 2024-04-20T10:10:33.399 DEBUG:teuthology.orchestra.remote:smithi123:/tmp/tmp.QAQOgSMN5M is 6KB 2024-04-20T10:10:33.448 DEBUG:teuthology.orchestra.run.smithi123:> rm -fr /tmp/tmp.QAQOgSMN5M 2024-04-20T10:10:33.462 DEBUG:teuthology.orchestra.run.smithi123:> sudo /bin/ls /boot/loader/entries || true 2024-04-20T10:10:33.523 INFO:teuthology.orchestra.run.smithi123.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-20T10:10:33.523 INFO:teuthology.orchestra.run.smithi123.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-20T10:10:33.523 INFO:teuthology.orchestra.run.smithi123.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-20T10:10:33.523 INFO:teuthology.orchestra.run.smithi123.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-20T10:10:33.524 DEBUG:teuthology.orchestra.run.smithi123:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-20T10:10:33.664 DEBUG:teuthology.orchestra.run.smithi123:> sudo shutdown -r now 2024-04-20T10:10:34.273 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-20T10:10:34.273 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-04-20T10:10:34.274 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:11:03.691 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-20T10:11:03.692 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-04-20T10:11:03.693 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:11:34.307 DEBUG:teuthology.orchestra.remote:timed out 2024-04-20T10:11:43.314 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-04-20T10:11:43.316 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:11:50.789 DEBUG:teuthology.orchestra.run.smithi121:> true 2024-04-20T10:11:51.425 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-04-20T10:11:51.425 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.9.0-rc3-g806ec36b2112"... 2024-04-20T10:11:51.425 DEBUG:teuthology.orchestra.run.smithi121:> uname -r 2024-04-20T10:11:51.464 INFO:teuthology.orchestra.run.smithi121.stdout:6.9.0-rc3-g806ec36b2112 2024-04-20T10:11:51.464 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-20T10:11:51.464 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-20T10:11:51.465 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-20T10:11:52.465 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-20T10:11:52.465 DEBUG:teuthology.orchestra.run.smithi121:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-20T10:11:52.572 INFO:teuthology.orchestra.run.smithi121.stdout:ttyS1 2024-04-20T10:11:52.637 DEBUG:teuthology.parallel:result is None 2024-04-20T10:12:03.695 DEBUG:teuthology.orchestra.remote:timed out 2024-04-20T10:12:12.696 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-04-20T10:12:12.697 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:12:20.215 DEBUG:teuthology.orchestra.run.smithi123:> true 2024-04-20T10:12:20.795 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-04-20T10:12:20.795 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.9.0-rc3-g806ec36b2112"... 2024-04-20T10:12:20.796 DEBUG:teuthology.orchestra.run.smithi123:> uname -r 2024-04-20T10:12:20.834 INFO:teuthology.orchestra.run.smithi123.stdout:6.9.0-rc3-g806ec36b2112 2024-04-20T10:12:20.834 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-20T10:12:20.834 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-20T10:12:20.834 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-20T10:12:21.836 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-20T10:12:21.836 DEBUG:teuthology.orchestra.run.smithi123:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-20T10:12:21.945 INFO:teuthology.orchestra.run.smithi123.stdout:ttyS1 2024-04-20T10:12:22.003 DEBUG:teuthology.parallel:result is None 2024-04-20T10:12:22.004 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-20T10:12:22.012 INFO:teuthology.task.internal:Creating test directory... 2024-04-20T10:12:22.012 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-20T10:12:22.022 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-20T10:12:22.070 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-20T10:12:22.077 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-20T10:12:22.088 INFO:teuthology.task.internal:Creating archive directory... 2024-04-20T10:12:22.089 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-20T10:12:22.097 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-20T10:12:22.160 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-20T10:12:22.167 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-20T10:12:22.167 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-20T10:12:22.173 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-20T10:12:22.259 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T10:12:22.291 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T10:12:22.316 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T10:12:22.348 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T10:12:22.352 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-20T10:12:22.360 INFO:teuthology.task.internal:Configuring sudo... 2024-04-20T10:12:22.361 DEBUG:teuthology.orchestra.run.smithi121:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-20T10:12:22.369 DEBUG:teuthology.orchestra.run.smithi123:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-20T10:12:22.447 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-20T10:12:22.457 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-20T10:12:22.458 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-20T10:12:22.472 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-20T10:12:22.517 DEBUG:teuthology.orchestra.run.smithi121:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T10:12:22.596 DEBUG:teuthology.orchestra.run.smithi121:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T10:12:22.694 DEBUG:teuthology.orchestra.run.smithi121:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T10:12:22.776 DEBUG:teuthology.orchestra.run.smithi121:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T10:12:22.860 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T10:12:22.860 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-20T10:12:22.940 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T10:12:23.016 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T10:12:23.106 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T10:12:23.179 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T10:12:23.254 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T10:12:23.254 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-20T10:12:23.320 DEBUG:teuthology.orchestra.run.smithi121:> sudo service rsyslog restart 2024-04-20T10:12:23.328 DEBUG:teuthology.orchestra.run.smithi123:> sudo service rsyslog restart 2024-04-20T10:12:23.446 INFO:teuthology.orchestra.run.smithi121.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T10:12:23.450 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T10:12:23.736 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-20T10:12:23.744 INFO:teuthology.task.internal:Starting timer... 2024-04-20T10:12:23.745 INFO:teuthology.run_tasks:Running task pcp... 2024-04-20T10:12:23.755 INFO:teuthology.run_tasks:Running task selinux... 2024-04-20T10:12:23.766 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-20T10:12:23.766 DEBUG:teuthology.orchestra.run.smithi121:> sudo service auditd rotate 2024-04-20T10:12:23.895 INFO:teuthology.orchestra.run.smithi121.stdout:Rotating logs: 2024-04-20T10:12:23.900 DEBUG:teuthology.orchestra.run.smithi123:> sudo service auditd rotate 2024-04-20T10:12:24.042 INFO:teuthology.orchestra.run.smithi123.stdout:Rotating logs: 2024-04-20T10:12:24.047 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-20T10:12:24.047 DEBUG:teuthology.orchestra.run.smithi121:> /usr/sbin/getenforce 2024-04-20T10:12:24.113 INFO:teuthology.orchestra.run.smithi121.stdout:Permissive 2024-04-20T10:12:24.113 DEBUG:teuthology.orchestra.run.smithi123:> /usr/sbin/getenforce 2024-04-20T10:12:24.170 INFO:teuthology.orchestra.run.smithi123.stdout:Permissive 2024-04-20T10:12:24.171 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi121.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi123.front.sepia.ceph.com': 'permissive'} 2024-04-20T10:12:24.171 DEBUG:teuthology.orchestra.run.smithi121:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T10:12:24.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:12:24.239 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T10:12:24.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:12:24.321 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-20T10:12:24.321 DEBUG:teuthology.orchestra.run.smithi121:> sudo /usr/sbin/setenforce permissive 2024-04-20T10:12:24.390 DEBUG:teuthology.orchestra.run.smithi123:> sudo /usr/sbin/setenforce permissive 2024-04-20T10:12:24.477 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-20T10:12:24.488 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-20T10:12:24.512 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-20T10:12:24.514 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi121.front.sepia.ceph.com,smithi123.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-20T10:14:57.900 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi121.front.sepia.ceph.com'), Remote(name='ubuntu@smithi123.front.sepia.ceph.com')] 2024-04-20T10:14:57.901 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-04-20T10:14:57.902 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:14:58.011 DEBUG:teuthology.orchestra.run.smithi121:> true 2024-04-20T10:14:58.170 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi121.front.sepia.ceph.com' 2024-04-20T10:14:58.170 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-04-20T10:14:58.171 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:14:58.278 DEBUG:teuthology.orchestra.run.smithi123:> true 2024-04-20T10:14:58.435 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-04-20T10:14:58.435 INFO:teuthology.run_tasks:Running task clock... 2024-04-20T10:14:58.452 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-20T10:14:58.453 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-20T10:14:58.453 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T10:14:58.462 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-20T10:14:58.462 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T10:14:58.551 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-20T10:14:58.556 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-20T10:14:58.592 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-20T10:14:58.597 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-20T10:14:58.665 INFO:teuthology.orchestra.run.smithi121.stderr:sudo: ntpd: command not found 2024-04-20T10:14:58.666 INFO:teuthology.orchestra.run.smithi123.stderr:sudo: ntpd: command not found 2024-04-20T10:14:58.695 INFO:teuthology.orchestra.run.smithi123.stdout:506 Cannot talk to daemon 2024-04-20T10:14:58.697 INFO:teuthology.orchestra.run.smithi121.stdout:506 Cannot talk to daemon 2024-04-20T10:14:58.734 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-20T10:14:58.738 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-20T10:14:58.775 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-20T10:14:58.782 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-20T10:14:58.908 INFO:teuthology.orchestra.run.smithi123.stderr:bash: line 1: ntpq: command not found 2024-04-20T10:14:58.909 INFO:teuthology.orchestra.run.smithi121.stderr:bash: line 1: ntpq: command not found 2024-04-20T10:14:58.917 INFO:teuthology.orchestra.run.smithi121.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T10:14:58.917 INFO:teuthology.orchestra.run.smithi121.stdout:=============================================================================== 2024-04-20T10:14:58.918 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:14:58.918 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:14:58.918 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:14:58.918 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:14:58.918 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T10:14:58.918 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2024-04-20T10:14:58.918 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:14:58.918 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:14:58.919 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:14:58.919 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:14:58.920 INFO:teuthology.run_tasks:Running task install... 2024-04-20T10:14:58.929 DEBUG:teuthology.task.install:project ceph 2024-04-20T10:14:58.929 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}} 2024-04-20T10:14:58.929 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'} 2024-04-20T10:14:58.929 INFO:teuthology.task.install:Using flavor: default 2024-04-20T10:14:58.936 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-20T10:14:58.936 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-20T10:14:58.936 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'tag': None, 'wait_for_package': False} 2024-04-20T10:14:58.937 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T10:14:58.939 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'tag': None, 'wait_for_package': False} 2024-04-20T10:14:58.940 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T10:14:59.082 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/ 2024-04-20T10:14:59.082 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3133.ga8c84f59 2024-04-20T10:14:59.172 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/ 2024-04-20T10:14:59.173 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3133.ga8c84f59 2024-04-20T10:14:59.401 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-20T10:14:59.401 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T10:14:59.401 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-20T10:14:59.491 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-20T10:14:59.491 DEBUG:teuthology.orchestra.run.smithi121:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a8c84f5997950fd7d008204839253291d52e6eda/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-20T10:14:59.552 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-20T10:14:59.552 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T10:14:59.552 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-20T10:14:59.617 DEBUG:teuthology.orchestra.run.smithi121:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-20T10:14:59.643 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-20T10:14:59.688 DEBUG:teuthology.orchestra.run.smithi123:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a8c84f5997950fd7d008204839253291d52e6eda/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-20T10:14:59.729 DEBUG:teuthology.orchestra.run.smithi121:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-20T10:14:59.771 DEBUG:teuthology.orchestra.run.smithi123:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-20T10:14:59.806 INFO:teuthology.orchestra.run.smithi121.stdout:check_obsoletes = 1 2024-04-20T10:14:59.810 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum clean all 2024-04-20T10:14:59.880 DEBUG:teuthology.orchestra.run.smithi123:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-20T10:14:59.956 INFO:teuthology.orchestra.run.smithi123.stdout:check_obsoletes = 1 2024-04-20T10:14:59.960 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum clean all 2024-04-20T10:15:00.124 INFO:teuthology.orchestra.run.smithi121.stdout:48 files removed 2024-04-20T10:15:00.162 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-20T10:15:00.283 INFO:teuthology.orchestra.run.smithi123.stdout:48 files removed 2024-04-20T10:15:00.320 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-20T10:15:01.539 INFO:teuthology.orchestra.run.smithi121.stdout:ceph packages for x86_64 89 kB/s | 84 kB 00:00 2024-04-20T10:15:01.604 INFO:teuthology.orchestra.run.smithi123.stdout:ceph packages for x86_64 98 kB/s | 84 kB 00:00 2024-04-20T10:15:02.597 INFO:teuthology.orchestra.run.smithi123.stdout:ceph noarch packages 13 kB/s | 12 kB 00:00 2024-04-20T10:15:02.598 INFO:teuthology.orchestra.run.smithi121.stdout:ceph noarch packages 12 kB/s | 12 kB 00:00 2024-04-20T10:15:03.532 INFO:teuthology.orchestra.run.smithi121.stdout:ceph source packages 2.2 kB/s | 1.9 kB 00:00 2024-04-20T10:15:03.597 INFO:teuthology.orchestra.run.smithi123.stdout:ceph source packages 2.1 kB/s | 1.9 kB 00:00 2024-04-20T10:15:04.405 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-04-20T10:15:04.915 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - BaseOS 6.5 MB/s | 8.0 MB 00:01 2024-04-20T10:15:09.304 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - AppStream 5.9 MB/s | 19 MB 00:03 2024-04-20T10:15:15.031 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 6.2 MB 00:01 2024-04-20T10:15:17.066 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - Extras packages 25 kB/s | 16 kB 00:00 2024-04-20T10:15:19.131 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-20T10:15:25.110 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-04-20T10:15:27.494 INFO:teuthology.orchestra.run.smithi123.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T10:15:27.495 INFO:teuthology.orchestra.run.smithi123.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T10:15:27.514 INFO:teuthology.orchestra.run.smithi123.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-20T10:15:27.515 INFO:teuthology.orchestra.run.smithi123.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-20T10:15:27.517 INFO:teuthology.orchestra.run.smithi123.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-20T10:15:27.518 INFO:teuthology.orchestra.run.smithi123.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-20T10:15:27.520 INFO:teuthology.orchestra.run.smithi123.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-20T10:15:27.523 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-20T10:15:27.525 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-20T10:15:27.594 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-04-20T10:15:27.612 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-04-20T10:15:27.612 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-04-20T10:15:27.612 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-04-20T10:15:27.612 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-04-20T10:15:27.612 INFO:teuthology.orchestra.run.smithi123.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-20T10:15:27.612 INFO:teuthology.orchestra.run.smithi123.stdout: ceph x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 6.6 k 2024-04-20T10:15:27.613 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T10:15:27.613 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 893 k 2024-04-20T10:15:27.613 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 145 k 2024-04-20T10:15:27.613 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 1.5 M 2024-04-20T10:15:27.613 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 142 k 2024-04-20T10:15:27.613 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 3.5 M 2024-04-20T10:15:27.614 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 7.4 M 2024-04-20T10:15:27.614 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 49 k 2024-04-20T10:15:27.614 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 11 M 2024-04-20T10:15:27.614 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 49 M 2024-04-20T10:15:27.614 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 273 k 2024-04-20T10:15:27.614 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 763 k 2024-04-20T10:15:27.614 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 225 k 2024-04-20T10:15:27.614 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-shell x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 27 k 2024-04-20T10:15:27.614 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 18 k 2024-04-20T10:15:27.615 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-20T10:15:27.615 INFO:teuthology.orchestra.run.smithi123.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-20T10:15:27.615 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-20T10:15:27.615 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 32 k 2024-04-20T10:15:27.615 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 748 k 2024-04-20T10:15:27.615 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 126 k 2024-04-20T10:15:27.615 INFO:teuthology.orchestra.run.smithi123.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-20T10:15:27.615 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-20T10:15:27.615 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-20T10:15:27.615 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-20T10:15:27.615 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-20T10:15:27.615 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-20T10:15:27.615 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 162 k 2024-04-20T10:15:27.616 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 321 k 2024-04-20T10:15:27.616 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 299 k 2024-04-20T10:15:27.616 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 99 k 2024-04-20T10:15:27.616 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 85 k 2024-04-20T10:15:27.616 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T10:15:27.616 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 170 k 2024-04-20T10:15:27.616 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-20T10:15:27.616 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrading: 2024-04-20T10:15:27.616 INFO:teuthology.orchestra.run.smithi123.stdout: librados2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.4 M 2024-04-20T10:15:27.616 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T10:15:27.616 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-04-20T10:15:27.616 INFO:teuthology.orchestra.run.smithi123.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-20T10:15:27.616 INFO:teuthology.orchestra.run.smithi123.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-20T10:15:27.616 INFO:teuthology.orchestra.run.smithi123.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-20T10:15:27.617 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 21 M 2024-04-20T10:15:27.617 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 34 k 2024-04-20T10:15:27.617 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 2.4 M 2024-04-20T10:15:27.617 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 249 k 2024-04-20T10:15:27.617 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 4.7 M 2024-04-20T10:15:27.617 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 17 M 2024-04-20T10:15:27.617 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 16 k 2024-04-20T10:15:27.617 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 25 k 2024-04-20T10:15:27.617 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-20T10:15:27.617 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-20T10:15:27.617 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-20T10:15:27.617 INFO:teuthology.orchestra.run.smithi123.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-20T10:15:27.617 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-20T10:15:27.618 INFO:teuthology.orchestra.run.smithi123.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-20T10:15:27.618 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-20T10:15:27.618 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-20T10:15:27.618 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 163 k 2024-04-20T10:15:27.618 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-20T10:15:27.618 INFO:teuthology.orchestra.run.smithi123.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-20T10:15:27.618 INFO:teuthology.orchestra.run.smithi123.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-20T10:15:27.618 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-20T10:15:27.618 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-20T10:15:27.618 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 501 k 2024-04-20T10:15:27.618 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-20T10:15:27.619 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T10:15:27.619 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-20T10:15:27.619 INFO:teuthology.orchestra.run.smithi123.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-20T10:15:27.619 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-20T10:15:27.619 INFO:teuthology.orchestra.run.smithi123.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-20T10:15:27.619 INFO:teuthology.orchestra.run.smithi123.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-20T10:15:27.619 INFO:teuthology.orchestra.run.smithi123.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-20T10:15:27.619 INFO:teuthology.orchestra.run.smithi123.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-20T10:15:27.620 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-20T10:15:27.620 INFO:teuthology.orchestra.run.smithi123.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-20T10:15:27.620 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-20T10:15:27.620 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-20T10:15:27.620 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-20T10:15:27.620 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-20T10:15:27.620 INFO:teuthology.orchestra.run.smithi123.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-20T10:15:27.620 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-20T10:15:27.620 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-20T10:15:27.620 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-20T10:15:27.620 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 45 k 2024-04-20T10:15:27.620 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 135 k 2024-04-20T10:15:27.620 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-20T10:15:27.621 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-20T10:15:27.621 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-20T10:15:27.621 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-20T10:15:27.621 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-20T10:15:27.621 INFO:teuthology.orchestra.run.smithi123.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-20T10:15:27.621 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-20T10:15:27.621 INFO:teuthology.orchestra.run.smithi123.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-20T10:15:27.621 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-20T10:15:27.622 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-20T10:15:27.622 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-20T10:15:27.622 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-20T10:15:27.622 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-20T10:15:27.623 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-20T10:15:27.623 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-20T10:15:27.623 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-20T10:15:27.623 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-20T10:15:27.623 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-20T10:15:27.623 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-20T10:15:27.624 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-20T10:15:27.624 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-20T10:15:27.624 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-20T10:15:27.624 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-20T10:15:27.624 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-20T10:15:27.624 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-20T10:15:27.624 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-20T10:15:27.625 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-20T10:15:27.625 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-20T10:15:27.625 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-20T10:15:27.625 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-20T10:15:27.625 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-20T10:15:27.625 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-20T10:15:27.626 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-20T10:15:27.626 INFO:teuthology.orchestra.run.smithi123.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-20T10:15:27.626 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-20T10:15:27.626 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-20T10:15:27.626 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-20T10:15:27.626 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-20T10:15:27.627 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-20T10:15:27.627 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-20T10:15:27.627 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-20T10:15:27.627 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-20T10:15:27.627 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-20T10:15:27.627 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-20T10:15:27.627 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-20T10:15:27.628 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-20T10:15:27.628 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-20T10:15:27.628 INFO:teuthology.orchestra.run.smithi123.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-20T10:15:27.628 INFO:teuthology.orchestra.run.smithi123.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-20T10:15:27.628 INFO:teuthology.orchestra.run.smithi123.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-20T10:15:27.628 INFO:teuthology.orchestra.run.smithi123.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-20T10:15:27.628 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-20T10:15:27.629 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-20T10:15:27.629 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-20T10:15:27.629 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-20T10:15:27.629 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-20T10:15:27.629 INFO:teuthology.orchestra.run.smithi123.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-20T10:15:27.629 INFO:teuthology.orchestra.run.smithi123.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-20T10:15:27.630 INFO:teuthology.orchestra.run.smithi123.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-20T10:15:27.630 INFO:teuthology.orchestra.run.smithi123.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-20T10:15:27.630 INFO:teuthology.orchestra.run.smithi123.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-20T10:15:27.630 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-20T10:15:27.630 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-20T10:15:27.631 INFO:teuthology.orchestra.run.smithi123.stdout:Installing weak dependencies: 2024-04-20T10:15:27.631 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-20T10:15:27.631 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:15:27.631 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-04-20T10:15:27.631 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-04-20T10:15:27.631 INFO:teuthology.orchestra.run.smithi123.stdout:Install 148 Packages 2024-04-20T10:15:27.631 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrade 2 Packages 2024-04-20T10:15:27.632 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:15:27.632 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 218 M 2024-04-20T10:15:27.633 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-04-20T10:15:28.936 INFO:teuthology.orchestra.run.smithi123.stdout:(1/150): ceph-19.0.0-3133.ga8c84f59.el9.x86_64. 13 kB/s | 6.6 kB 00:00 2024-04-20T10:15:29.836 INFO:teuthology.orchestra.run.smithi123.stdout:(2/150): ceph-fuse-19.0.0-3133.ga8c84f59.el9.x8 993 kB/s | 893 kB 00:00 2024-04-20T10:15:29.953 INFO:teuthology.orchestra.run.smithi123.stdout:(3/150): ceph-immutable-object-cache-19.0.0-313 1.2 MB/s | 145 kB 00:00 2024-04-20T10:15:32.878 INFO:teuthology.orchestra.run.smithi123.stdout:(4/150): ceph-mds-19.0.0-3133.ga8c84f59.el9.x86 827 kB/s | 2.4 MB 00:02 2024-04-20T10:15:35.645 INFO:teuthology.orchestra.run.smithi123.stdout:(5/150): ceph-mgr-19.0.0-3133.ga8c84f59.el9.x86 555 kB/s | 1.5 MB 00:02 2024-04-20T10:15:37.250 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - AppStream 616 kB/s | 19 MB 00:31 2024-04-20T10:15:41.684 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-20T10:15:43.476 INFO:teuthology.orchestra.run.smithi121.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 16 kB 00:00 2024-04-20T10:15:45.067 INFO:teuthology.orchestra.run.smithi121.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-20T10:15:45.460 INFO:teuthology.orchestra.run.smithi123.stdout:(6/150): ceph-base-19.0.0-3133.ga8c84f59.el9.x8 322 kB/s | 5.4 MB 00:17 2024-04-20T10:15:48.877 INFO:teuthology.orchestra.run.smithi123.stdout:(7/150): ceph-mon-19.0.0-3133.ga8c84f59.el9.x86 364 kB/s | 4.7 MB 00:13 2024-04-20T10:15:50.971 INFO:teuthology.orchestra.run.smithi121.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-04-20T10:15:53.413 INFO:teuthology.orchestra.run.smithi121.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T10:15:53.414 INFO:teuthology.orchestra.run.smithi121.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T10:15:53.433 INFO:teuthology.orchestra.run.smithi121.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-20T10:15:53.434 INFO:teuthology.orchestra.run.smithi121.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-20T10:15:53.435 INFO:teuthology.orchestra.run.smithi121.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-20T10:15:53.437 INFO:teuthology.orchestra.run.smithi121.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-20T10:15:53.439 INFO:teuthology.orchestra.run.smithi121.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-20T10:15:53.442 INFO:teuthology.orchestra.run.smithi121.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-20T10:15:53.444 INFO:teuthology.orchestra.run.smithi121.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-20T10:15:53.513 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-04-20T10:15:53.533 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================= 2024-04-20T10:15:53.533 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2024-04-20T10:15:53.533 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================= 2024-04-20T10:15:53.534 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-04-20T10:15:53.534 INFO:teuthology.orchestra.run.smithi121.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-20T10:15:53.534 INFO:teuthology.orchestra.run.smithi121.stdout: ceph x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 6.6 k 2024-04-20T10:15:53.534 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-base x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T10:15:53.534 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 893 k 2024-04-20T10:15:53.534 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 145 k 2024-04-20T10:15:53.534 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 1.5 M 2024-04-20T10:15:53.534 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 142 k 2024-04-20T10:15:53.534 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 3.5 M 2024-04-20T10:15:53.534 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 7.4 M 2024-04-20T10:15:53.534 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-rook noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 49 k 2024-04-20T10:15:53.535 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-radosgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 11 M 2024-04-20T10:15:53.535 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-test x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 49 M 2024-04-20T10:15:53.535 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-volume noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 273 k 2024-04-20T10:15:53.535 INFO:teuthology.orchestra.run.smithi121.stdout: cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 763 k 2024-04-20T10:15:53.535 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 225 k 2024-04-20T10:15:53.535 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-shell x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 27 k 2024-04-20T10:15:53.535 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-top noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 18 k 2024-04-20T10:15:53.535 INFO:teuthology.orchestra.run.smithi121.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-20T10:15:53.535 INFO:teuthology.orchestra.run.smithi121.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-20T10:15:53.535 INFO:teuthology.orchestra.run.smithi121.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-20T10:15:53.535 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 32 k 2024-04-20T10:15:53.535 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 748 k 2024-04-20T10:15:53.535 INFO:teuthology.orchestra.run.smithi121.stdout: librados-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 126 k 2024-04-20T10:15:53.536 INFO:teuthology.orchestra.run.smithi121.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-20T10:15:53.536 INFO:teuthology.orchestra.run.smithi121.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-20T10:15:53.536 INFO:teuthology.orchestra.run.smithi121.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-20T10:15:53.536 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-20T10:15:53.536 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-20T10:15:53.536 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-20T10:15:53.536 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cephfs x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 162 k 2024-04-20T10:15:53.536 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rados x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 321 k 2024-04-20T10:15:53.536 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 299 k 2024-04-20T10:15:53.536 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 99 k 2024-04-20T10:15:53.536 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 85 k 2024-04-20T10:15:53.536 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T10:15:53.536 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-nbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 170 k 2024-04-20T10:15:53.537 INFO:teuthology.orchestra.run.smithi121.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-20T10:15:53.537 INFO:teuthology.orchestra.run.smithi121.stdout:Upgrading: 2024-04-20T10:15:53.537 INFO:teuthology.orchestra.run.smithi121.stdout: librados2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.4 M 2024-04-20T10:15:53.537 INFO:teuthology.orchestra.run.smithi121.stdout: librbd1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T10:15:53.537 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2024-04-20T10:15:53.537 INFO:teuthology.orchestra.run.smithi121.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-20T10:15:53.537 INFO:teuthology.orchestra.run.smithi121.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-20T10:15:53.537 INFO:teuthology.orchestra.run.smithi121.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-20T10:15:53.537 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 21 M 2024-04-20T10:15:53.537 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 34 k 2024-04-20T10:15:53.537 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mds x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 2.4 M 2024-04-20T10:15:53.537 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 249 k 2024-04-20T10:15:53.537 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mon x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 4.7 M 2024-04-20T10:15:53.537 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-osd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 17 M 2024-04-20T10:15:53.538 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 16 k 2024-04-20T10:15:53.538 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-selinux x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 25 k 2024-04-20T10:15:53.538 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-20T10:15:53.538 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-20T10:15:53.538 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-20T10:15:53.538 INFO:teuthology.orchestra.run.smithi121.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-20T10:15:53.538 INFO:teuthology.orchestra.run.smithi121.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-20T10:15:53.538 INFO:teuthology.orchestra.run.smithi121.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-20T10:15:53.538 INFO:teuthology.orchestra.run.smithi121.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-20T10:15:53.538 INFO:teuthology.orchestra.run.smithi121.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-20T10:15:53.538 INFO:teuthology.orchestra.run.smithi121.stdout: libcephsqlite x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 163 k 2024-04-20T10:15:53.538 INFO:teuthology.orchestra.run.smithi121.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-20T10:15:53.539 INFO:teuthology.orchestra.run.smithi121.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-20T10:15:53.539 INFO:teuthology.orchestra.run.smithi121.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-20T10:15:53.539 INFO:teuthology.orchestra.run.smithi121.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-20T10:15:53.539 INFO:teuthology.orchestra.run.smithi121.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-20T10:15:53.539 INFO:teuthology.orchestra.run.smithi121.stdout: libradosstriper1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 501 k 2024-04-20T10:15:53.539 INFO:teuthology.orchestra.run.smithi121.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-20T10:15:53.539 INFO:teuthology.orchestra.run.smithi121.stdout: librgw2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T10:15:53.539 INFO:teuthology.orchestra.run.smithi121.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-20T10:15:53.539 INFO:teuthology.orchestra.run.smithi121.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-20T10:15:53.539 INFO:teuthology.orchestra.run.smithi121.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-20T10:15:53.539 INFO:teuthology.orchestra.run.smithi121.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-20T10:15:53.539 INFO:teuthology.orchestra.run.smithi121.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-20T10:15:53.539 INFO:teuthology.orchestra.run.smithi121.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-20T10:15:53.540 INFO:teuthology.orchestra.run.smithi121.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-20T10:15:53.540 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-20T10:15:53.540 INFO:teuthology.orchestra.run.smithi121.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-20T10:15:53.540 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-20T10:15:53.540 INFO:teuthology.orchestra.run.smithi121.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-20T10:15:53.540 INFO:teuthology.orchestra.run.smithi121.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-20T10:15:53.540 INFO:teuthology.orchestra.run.smithi121.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-20T10:15:53.540 INFO:teuthology.orchestra.run.smithi121.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-20T10:15:53.540 INFO:teuthology.orchestra.run.smithi121.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-20T10:15:53.540 INFO:teuthology.orchestra.run.smithi121.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-20T10:15:53.540 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-20T10:15:53.540 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-argparse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 45 k 2024-04-20T10:15:53.540 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 135 k 2024-04-20T10:15:53.541 INFO:teuthology.orchestra.run.smithi121.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-20T10:15:53.541 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-20T10:15:53.541 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-20T10:15:53.541 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-20T10:15:53.541 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-20T10:15:53.541 INFO:teuthology.orchestra.run.smithi121.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-20T10:15:53.541 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-20T10:15:53.541 INFO:teuthology.orchestra.run.smithi121.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-20T10:15:53.541 INFO:teuthology.orchestra.run.smithi121.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-20T10:15:53.541 INFO:teuthology.orchestra.run.smithi121.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-20T10:15:53.541 INFO:teuthology.orchestra.run.smithi121.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-20T10:15:53.541 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-20T10:15:53.542 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-20T10:15:53.542 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-20T10:15:53.542 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-20T10:15:53.542 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-20T10:15:53.542 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-20T10:15:53.542 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-20T10:15:53.542 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-20T10:15:53.542 INFO:teuthology.orchestra.run.smithi121.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-20T10:15:53.542 INFO:teuthology.orchestra.run.smithi121.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-20T10:15:53.542 INFO:teuthology.orchestra.run.smithi121.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-20T10:15:53.542 INFO:teuthology.orchestra.run.smithi121.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-20T10:15:53.542 INFO:teuthology.orchestra.run.smithi121.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-20T10:15:53.543 INFO:teuthology.orchestra.run.smithi121.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-20T10:15:53.543 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-20T10:15:53.543 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-20T10:15:53.543 INFO:teuthology.orchestra.run.smithi121.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-20T10:15:53.543 INFO:teuthology.orchestra.run.smithi121.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-20T10:15:53.543 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-20T10:15:53.543 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-20T10:15:53.543 INFO:teuthology.orchestra.run.smithi121.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-20T10:15:53.543 INFO:teuthology.orchestra.run.smithi121.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-20T10:15:53.543 INFO:teuthology.orchestra.run.smithi121.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-20T10:15:53.543 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-20T10:15:53.543 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-20T10:15:53.543 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-20T10:15:53.544 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-20T10:15:53.544 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-20T10:15:53.544 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-20T10:15:53.544 INFO:teuthology.orchestra.run.smithi121.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-20T10:15:53.544 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-20T10:15:53.544 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-20T10:15:53.544 INFO:teuthology.orchestra.run.smithi121.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-20T10:15:53.544 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-20T10:15:53.544 INFO:teuthology.orchestra.run.smithi121.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-20T10:15:53.544 INFO:teuthology.orchestra.run.smithi121.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-20T10:15:53.544 INFO:teuthology.orchestra.run.smithi121.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-20T10:15:53.544 INFO:teuthology.orchestra.run.smithi121.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-20T10:15:53.544 INFO:teuthology.orchestra.run.smithi121.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-20T10:15:53.545 INFO:teuthology.orchestra.run.smithi121.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-20T10:15:53.545 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-20T10:15:53.545 INFO:teuthology.orchestra.run.smithi121.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-20T10:15:53.545 INFO:teuthology.orchestra.run.smithi121.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-20T10:15:53.545 INFO:teuthology.orchestra.run.smithi121.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-20T10:15:53.545 INFO:teuthology.orchestra.run.smithi121.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-20T10:15:53.545 INFO:teuthology.orchestra.run.smithi121.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-20T10:15:53.545 INFO:teuthology.orchestra.run.smithi121.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-20T10:15:53.545 INFO:teuthology.orchestra.run.smithi121.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-20T10:15:53.545 INFO:teuthology.orchestra.run.smithi121.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-20T10:15:53.545 INFO:teuthology.orchestra.run.smithi121.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-20T10:15:53.545 INFO:teuthology.orchestra.run.smithi121.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-20T10:15:53.546 INFO:teuthology.orchestra.run.smithi121.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-20T10:15:53.546 INFO:teuthology.orchestra.run.smithi121.stdout:Installing weak dependencies: 2024-04-20T10:15:53.546 INFO:teuthology.orchestra.run.smithi121.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-20T10:15:53.546 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:15:53.546 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-04-20T10:15:53.546 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================= 2024-04-20T10:15:53.546 INFO:teuthology.orchestra.run.smithi121.stdout:Install 148 Packages 2024-04-20T10:15:53.546 INFO:teuthology.orchestra.run.smithi121.stdout:Upgrade 2 Packages 2024-04-20T10:15:53.546 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:15:53.547 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 218 M 2024-04-20T10:15:53.547 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-04-20T10:15:54.798 INFO:teuthology.orchestra.run.smithi121.stdout:(1/150): ceph-19.0.0-3133.ga8c84f59.el9.x86_64. 13 kB/s | 6.6 kB 00:00 2024-04-20T10:15:55.689 INFO:teuthology.orchestra.run.smithi121.stdout:(2/150): ceph-fuse-19.0.0-3133.ga8c84f59.el9.x8 1.0 MB/s | 893 kB 00:00 2024-04-20T10:15:55.806 INFO:teuthology.orchestra.run.smithi121.stdout:(3/150): ceph-immutable-object-cache-19.0.0-313 1.2 MB/s | 145 kB 00:00 2024-04-20T10:15:57.640 INFO:teuthology.orchestra.run.smithi121.stdout:(4/150): ceph-mds-19.0.0-3133.ga8c84f59.el9.x86 1.3 MB/s | 2.4 MB 00:01 2024-04-20T10:15:59.365 INFO:teuthology.orchestra.run.smithi121.stdout:(5/150): ceph-mgr-19.0.0-3133.ga8c84f59.el9.x86 890 kB/s | 1.5 MB 00:01 2024-04-20T10:16:10.571 INFO:teuthology.orchestra.run.smithi121.stdout:(6/150): ceph-base-19.0.0-3133.ga8c84f59.el9.x8 337 kB/s | 5.4 MB 00:16 2024-04-20T10:16:13.796 INFO:teuthology.orchestra.run.smithi121.stdout:(7/150): ceph-mon-19.0.0-3133.ga8c84f59.el9.x86 333 kB/s | 4.7 MB 00:14 2024-04-20T10:16:15.474 INFO:teuthology.orchestra.run.smithi123.stdout:(8/150): ceph-common-19.0.0-3133.ga8c84f59.el9. 460 kB/s | 21 MB 00:47 2024-04-20T10:16:16.082 INFO:teuthology.orchestra.run.smithi123.stdout:(9/150): ceph-selinux-19.0.0-3133.ga8c84f59.el9 41 kB/s | 25 kB 00:00 2024-04-20T10:16:24.073 INFO:teuthology.orchestra.run.smithi123.stdout:(10/150): ceph-radosgw-19.0.0-3133.ga8c84f59.el 308 kB/s | 11 MB 00:35 2024-04-20T10:16:24.765 INFO:teuthology.orchestra.run.smithi123.stdout:(11/150): cephfs-mirror-19.0.0-3133.ga8c84f59.e 325 kB/s | 225 kB 00:00 2024-04-20T10:16:24.882 INFO:teuthology.orchestra.run.smithi123.stdout:(12/150): cephfs-shell-19.0.0-3133.ga8c84f59.el 233 kB/s | 27 kB 00:00 2024-04-20T10:16:25.099 INFO:teuthology.orchestra.run.smithi123.stdout:(13/150): libcephfs-devel-19.0.0-3133.ga8c84f59 147 kB/s | 32 kB 00:00 2024-04-20T10:16:28.640 INFO:teuthology.orchestra.run.smithi123.stdout:(14/150): libcephfs2-19.0.0-3133.ga8c84f59.el9. 211 kB/s | 748 kB 00:03 2024-04-20T10:16:30.693 INFO:teuthology.orchestra.run.smithi123.stdout:(15/150): libcephsqlite-19.0.0-3133.ga8c84f59.e 79 kB/s | 163 kB 00:02 2024-04-20T10:16:31.665 INFO:teuthology.orchestra.run.smithi123.stdout:(16/150): librados-devel-19.0.0-3133.ga8c84f59. 130 kB/s | 126 kB 00:00 2024-04-20T10:16:33.824 INFO:teuthology.orchestra.run.smithi123.stdout:(17/150): libradosstriper1-19.0.0-3133.ga8c84f5 232 kB/s | 501 kB 00:02 2024-04-20T10:16:35.690 INFO:teuthology.orchestra.run.smithi123.stdout:(18/150): ceph-osd-19.0.0-3133.ga8c84f59.el9.x8 343 kB/s | 17 MB 00:50 2024-04-20T10:16:35.999 INFO:teuthology.orchestra.run.smithi123.stdout:(19/150): python3-ceph-argparse-19.0.0-3133.ga8 147 kB/s | 45 kB 00:00 2024-04-20T10:16:36.591 INFO:teuthology.orchestra.run.smithi123.stdout:(20/150): python3-ceph-common-19.0.0-3133.ga8c8 228 kB/s | 135 kB 00:00 2024-04-20T10:16:37.282 INFO:teuthology.orchestra.run.smithi123.stdout:(21/150): python3-cephfs-19.0.0-3133.ga8c84f59. 235 kB/s | 162 kB 00:00 2024-04-20T10:16:38.707 INFO:teuthology.orchestra.run.smithi123.stdout:(22/150): python3-rados-19.0.0-3133.ga8c84f59.e 225 kB/s | 321 kB 00:01 2024-04-20T10:16:40.524 INFO:teuthology.orchestra.run.smithi123.stdout:(23/150): python3-rbd-19.0.0-3133.ga8c84f59.el9 164 kB/s | 299 kB 00:01 2024-04-20T10:16:41.248 INFO:teuthology.orchestra.run.smithi123.stdout:(24/150): python3-rgw-19.0.0-3133.ga8c84f59.el9 138 kB/s | 99 kB 00:00 2024-04-20T10:16:41.841 INFO:teuthology.orchestra.run.smithi123.stdout:(25/150): rbd-fuse-19.0.0-3133.ga8c84f59.el9.x8 144 kB/s | 85 kB 00:00 2024-04-20T10:16:47.798 INFO:teuthology.orchestra.run.smithi121.stdout:(8/150): ceph-radosgw-19.0.0-3133.ga8c84f59.el9 319 kB/s | 11 MB 00:34 2024-04-20T10:16:48.015 INFO:teuthology.orchestra.run.smithi121.stdout:(9/150): ceph-selinux-19.0.0-3133.ga8c84f59.el9 116 kB/s | 25 kB 00:00 2024-04-20T10:16:53.557 INFO:teuthology.orchestra.run.smithi123.stdout:(26/150): rbd-mirror-19.0.0-3133.ga8c84f59.el9. 270 kB/s | 3.1 MB 00:11 2024-04-20T10:16:54.540 INFO:teuthology.orchestra.run.smithi123.stdout:(27/150): rbd-nbd-19.0.0-3133.ga8c84f59.el9.x86 173 kB/s | 170 kB 00:00 2024-04-20T10:16:54.757 INFO:teuthology.orchestra.run.smithi123.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3133.g 158 kB/s | 34 kB 00:00 2024-04-20T10:16:55.849 INFO:teuthology.orchestra.run.smithi123.stdout:(29/150): ceph-mgr-cephadm-19.0.0-3133.ga8c84f5 130 kB/s | 142 kB 00:01 2024-04-20T10:16:56.341 INFO:teuthology.orchestra.run.smithi123.stdout:(30/150): librgw2-19.0.0-3133.ga8c84f59.el9.x86 244 kB/s | 5.4 MB 00:22 2024-04-20T10:16:58.074 INFO:teuthology.orchestra.run.smithi121.stdout:(10/150): ceph-osd-19.0.0-3133.ga8c84f59.el9.x8 363 kB/s | 17 MB 00:47 2024-04-20T10:16:59.105 INFO:teuthology.orchestra.run.smithi121.stdout:(11/150): cephfs-mirror-19.0.0-3133.ga8c84f59.e 218 kB/s | 225 kB 00:01 2024-04-20T10:16:59.280 INFO:teuthology.orchestra.run.smithi121.stdout:(12/150): cephfs-shell-19.0.0-3133.ga8c84f59.el 155 kB/s | 27 kB 00:00 2024-04-20T10:16:59.456 INFO:teuthology.orchestra.run.smithi121.stdout:(13/150): libcephfs-devel-19.0.0-3133.ga8c84f59 182 kB/s | 32 kB 00:00 2024-04-20T10:17:01.572 INFO:teuthology.orchestra.run.smithi121.stdout:(14/150): libcephfs2-19.0.0-3133.ga8c84f59.el9. 353 kB/s | 748 kB 00:02 2024-04-20T10:17:02.147 INFO:teuthology.orchestra.run.smithi121.stdout:(15/150): libcephsqlite-19.0.0-3133.ga8c84f59.e 284 kB/s | 163 kB 00:00 2024-04-20T10:17:02.564 INFO:teuthology.orchestra.run.smithi121.stdout:(16/150): librados-devel-19.0.0-3133.ga8c84f59. 304 kB/s | 126 kB 00:00 2024-04-20T10:17:03.772 INFO:teuthology.orchestra.run.smithi121.stdout:(17/150): libradosstriper1-19.0.0-3133.ga8c84f5 415 kB/s | 501 kB 00:01 2024-04-20T10:17:10.364 INFO:teuthology.orchestra.run.smithi123.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3133.ga8c84 250 kB/s | 3.5 MB 00:14 2024-04-20T10:17:11.423 INFO:teuthology.orchestra.run.smithi123.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3133.ga8 235 kB/s | 249 kB 00:01 2024-04-20T10:17:11.631 INFO:teuthology.orchestra.run.smithi123.stdout:(33/150): ceph-mgr-rook-19.0.0-3133.ga8c84f59.e 237 kB/s | 49 kB 00:00 2024-04-20T10:17:11.748 INFO:teuthology.orchestra.run.smithi123.stdout:(34/150): ceph-prometheus-alerts-19.0.0-3133.ga 142 kB/s | 16 kB 00:00 2024-04-20T10:17:12.623 INFO:teuthology.orchestra.run.smithi123.stdout:(35/150): ceph-volume-19.0.0-3133.ga8c84f59.el9 312 kB/s | 273 kB 00:00 2024-04-20T10:17:15.940 INFO:teuthology.orchestra.run.smithi123.stdout:(36/150): cephadm-19.0.0-3133.ga8c84f59.el9.noa 230 kB/s | 763 kB 00:03 2024-04-20T10:17:16.073 INFO:teuthology.orchestra.run.smithi123.stdout:(37/150): cephfs-top-19.0.0-3133.ga8c84f59.el9. 138 kB/s | 18 kB 00:00 2024-04-20T10:17:16.507 INFO:teuthology.orchestra.run.smithi123.stdout:(38/150): fuse-2.9.9-15.el9.x86_64.rpm 186 kB/s | 80 kB 00:00 2024-04-20T10:17:17.032 INFO:teuthology.orchestra.run.smithi123.stdout:(39/150): libgfortran-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 803 kB 00:00 2024-04-20T10:17:17.149 INFO:teuthology.orchestra.run.smithi123.stdout:(40/150): libquadmath-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 195 kB 00:00 2024-04-20T10:17:17.266 INFO:teuthology.orchestra.run.smithi123.stdout:(41/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-04-20T10:17:17.354 INFO:teuthology.orchestra.run.smithi121.stdout:(18/150): ceph-common-19.0.0-3133.ga8c84f59.el9 260 kB/s | 21 MB 01:23 2024-04-20T10:17:17.566 INFO:teuthology.orchestra.run.smithi123.stdout:(42/150): python3-cryptography-36.0.1-4.el9.x86 4.1 MB/s | 1.2 MB 00:00 2024-04-20T10:17:17.675 INFO:teuthology.orchestra.run.smithi123.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 991 kB/s | 106 kB 00:00 2024-04-20T10:17:17.687 INFO:teuthology.orchestra.run.smithi121.stdout:(19/150): python3-ceph-argparse-19.0.0-3133.ga8 136 kB/s | 45 kB 00:00 2024-04-20T10:17:17.783 INFO:teuthology.orchestra.run.smithi123.stdout:(44/150): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-04-20T10:17:17.900 INFO:teuthology.orchestra.run.smithi123.stdout:(45/150): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-04-20T10:17:18.009 INFO:teuthology.orchestra.run.smithi123.stdout:(46/150): python3-urllib3-1.26.5-5.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-04-20T10:17:18.117 INFO:teuthology.orchestra.run.smithi123.stdout:(47/150): automake-1.16.2-8.el9.noarch.rpm 6.3 MB/s | 689 kB 00:00 2024-04-20T10:17:18.234 INFO:teuthology.orchestra.run.smithi123.stdout:(48/150): bison-3.7.4-5.el9.x86_64.rpm 8.0 MB/s | 943 kB 00:00 2024-04-20T10:17:18.287 INFO:teuthology.orchestra.run.smithi121.stdout:(20/150): librgw2-19.0.0-3133.ga8c84f59.el9.x86 379 kB/s | 5.4 MB 00:14 2024-04-20T10:17:18.301 INFO:teuthology.orchestra.run.smithi123.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-20T10:17:18.352 INFO:teuthology.orchestra.run.smithi123.stdout:(50/150): elfutils-libelf-devel-0.190-2.el9.x86 433 kB/s | 21 kB 00:00 2024-04-20T10:17:18.370 INFO:teuthology.orchestra.run.smithi121.stdout:(21/150): python3-ceph-common-19.0.0-3133.ga8c8 197 kB/s | 135 kB 00:00 2024-04-20T10:17:18.452 INFO:teuthology.orchestra.run.smithi123.stdout:(51/150): flex-2.6.4-9.el9.x86_64.rpm 3.1 MB/s | 313 kB 00:00 2024-04-20T10:17:18.502 INFO:teuthology.orchestra.run.smithi123.stdout:(52/150): flexiblas-3.0.4-8.el9.x86_64.rpm 668 kB/s | 33 kB 00:00 2024-04-20T10:17:18.679 INFO:teuthology.orchestra.run.smithi123.stdout:(53/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 17 MB/s | 3.0 MB 00:00 2024-04-20T10:17:18.736 INFO:teuthology.orchestra.run.smithi123.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 319 kB/s | 18 kB 00:00 2024-04-20T10:17:18.770 INFO:teuthology.orchestra.run.smithi123.stdout:(55/150): libaio-devel-0.3.111-13.el9.x86_64.rp 345 kB/s | 11 kB 00:00 2024-04-20T10:17:18.846 INFO:teuthology.orchestra.run.smithi123.stdout:(56/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-04-20T10:17:18.904 INFO:teuthology.orchestra.run.smithi123.stdout:(57/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 789 kB/s | 45 kB 00:00 2024-04-20T10:17:18.954 INFO:teuthology.orchestra.run.smithi121.stdout:(22/150): python3-cephfs-19.0.0-3133.ga8c84f59. 243 kB/s | 162 kB 00:00 2024-04-20T10:17:18.996 INFO:teuthology.orchestra.run.smithi123.stdout:(58/150): librdkafka-1.6.1-102.el9.x86_64.rpm 7.1 MB/s | 662 kB 00:00 2024-04-20T10:17:19.087 INFO:teuthology.orchestra.run.smithi123.stdout:(59/150): libtool-2.4.6-45.el9.x86_64.rpm 6.3 MB/s | 582 kB 00:00 2024-04-20T10:17:19.163 INFO:teuthology.orchestra.run.smithi123.stdout:(60/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 341 kB/s | 25 kB 00:00 2024-04-20T10:17:19.222 INFO:teuthology.orchestra.run.smithi123.stdout:(61/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 824 kB/s | 47 kB 00:00 2024-04-20T10:17:19.305 INFO:teuthology.orchestra.run.smithi123.stdout:(62/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.5 MB/s | 292 kB 00:00 2024-04-20T10:17:19.381 INFO:teuthology.orchestra.run.smithi123.stdout:(63/150): lua-5.4.4-4.el9.x86_64.rpm 2.5 MB/s | 188 kB 00:00 2024-04-20T10:17:19.431 INFO:teuthology.orchestra.run.smithi123.stdout:(64/150): openblas-0.3.21-2.el9.x86_64.rpm 715 kB/s | 35 kB 00:00 2024-04-20T10:17:19.673 INFO:teuthology.orchestra.run.smithi123.stdout:(65/150): openblas-openmp-0.3.21-2.el9.x86_64.r 20 MB/s | 4.7 MB 00:00 2024-04-20T10:17:19.882 INFO:teuthology.orchestra.run.smithi123.stdout:(66/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 20 MB/s | 4.1 MB 00:00 2024-04-20T10:17:20.007 INFO:teuthology.orchestra.run.smithi123.stdout:(67/150): postgresql-13.14-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-20T10:17:20.132 INFO:teuthology.orchestra.run.smithi123.stdout:(68/150): postgresql-contrib-13.14-1.el9.x86_64 7.2 MB/s | 911 kB 00:00 2024-04-20T10:17:20.154 INFO:teuthology.orchestra.run.smithi121.stdout:(23/150): python3-rbd-19.0.0-3133.ga8c84f59.el9 249 kB/s | 299 kB 00:01 2024-04-20T10:17:20.188 INFO:teuthology.orchestra.run.smithi121.stdout:(24/150): python3-rados-19.0.0-3133.ga8c84f59.e 177 kB/s | 321 kB 00:01 2024-04-20T10:17:20.232 INFO:teuthology.orchestra.run.smithi123.stdout:(69/150): postgresql-private-libs-13.14-1.el9.x 1.3 MB/s | 133 kB 00:00 2024-04-20T10:17:20.458 INFO:teuthology.orchestra.run.smithi123.stdout:(70/150): postgresql-server-13.14-1.el9.x86_64. 26 MB/s | 5.8 MB 00:00 2024-04-20T10:17:20.529 INFO:teuthology.orchestra.run.smithi121.stdout:(25/150): rbd-fuse-19.0.0-3133.ga8c84f59.el9.x8 250 kB/s | 85 kB 00:00 2024-04-20T10:17:20.566 INFO:teuthology.orchestra.run.smithi123.stdout:(71/150): protobuf-3.14.0-13.el9.x86_64.rpm 9.4 MB/s | 1.0 MB 00:00 2024-04-20T10:17:20.580 INFO:teuthology.orchestra.run.smithi121.stdout:(26/150): python3-rgw-19.0.0-3133.ga8c84f59.el9 234 kB/s | 99 kB 00:00 2024-04-20T10:17:20.625 INFO:teuthology.orchestra.run.smithi123.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-20T10:17:20.859 INFO:teuthology.orchestra.run.smithi123.stdout:(73/150): python3-babel-2.9.1-2.el9.noarch.rpm 26 MB/s | 6.0 MB 00:00 2024-04-20T10:17:20.942 INFO:teuthology.orchestra.run.smithi123.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2024-04-20T10:17:21.017 INFO:teuthology.orchestra.run.smithi123.stdout:(75/150): python3-jinja2-2.11.3-5.el9.noarch.rp 3.2 MB/s | 248 kB 00:00 2024-04-20T10:17:21.067 INFO:teuthology.orchestra.run.smithi123.stdout:(76/150): python3-jsonpatch-1.21-16.el9.noarch. 525 kB/s | 26 kB 00:00 2024-04-20T10:17:21.118 INFO:teuthology.orchestra.run.smithi123.stdout:(77/150): python3-jsonpointer-2.0-4.el9.noarch. 384 kB/s | 19 kB 00:00 2024-04-20T10:17:21.193 INFO:teuthology.orchestra.run.smithi123.stdout:(78/150): python3-mako-1.1.4-6.el9.noarch.rpm 2.3 MB/s | 172 kB 00:00 2024-04-20T10:17:21.244 INFO:teuthology.orchestra.run.smithi123.stdout:(79/150): python3-markupsafe-1.1.1-12.el9.x86_6 699 kB/s | 35 kB 00:00 2024-04-20T10:17:21.477 INFO:teuthology.orchestra.run.smithi123.stdout:(80/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 22 MB/s | 5.1 MB 00:00 2024-04-20T10:17:21.578 INFO:teuthology.orchestra.run.smithi123.stdout:(81/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.5 MB/s | 250 kB 00:00 2024-04-20T10:17:21.661 INFO:teuthology.orchestra.run.smithi123.stdout:(82/150): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-04-20T10:17:21.720 INFO:teuthology.orchestra.run.smithi123.stdout:(83/150): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-20T10:17:21.779 INFO:teuthology.orchestra.run.smithi123.stdout:(84/150): python3-prettytable-0.7.2-27.el9.noar 734 kB/s | 42 kB 00:00 2024-04-20T10:17:21.788 INFO:teuthology.orchestra.run.smithi121.stdout:(27/150): rbd-nbd-19.0.0-3133.ga8c84f59.el9.x86 141 kB/s | 170 kB 00:01 2024-04-20T10:17:21.870 INFO:teuthology.orchestra.run.smithi123.stdout:(85/150): python3-protobuf-3.14.0-13.el9.noarch 2.9 MB/s | 269 kB 00:00 2024-04-20T10:17:21.937 INFO:teuthology.orchestra.run.smithi123.stdout:(86/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.4 MB/s | 159 kB 00:00 2024-04-20T10:17:22.013 INFO:teuthology.orchestra.run.smithi123.stdout:(87/150): python3-pyasn1-modules-0.4.8-6.el9.no 3.7 MB/s | 279 kB 00:00 2024-04-20T10:17:22.071 INFO:teuthology.orchestra.run.smithi123.stdout:(88/150): python3-pytz-2021.1-5.el9.noarch.rpm 877 kB/s | 51 kB 00:00 2024-04-20T10:17:22.072 INFO:teuthology.orchestra.run.smithi121.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3133.g 121 kB/s | 34 kB 00:00 2024-04-20T10:17:22.130 INFO:teuthology.orchestra.run.smithi123.stdout:(89/150): python3-requests-oauthlib-1.3.0-12.el 936 kB/s | 54 kB 00:00 2024-04-20T10:17:22.630 INFO:teuthology.orchestra.run.smithi123.stdout:(90/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-04-20T10:17:22.689 INFO:teuthology.orchestra.run.smithi123.stdout:(91/150): python3-toml-0.10.2-6.el9.noarch.rpm 729 kB/s | 42 kB 00:00 2024-04-20T10:17:22.747 INFO:teuthology.orchestra.run.smithi123.stdout:(92/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 768 kB/s | 44 kB 00:00 2024-04-20T10:17:22.748 INFO:teuthology.orchestra.run.smithi121.stdout:(29/150): ceph-mgr-cephadm-19.0.0-3133.ga8c84f5 210 kB/s | 142 kB 00:00 2024-04-20T10:17:22.823 INFO:teuthology.orchestra.run.smithi123.stdout:(93/150): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-04-20T10:17:22.873 INFO:teuthology.orchestra.run.smithi123.stdout:(94/150): qatlib-service-24.02.0-1.el9.x86_64.r 731 kB/s | 36 kB 00:00 2024-04-20T10:17:22.932 INFO:teuthology.orchestra.run.smithi123.stdout:(95/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 820 kB/s | 47 kB 00:00 2024-04-20T10:17:23.049 INFO:teuthology.orchestra.run.smithi123.stdout:(96/150): socat-1.7.4.1-5.el9.x86_64.rpm 2.6 MB/s | 305 kB 00:00 2024-04-20T10:17:23.107 INFO:teuthology.orchestra.run.smithi123.stdout:(97/150): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 58 kB 00:00 2024-04-20T10:17:23.174 INFO:teuthology.orchestra.run.smithi123.stdout:(98/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.4 MB/s | 94 kB 00:00 2024-04-20T10:17:23.233 INFO:teuthology.orchestra.run.smithi123.stdout:(99/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-20T10:17:23.292 INFO:teuthology.orchestra.run.smithi123.stdout:(100/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 780 kB/s | 45 kB 00:00 2024-04-20T10:17:23.458 INFO:teuthology.orchestra.run.smithi123.stdout:(101/150): lua-devel-5.4.4-4.el9.x86_64.rpm 134 kB/s | 22 kB 00:00 2024-04-20T10:17:23.617 INFO:teuthology.orchestra.run.smithi123.stdout:(102/150): protobuf-compiler-3.14.0-13.el9.x86_ 5.4 MB/s | 863 kB 00:00 2024-04-20T10:17:24.159 INFO:teuthology.orchestra.run.smithi123.stdout:(103/150): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-04-20T10:17:24.267 INFO:teuthology.orchestra.run.smithi123.stdout:(104/150): gperftools-libs-2.9.1-2.el9.x86_64.r 2.8 MB/s | 309 kB 00:00 2024-04-20T10:17:24.418 INFO:teuthology.orchestra.run.smithi123.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 130 kB/s | 19 kB 00:00 2024-04-20T10:17:24.818 INFO:teuthology.orchestra.run.smithi123.stdout:(106/150): libarrow-9.0.0-10.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-04-20T10:17:25.085 INFO:teuthology.orchestra.run.smithi123.stdout:(107/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 98 kB/s | 26 kB 00:00 2024-04-20T10:17:25.344 INFO:teuthology.orchestra.run.smithi123.stdout:(108/150): liboath-2.6.7-2.el9.x86_64.rpm 188 kB/s | 49 kB 00:00 2024-04-20T10:17:25.467 INFO:teuthology.orchestra.run.smithi123.stdout:(109/150): libunwind-1.6.2-1.el9.x86_64.rpm 550 kB/s | 67 kB 00:00 2024-04-20T10:17:25.569 INFO:teuthology.orchestra.run.smithi123.stdout:(110/150): luarocks-3.9.2-1.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-04-20T10:17:25.695 INFO:teuthology.orchestra.run.smithi123.stdout:(111/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 6.6 MB/s | 838 kB 00:00 2024-04-20T10:17:25.812 INFO:teuthology.orchestra.run.smithi123.stdout:(112/150): python3-asyncssh-2.13.2-5.el9.noarch 4.6 MB/s | 548 kB 00:00 2024-04-20T10:17:25.912 INFO:teuthology.orchestra.run.smithi123.stdout:(113/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 437 kB/s | 43 kB 00:00 2024-04-20T10:17:26.020 INFO:teuthology.orchestra.run.smithi123.stdout:(114/150): python3-cachetools-4.2.4-1.el9.noarc 299 kB/s | 32 kB 00:00 2024-04-20T10:17:26.121 INFO:teuthology.orchestra.run.smithi123.stdout:(115/150): python3-certifi-2021.10.8-2.el9.noar 151 kB/s | 15 kB 00:00 2024-04-20T10:17:26.221 INFO:teuthology.orchestra.run.smithi123.stdout:(116/150): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-04-20T10:17:26.329 INFO:teuthology.orchestra.run.smithi123.stdout:(117/150): python3-cherrypy-18.6.1-2.el9.noarch 3.3 MB/s | 358 kB 00:00 2024-04-20T10:17:26.455 INFO:teuthology.orchestra.run.smithi123.stdout:(118/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-04-20T10:17:26.547 INFO:teuthology.orchestra.run.smithi123.stdout:(119/150): python3-colorama-0.4.4-12.el9.noarch 389 kB/s | 35 kB 00:00 2024-04-20T10:17:26.647 INFO:teuthology.orchestra.run.smithi123.stdout:(120/150): python3-google-auth-2.28.2-1.el9.noa 2.0 MB/s | 203 kB 00:00 2024-04-20T10:17:26.805 INFO:teuthology.orchestra.run.smithi123.stdout:(121/150): python3-grpcio-1.46.7-10.el9.x86_64. 13 MB/s | 2.0 MB 00:00 2024-04-20T10:17:26.914 INFO:teuthology.orchestra.run.smithi123.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-04-20T10:17:27.015 INFO:teuthology.orchestra.run.smithi123.stdout:(123/150): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-04-20T10:17:27.115 INFO:teuthology.orchestra.run.smithi123.stdout:(124/150): python3-jaraco-classes-3.2.1-5.el9.n 177 kB/s | 18 kB 00:00 2024-04-20T10:17:27.215 INFO:teuthology.orchestra.run.smithi123.stdout:(125/150): python3-jaraco-collections-3.0.0-8.e 233 kB/s | 23 kB 00:00 2024-04-20T10:17:27.315 INFO:teuthology.orchestra.run.smithi123.stdout:(126/150): python3-jaraco-functools-3.5.0-2.el9 196 kB/s | 19 kB 00:00 2024-04-20T10:17:27.416 INFO:teuthology.orchestra.run.smithi123.stdout:(127/150): python3-jaraco-text-3.2.0-6.el9.noar 198 kB/s | 20 kB 00:00 2024-04-20T10:17:27.658 INFO:teuthology.orchestra.run.smithi123.stdout:(128/150): python3-kubernetes-26.1.0-2.el9.noar 4.2 MB/s | 1.0 MB 00:00 2024-04-20T10:17:27.758 INFO:teuthology.orchestra.run.smithi123.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 469 kB/s | 46 kB 00:00 2024-04-20T10:17:27.866 INFO:teuthology.orchestra.run.smithi123.stdout:(130/150): python3-more-itertools-8.12.0-2.el9. 733 kB/s | 79 kB 00:00 2024-04-20T10:17:27.966 INFO:teuthology.orchestra.run.smithi123.stdout:(131/150): python3-natsort-7.1.1-5.el9.noarch.r 582 kB/s | 58 kB 00:00 2024-04-20T10:17:28.083 INFO:teuthology.orchestra.run.smithi123.stdout:(132/150): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-04-20T10:17:28.184 INFO:teuthology.orchestra.run.smithi123.stdout:(133/150): python3-portend-3.1.0-2.el9.noarch.r 165 kB/s | 16 kB 00:00 2024-04-20T10:17:28.284 INFO:teuthology.orchestra.run.smithi123.stdout:(134/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 904 kB/s | 90 kB 00:00 2024-04-20T10:17:28.393 INFO:teuthology.orchestra.run.smithi123.stdout:(135/150): python3-pyperclip-1.8.2-2.el9.noarch 262 kB/s | 28 kB 00:00 2024-04-20T10:17:28.493 INFO:teuthology.orchestra.run.smithi123.stdout:(136/150): python3-repoze-lru-0.7-16.el9.noarch 310 kB/s | 31 kB 00:00 2024-04-20T10:17:28.594 INFO:teuthology.orchestra.run.smithi123.stdout:(137/150): python3-routes-2.5.1-5.el9.noarch.rp 1.9 MB/s | 188 kB 00:00 2024-04-20T10:17:28.777 INFO:teuthology.orchestra.run.smithi123.stdout:(138/150): python3-rsa-4.9-2.el9.noarch.rpm 324 kB/s | 59 kB 00:00 2024-04-20T10:17:28.877 INFO:teuthology.orchestra.run.smithi123.stdout:(139/150): python3-tempora-5.0.0-2.el9.noarch.r 360 kB/s | 36 kB 00:00 2024-04-20T10:17:28.978 INFO:teuthology.orchestra.run.smithi123.stdout:(140/150): python3-typing-extensions-4.4.0-2.el 512 kB/s | 51 kB 00:00 2024-04-20T10:17:29.086 INFO:teuthology.orchestra.run.smithi123.stdout:(141/150): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-04-20T10:17:29.187 INFO:teuthology.orchestra.run.smithi123.stdout:(142/150): python3-websocket-client-1.2.3-2.el9 903 kB/s | 90 kB 00:00 2024-04-20T10:17:29.328 INFO:teuthology.orchestra.run.smithi123.stdout:(143/150): python3-werkzeug-2.0.3-3.el9.1.noarc 3.0 MB/s | 427 kB 00:00 2024-04-20T10:17:29.420 INFO:teuthology.orchestra.run.smithi123.stdout:(144/150): python3-zc-lockfile-2.0-10.el9.noarc 220 kB/s | 20 kB 00:00 2024-04-20T10:17:29.588 INFO:teuthology.orchestra.run.smithi123.stdout:(145/150): re2-20211101-3.el9.x86_64.rpm 1.1 MB/s | 192 kB 00:00 2024-04-20T10:17:29.613 INFO:teuthology.orchestra.run.smithi121.stdout:(30/150): rbd-mirror-19.0.0-3133.ga8c84f59.el9. 348 kB/s | 3.1 MB 00:09 2024-04-20T10:17:29.746 INFO:teuthology.orchestra.run.smithi123.stdout:(146/150): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-04-20T10:17:31.846 INFO:teuthology.orchestra.run.smithi121.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3133.ga8c84 398 kB/s | 3.5 MB 00:09 2024-04-20T10:17:32.896 INFO:teuthology.orchestra.run.smithi121.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3133.ga8 237 kB/s | 249 kB 00:01 2024-04-20T10:17:33.088 INFO:teuthology.orchestra.run.smithi121.stdout:(33/150): ceph-mgr-rook-19.0.0-3133.ga8c84f59.e 259 kB/s | 49 kB 00:00 2024-04-20T10:17:33.196 INFO:teuthology.orchestra.run.smithi121.stdout:(34/150): ceph-prometheus-alerts-19.0.0-3133.ga 153 kB/s | 16 kB 00:00 2024-04-20T10:17:33.847 INFO:teuthology.orchestra.run.smithi121.stdout:(35/150): ceph-volume-19.0.0-3133.ga8c84f59.el9 421 kB/s | 273 kB 00:00 2024-04-20T10:17:35.355 INFO:teuthology.orchestra.run.smithi121.stdout:(36/150): cephadm-19.0.0-3133.ga8c84f59.el9.noa 506 kB/s | 763 kB 00:01 2024-04-20T10:17:35.464 INFO:teuthology.orchestra.run.smithi121.stdout:(37/150): cephfs-top-19.0.0-3133.ga8c84f59.el9. 169 kB/s | 18 kB 00:00 2024-04-20T10:17:35.581 INFO:teuthology.orchestra.run.smithi121.stdout:(38/150): fuse-2.9.9-15.el9.x86_64.rpm 695 kB/s | 80 kB 00:00 2024-04-20T10:17:35.687 INFO:teuthology.orchestra.run.smithi123.stdout:(147/150): ceph-mgr-diskprediction-local-19.0.0 193 kB/s | 7.4 MB 00:39 2024-04-20T10:17:35.689 INFO:teuthology.orchestra.run.smithi121.stdout:(39/150): libgfortran-11.4.1-3.el9.x86_64.rpm 7.3 MB/s | 803 kB 00:00 2024-04-20T10:17:35.731 INFO:teuthology.orchestra.run.smithi121.stdout:(40/150): libquadmath-11.4.1-3.el9.x86_64.rpm 4.6 MB/s | 195 kB 00:00 2024-04-20T10:17:35.781 INFO:teuthology.orchestra.run.smithi121.stdout:(41/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-04-20T10:17:35.865 INFO:teuthology.orchestra.run.smithi121.stdout:(42/150): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-04-20T10:17:35.907 INFO:teuthology.orchestra.run.smithi121.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-04-20T10:17:35.949 INFO:teuthology.orchestra.run.smithi121.stdout:(44/150): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-04-20T10:17:35.991 INFO:teuthology.orchestra.run.smithi121.stdout:(45/150): python3-requests-2.25.1-8.el9.noarch. 3.0 MB/s | 125 kB 00:00 2024-04-20T10:17:36.041 INFO:teuthology.orchestra.run.smithi121.stdout:(46/150): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-20T10:17:36.233 INFO:teuthology.orchestra.run.smithi121.stdout:(47/150): automake-1.16.2-8.el9.noarch.rpm 3.5 MB/s | 689 kB 00:00 2024-04-20T10:17:36.458 INFO:teuthology.orchestra.run.smithi121.stdout:(48/150): bison-3.7.4-5.el9.x86_64.rpm 4.1 MB/s | 943 kB 00:00 2024-04-20T10:17:36.508 INFO:teuthology.orchestra.run.smithi121.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-04-20T10:17:36.551 INFO:teuthology.orchestra.run.smithi121.stdout:(50/150): elfutils-libelf-devel-0.190-2.el9.x86 519 kB/s | 21 kB 00:00 2024-04-20T10:17:36.601 INFO:teuthology.orchestra.run.smithi121.stdout:(51/150): flex-2.6.4-9.el9.x86_64.rpm 6.2 MB/s | 313 kB 00:00 2024-04-20T10:17:36.651 INFO:teuthology.orchestra.run.smithi121.stdout:(52/150): flexiblas-3.0.4-8.el9.x86_64.rpm 657 kB/s | 33 kB 00:00 2024-04-20T10:17:36.777 INFO:teuthology.orchestra.run.smithi121.stdout:(53/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-04-20T10:17:36.855 INFO:teuthology.orchestra.run.smithi121.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 236 kB/s | 18 kB 00:00 2024-04-20T10:17:36.902 INFO:teuthology.orchestra.run.smithi121.stdout:(55/150): libaio-devel-0.3.111-13.el9.x86_64.rp 241 kB/s | 11 kB 00:00 2024-04-20T10:17:36.986 INFO:teuthology.orchestra.run.smithi121.stdout:(56/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-04-20T10:17:37.036 INFO:teuthology.orchestra.run.smithi121.stdout:(57/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 915 kB/s | 45 kB 00:00 2024-04-20T10:17:37.136 INFO:teuthology.orchestra.run.smithi121.stdout:(58/150): librdkafka-1.6.1-102.el9.x86_64.rpm 6.5 MB/s | 662 kB 00:00 2024-04-20T10:17:37.195 INFO:teuthology.orchestra.run.smithi121.stdout:(59/150): libtool-2.4.6-45.el9.x86_64.rpm 9.9 MB/s | 582 kB 00:00 2024-04-20T10:17:37.245 INFO:teuthology.orchestra.run.smithi121.stdout:(60/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 516 kB/s | 25 kB 00:00 2024-04-20T10:17:37.295 INFO:teuthology.orchestra.run.smithi121.stdout:(61/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 962 kB/s | 47 kB 00:00 2024-04-20T10:17:37.429 INFO:teuthology.orchestra.run.smithi121.stdout:(62/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.2 MB/s | 292 kB 00:00 2024-04-20T10:17:37.541 INFO:teuthology.orchestra.run.smithi121.stdout:(63/150): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-04-20T10:17:37.596 INFO:teuthology.orchestra.run.smithi121.stdout:(64/150): openblas-0.3.21-2.el9.x86_64.rpm 650 kB/s | 35 kB 00:00 2024-04-20T10:17:37.822 INFO:teuthology.orchestra.run.smithi121.stdout:(65/150): openblas-openmp-0.3.21-2.el9.x86_64.r 21 MB/s | 4.7 MB 00:00 2024-04-20T10:17:37.964 INFO:teuthology.orchestra.run.smithi121.stdout:(66/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 29 MB/s | 4.1 MB 00:00 2024-04-20T10:17:38.056 INFO:teuthology.orchestra.run.smithi121.stdout:(67/150): postgresql-13.14-1.el9.x86_64.rpm 18 MB/s | 1.6 MB 00:00 2024-04-20T10:17:38.173 INFO:teuthology.orchestra.run.smithi121.stdout:(68/150): postgresql-contrib-13.14-1.el9.x86_64 7.7 MB/s | 911 kB 00:00 2024-04-20T10:17:38.265 INFO:teuthology.orchestra.run.smithi121.stdout:(69/150): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-04-20T10:17:38.440 INFO:teuthology.orchestra.run.smithi121.stdout:(70/150): postgresql-server-13.14-1.el9.x86_64. 33 MB/s | 5.8 MB 00:00 2024-04-20T10:17:38.524 INFO:teuthology.orchestra.run.smithi121.stdout:(71/150): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-04-20T10:17:38.574 INFO:teuthology.orchestra.run.smithi121.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-04-20T10:17:38.612 INFO:teuthology.orchestra.run.smithi123.stdout:(148/150): librados2-19.0.0-3133.ga8c84f59.el9. 392 kB/s | 3.4 MB 00:08 2024-04-20T10:17:38.758 INFO:teuthology.orchestra.run.smithi121.stdout:(73/150): python3-babel-2.9.1-2.el9.noarch.rpm 33 MB/s | 6.0 MB 00:00 2024-04-20T10:17:38.808 INFO:teuthology.orchestra.run.smithi121.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 4.9 MB/s | 244 kB 00:00 2024-04-20T10:17:38.858 INFO:teuthology.orchestra.run.smithi121.stdout:(75/150): python3-jinja2-2.11.3-5.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-04-20T10:17:38.908 INFO:teuthology.orchestra.run.smithi121.stdout:(76/150): python3-jsonpatch-1.21-16.el9.noarch. 518 kB/s | 26 kB 00:00 2024-04-20T10:17:38.967 INFO:teuthology.orchestra.run.smithi121.stdout:(77/150): python3-jsonpointer-2.0-4.el9.noarch. 332 kB/s | 19 kB 00:00 2024-04-20T10:17:39.017 INFO:teuthology.orchestra.run.smithi121.stdout:(78/150): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-20T10:17:39.151 INFO:teuthology.orchestra.run.smithi121.stdout:(79/150): python3-markupsafe-1.1.1-12.el9.x86_6 261 kB/s | 35 kB 00:00 2024-04-20T10:17:39.376 INFO:teuthology.orchestra.run.smithi121.stdout:(80/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 5.1 MB 00:00 2024-04-20T10:17:39.435 INFO:teuthology.orchestra.run.smithi121.stdout:(81/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.3 MB/s | 250 kB 00:00 2024-04-20T10:17:39.485 INFO:teuthology.orchestra.run.smithi121.stdout:(82/150): python3-oauthlib-3.1.1-5.el9.noarch.r 4.4 MB/s | 222 kB 00:00 2024-04-20T10:17:39.585 INFO:teuthology.orchestra.run.smithi121.stdout:(83/150): python3-packaging-20.9-5.el9.noarch.r 777 kB/s | 77 kB 00:00 2024-04-20T10:17:39.669 INFO:teuthology.orchestra.run.smithi121.stdout:(84/150): python3-prettytable-0.7.2-27.el9.noar 512 kB/s | 42 kB 00:00 2024-04-20T10:17:39.719 INFO:teuthology.orchestra.run.smithi121.stdout:(85/150): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-04-20T10:17:39.778 INFO:teuthology.orchestra.run.smithi121.stdout:(86/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-04-20T10:17:39.836 INFO:teuthology.orchestra.run.smithi121.stdout:(87/150): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-04-20T10:17:39.887 INFO:teuthology.orchestra.run.smithi121.stdout:(88/150): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-20T10:17:39.937 INFO:teuthology.orchestra.run.smithi121.stdout:(89/150): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-04-20T10:17:40.296 INFO:teuthology.orchestra.run.smithi121.stdout:(90/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 45 MB/s | 16 MB 00:00 2024-04-20T10:17:40.371 INFO:teuthology.orchestra.run.smithi121.stdout:(91/150): python3-toml-0.10.2-6.el9.noarch.rpm 565 kB/s | 42 kB 00:00 2024-04-20T10:17:40.422 INFO:teuthology.orchestra.run.smithi121.stdout:(92/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 890 kB/s | 44 kB 00:00 2024-04-20T10:17:40.480 INFO:teuthology.orchestra.run.smithi121.stdout:(93/150): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-20T10:17:40.530 INFO:teuthology.orchestra.run.smithi121.stdout:(94/150): qatlib-service-24.02.0-1.el9.x86_64.r 729 kB/s | 36 kB 00:00 2024-04-20T10:17:40.580 INFO:teuthology.orchestra.run.smithi121.stdout:(95/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 963 kB/s | 47 kB 00:00 2024-04-20T10:17:40.639 INFO:teuthology.orchestra.run.smithi121.stdout:(96/150): socat-1.7.4.1-5.el9.x86_64.rpm 5.2 MB/s | 305 kB 00:00 2024-04-20T10:17:40.706 INFO:teuthology.orchestra.run.smithi121.stdout:(97/150): uuid-1.6.2-55.el9.x86_64.rpm 875 kB/s | 58 kB 00:00 2024-04-20T10:17:40.757 INFO:teuthology.orchestra.run.smithi121.stdout:(98/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.9 MB/s | 94 kB 00:00 2024-04-20T10:17:40.807 INFO:teuthology.orchestra.run.smithi121.stdout:(99/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-04-20T10:17:40.857 INFO:teuthology.orchestra.run.smithi121.stdout:(100/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 905 kB/s | 45 kB 00:00 2024-04-20T10:17:40.949 INFO:teuthology.orchestra.run.smithi121.stdout:(101/150): lua-devel-5.4.4-4.el9.x86_64.rpm 245 kB/s | 22 kB 00:00 2024-04-20T10:17:41.224 INFO:teuthology.orchestra.run.smithi121.stdout:(102/150): protobuf-compiler-3.14.0-13.el9.x86_ 3.1 MB/s | 863 kB 00:00 2024-04-20T10:17:41.773 INFO:teuthology.orchestra.run.smithi121.stdout:(103/150): abseil-cpp-20211102.0-3.el9.x86_64.r 1.0 MB/s | 548 kB 00:00 2024-04-20T10:17:41.875 INFO:teuthology.orchestra.run.smithi121.stdout:(104/150): gperftools-libs-2.9.1-2.el9.x86_64.r 3.0 MB/s | 309 kB 00:00 2024-04-20T10:17:41.975 INFO:teuthology.orchestra.run.smithi121.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 196 kB/s | 19 kB 00:00 2024-04-20T10:17:42.500 INFO:teuthology.orchestra.run.smithi121.stdout:(106/150): libarrow-9.0.0-10.el9.x86_64.rpm 8.4 MB/s | 4.4 MB 00:00 2024-04-20T10:17:42.601 INFO:teuthology.orchestra.run.smithi121.stdout:(107/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 262 kB/s | 26 kB 00:00 2024-04-20T10:17:42.701 INFO:teuthology.orchestra.run.smithi121.stdout:(108/150): liboath-2.6.7-2.el9.x86_64.rpm 488 kB/s | 49 kB 00:00 2024-04-20T10:17:42.809 INFO:teuthology.orchestra.run.smithi121.stdout:(109/150): libunwind-1.6.2-1.el9.x86_64.rpm 626 kB/s | 67 kB 00:00 2024-04-20T10:17:42.910 INFO:teuthology.orchestra.run.smithi121.stdout:(110/150): luarocks-3.9.2-1.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-04-20T10:17:43.035 INFO:teuthology.orchestra.run.smithi121.stdout:(111/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 6.6 MB/s | 838 kB 00:00 2024-04-20T10:17:43.170 INFO:teuthology.orchestra.run.smithi121.stdout:(112/150): python3-asyncssh-2.13.2-5.el9.noarch 4.0 MB/s | 548 kB 00:00 2024-04-20T10:17:43.303 INFO:teuthology.orchestra.run.smithi121.stdout:(113/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 331 kB/s | 43 kB 00:00 2024-04-20T10:17:43.402 INFO:teuthology.orchestra.run.smithi121.stdout:(114/150): python3-cachetools-4.2.4-1.el9.noarc 324 kB/s | 32 kB 00:00 2024-04-20T10:17:43.536 INFO:teuthology.orchestra.run.smithi121.stdout:(115/150): python3-certifi-2021.10.8-2.el9.noar 113 kB/s | 15 kB 00:00 2024-04-20T10:17:43.670 INFO:teuthology.orchestra.run.smithi121.stdout:(116/150): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-04-20T10:17:43.779 INFO:teuthology.orchestra.run.smithi121.stdout:(117/150): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-04-20T10:17:43.904 INFO:teuthology.orchestra.run.smithi121.stdout:(118/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.4 MB/s | 303 kB 00:00 2024-04-20T10:17:44.004 INFO:teuthology.orchestra.run.smithi121.stdout:(119/150): python3-colorama-0.4.4-12.el9.noarch 354 kB/s | 35 kB 00:00 2024-04-20T10:17:44.105 INFO:teuthology.orchestra.run.smithi121.stdout:(120/150): python3-google-auth-2.28.2-1.el9.noa 2.0 MB/s | 203 kB 00:00 2024-04-20T10:17:44.372 INFO:teuthology.orchestra.run.smithi121.stdout:(121/150): python3-grpcio-1.46.7-10.el9.x86_64. 7.7 MB/s | 2.0 MB 00:00 2024-04-20T10:17:44.472 INFO:teuthology.orchestra.run.smithi121.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-04-20T10:17:44.589 INFO:teuthology.orchestra.run.smithi121.stdout:(123/150): python3-jaraco-8.2.1-3.el9.noarch.rp 92 kB/s | 11 kB 00:00 2024-04-20T10:17:44.681 INFO:teuthology.orchestra.run.smithi121.stdout:(124/150): python3-jaraco-classes-3.2.1-5.el9.n 194 kB/s | 18 kB 00:00 2024-04-20T10:17:44.764 INFO:teuthology.orchestra.run.smithi121.stdout:(125/150): python3-jaraco-collections-3.0.0-8.e 280 kB/s | 23 kB 00:00 2024-04-20T10:17:44.848 INFO:teuthology.orchestra.run.smithi121.stdout:(126/150): python3-jaraco-functools-3.5.0-2.el9 235 kB/s | 19 kB 00:00 2024-04-20T10:17:44.932 INFO:teuthology.orchestra.run.smithi121.stdout:(127/150): python3-jaraco-text-3.2.0-6.el9.noar 238 kB/s | 20 kB 00:00 2024-04-20T10:17:45.057 INFO:teuthology.orchestra.run.smithi121.stdout:(128/150): python3-kubernetes-26.1.0-2.el9.noar 8.2 MB/s | 1.0 MB 00:00 2024-04-20T10:17:45.141 INFO:teuthology.orchestra.run.smithi121.stdout:(129/150): python3-logutils-0.3.5-21.el9.noarch 561 kB/s | 46 kB 00:00 2024-04-20T10:17:45.233 INFO:teuthology.orchestra.run.smithi121.stdout:(130/150): python3-more-itertools-8.12.0-2.el9. 865 kB/s | 79 kB 00:00 2024-04-20T10:17:45.333 INFO:teuthology.orchestra.run.smithi121.stdout:(131/150): python3-natsort-7.1.1-5.el9.noarch.r 581 kB/s | 58 kB 00:00 2024-04-20T10:17:45.433 INFO:teuthology.orchestra.run.smithi121.stdout:(132/150): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-04-20T10:17:45.559 INFO:teuthology.orchestra.run.smithi121.stdout:(133/150): python3-portend-3.1.0-2.el9.noarch.r 132 kB/s | 16 kB 00:00 2024-04-20T10:17:45.701 INFO:teuthology.orchestra.run.smithi121.stdout:(134/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 639 kB/s | 90 kB 00:00 2024-04-20T10:17:45.826 INFO:teuthology.orchestra.run.smithi121.stdout:(135/150): python3-pyperclip-1.8.2-2.el9.noarch 227 kB/s | 28 kB 00:00 2024-04-20T10:17:45.926 INFO:teuthology.orchestra.run.smithi121.stdout:(136/150): python3-repoze-lru-0.7-16.el9.noarch 309 kB/s | 31 kB 00:00 2024-04-20T10:17:46.026 INFO:teuthology.orchestra.run.smithi121.stdout:(137/150): python3-routes-2.5.1-5.el9.noarch.rp 1.9 MB/s | 188 kB 00:00 2024-04-20T10:17:46.135 INFO:teuthology.orchestra.run.smithi121.stdout:(138/150): python3-rsa-4.9-2.el9.noarch.rpm 550 kB/s | 59 kB 00:00 2024-04-20T10:17:46.227 INFO:teuthology.orchestra.run.smithi121.stdout:(139/150): python3-tempora-5.0.0-2.el9.noarch.r 394 kB/s | 36 kB 00:00 2024-04-20T10:17:46.319 INFO:teuthology.orchestra.run.smithi121.stdout:(140/150): python3-typing-extensions-4.4.0-2.el 562 kB/s | 51 kB 00:00 2024-04-20T10:17:46.419 INFO:teuthology.orchestra.run.smithi121.stdout:(141/150): python3-webob-1.8.7-6.el9.noarch.rpm 2.3 MB/s | 230 kB 00:00 2024-04-20T10:17:46.520 INFO:teuthology.orchestra.run.smithi121.stdout:(142/150): python3-websocket-client-1.2.3-2.el9 903 kB/s | 90 kB 00:00 2024-04-20T10:17:46.628 INFO:teuthology.orchestra.run.smithi121.stdout:(143/150): python3-werkzeug-2.0.3-3.el9.1.noarc 3.9 MB/s | 427 kB 00:00 2024-04-20T10:17:46.728 INFO:teuthology.orchestra.run.smithi121.stdout:(144/150): python3-zc-lockfile-2.0-10.el9.noarc 201 kB/s | 20 kB 00:00 2024-04-20T10:17:46.829 INFO:teuthology.orchestra.run.smithi121.stdout:(145/150): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-04-20T10:17:47.029 INFO:teuthology.orchestra.run.smithi121.stdout:(146/150): thrift-0.15.0-2.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2024-04-20T10:17:52.411 INFO:teuthology.orchestra.run.smithi123.stdout:(149/150): librbd1-19.0.0-3133.ga8c84f59.el9.x8 192 kB/s | 3.1 MB 00:16 2024-04-20T10:17:54.828 INFO:teuthology.orchestra.run.smithi121.stdout:(147/150): ceph-mgr-diskprediction-local-19.0.0 301 kB/s | 7.4 MB 00:25 2024-04-20T10:17:55.370 INFO:teuthology.orchestra.run.smithi121.stdout:(148/150): librados2-19.0.0-3133.ga8c84f59.el9. 417 kB/s | 3.4 MB 00:08 2024-04-20T10:18:01.786 INFO:teuthology.orchestra.run.smithi121.stdout:(149/150): librbd1-19.0.0-3133.ga8c84f59.el9.x8 462 kB/s | 3.1 MB 00:06 2024-04-20T10:18:25.582 INFO:teuthology.orchestra.run.smithi123.stdout:(150/150): ceph-test-19.0.0-3133.ga8c84f59.el9. 388 kB/s | 49 MB 02:09 2024-04-20T10:18:25.585 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:18:25.585 INFO:teuthology.orchestra.run.smithi123.stdout:Total 1.2 MB/s | 218 MB 02:57 2024-04-20T10:18:28.187 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-04-20T10:18:28.398 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-04-20T10:18:28.398 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-04-20T10:18:30.973 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-04-20T10:18:30.974 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-04-20T10:18:37.071 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-04-20T10:18:37.156 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-20T10:18:37.194 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-20T10:18:37.329 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-20T10:18:37.736 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-20T10:18:37.843 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T10:18:38.017 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T10:18:38.044 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T10:18:38.172 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T10:18:38.236 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T10:18:38.276 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-20T10:18:38.312 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-20T10:18:38.718 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-20T10:18:38.782 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-20T10:18:38.807 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T10:18:38.968 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T10:18:39.000 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T10:18:39.130 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T10:18:39.159 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T10:18:39.227 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T10:18:39.712 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T10:18:39.768 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T10:18:39.964 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-20T10:18:40.088 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-20T10:18:40.126 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-20T10:18:40.173 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-20T10:18:40.286 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-20T10:18:40.361 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-20T10:18:40.431 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-20T10:18:40.488 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-20T10:18:40.526 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-20T10:18:40.645 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-20T10:18:40.715 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-20T10:18:40.752 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-20T10:18:40.818 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-20T10:18:40.925 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-20T10:18:40.963 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 30/152 2024-04-20T10:18:41.016 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 31/152 2024-04-20T10:18:41.053 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 32/152 2024-04-20T10:18:41.182 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-20T10:18:41.239 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-20T10:18:41.337 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-20T10:18:41.521 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-20T10:18:41.586 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-20T10:18:41.970 INFO:teuthology.orchestra.run.smithi121.stdout:(150/150): ceph-test-19.0.0-3133.ga8c84f59.el9. 441 kB/s | 49 MB 01:53 2024-04-20T10:18:41.977 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:18:41.978 INFO:teuthology.orchestra.run.smithi121.stdout:Total 1.3 MB/s | 218 MB 02:48 2024-04-20T10:18:42.289 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-20T10:18:42.399 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-20T10:18:42.434 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-20T10:18:42.479 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-20T10:18:42.528 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 42/152 2024-04-20T10:18:42.571 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-20T10:18:42.653 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-20T10:18:42.778 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-20T10:18:42.819 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-20T10:18:42.858 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-20T10:18:42.928 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-20T10:18:42.975 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-20T10:18:43.037 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-20T10:18:43.095 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-20T10:18:43.295 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-20T10:18:43.356 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-20T10:18:43.394 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-20T10:18:43.451 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-20T10:18:43.591 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-20T10:18:43.634 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-20T10:18:43.690 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-20T10:18:43.742 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-20T10:18:43.775 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-20T10:18:43.970 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-20T10:18:44.144 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-20T10:18:44.213 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-20T10:18:44.261 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-20T10:18:44.319 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-20T10:18:44.413 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-20T10:18:44.474 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-20T10:18:44.527 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-20T10:18:44.569 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-20T10:18:44.613 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-20T10:18:44.663 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-20T10:18:44.748 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-20T10:18:44.754 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-04-20T10:18:44.795 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-20T10:18:44.939 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-20T10:18:44.972 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-04-20T10:18:44.972 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-04-20T10:18:44.999 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-20T10:18:45.044 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-20T10:18:45.114 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-20T10:18:45.291 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-20T10:18:45.357 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-20T10:18:46.336 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-20T10:18:46.389 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-20T10:18:46.422 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-20T10:18:46.548 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-20T10:18:46.609 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-20T10:18:46.714 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-20T10:18:46.749 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-20T10:18:46.976 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-20T10:18:47.013 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-20T10:18:47.120 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-20T10:18:47.534 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-04-20T10:18:47.535 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-04-20T10:18:47.968 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-20T10:18:48.338 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-20T10:18:49.976 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-20T10:18:50.077 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-20T10:18:50.118 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-20T10:18:50.160 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-20T10:18:50.559 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-20T10:18:50.597 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T10:18:50.721 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T10:18:50.754 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 98/152 2024-04-20T10:18:50.813 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-20T10:18:51.398 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-20T10:18:51.433 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T10:18:51.535 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T10:18:51.560 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 102/152 2024-04-20T10:18:53.656 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-04-20T10:18:53.753 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-20T10:18:53.798 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-20T10:18:53.941 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-20T10:18:54.251 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T10:18:54.300 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T10:18:54.357 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-20T10:18:54.472 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T10:18:54.638 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T10:18:54.664 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T10:18:54.789 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T10:18:54.844 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T10:18:54.888 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-20T10:18:54.925 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-20T10:18:54.933 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T10:18:54.986 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T10:18:55.132 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T10:18:55.132 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-20T10:18:55.132 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-20T10:18:55.132 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:18:55.170 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T10:18:55.357 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-20T10:18:55.428 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-20T10:18:55.453 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T10:18:55.604 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T10:18:55.637 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T10:18:55.741 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T10:18:55.772 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T10:18:55.838 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T10:18:56.258 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T10:18:56.301 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T10:18:56.481 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-20T10:18:56.603 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-20T10:18:56.647 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-20T10:18:56.696 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-20T10:18:56.798 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-20T10:18:56.843 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-20T10:18:56.903 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-20T10:18:56.948 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-20T10:18:56.987 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-20T10:18:57.103 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-20T10:18:57.174 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-20T10:18:57.215 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-20T10:18:57.269 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-20T10:18:57.380 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-20T10:18:57.418 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 30/152 2024-04-20T10:18:57.466 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 31/152 2024-04-20T10:18:57.499 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 32/152 2024-04-20T10:18:57.617 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-20T10:18:57.677 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-20T10:18:57.776 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-20T10:18:57.956 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-20T10:18:58.028 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-20T10:18:58.670 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-20T10:18:58.779 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-20T10:18:58.814 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-20T10:18:58.859 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-20T10:18:58.900 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 42/152 2024-04-20T10:18:58.941 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-20T10:18:59.024 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-20T10:18:59.123 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-20T10:18:59.163 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-20T10:18:59.232 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-20T10:18:59.307 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-20T10:18:59.354 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-20T10:18:59.416 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-20T10:18:59.485 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-20T10:18:59.683 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-20T10:18:59.743 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-20T10:18:59.781 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-20T10:18:59.838 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-20T10:18:59.977 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-20T10:19:00.021 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-20T10:19:00.075 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-20T10:19:00.112 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-20T10:19:00.145 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-20T10:19:00.346 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-20T10:19:00.526 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-20T10:19:00.568 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-20T10:19:00.627 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-20T10:19:00.686 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-20T10:19:00.764 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-20T10:19:00.830 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-20T10:19:00.883 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-20T10:19:00.932 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-20T10:19:00.978 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-20T10:19:01.027 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-20T10:19:01.103 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-20T10:19:01.153 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-20T10:19:01.294 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-20T10:19:01.373 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-20T10:19:01.417 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-20T10:19:01.488 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-20T10:19:01.634 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-20T10:19:01.706 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-20T10:19:02.670 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-20T10:19:02.754 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-20T10:19:02.788 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-20T10:19:02.902 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-20T10:19:02.965 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-20T10:19:03.062 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-20T10:19:03.105 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-20T10:19:03.334 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-20T10:19:03.378 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-20T10:19:03.489 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-20T10:19:04.310 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-20T10:19:04.458 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-20T10:19:06.081 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-20T10:19:06.184 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-20T10:19:06.225 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-20T10:19:06.284 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-20T10:19:06.648 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-20T10:19:06.688 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T10:19:06.792 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T10:19:06.828 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 98/152 2024-04-20T10:19:06.884 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-20T10:19:07.430 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-20T10:19:07.465 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T10:19:07.575 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T10:19:07.601 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 102/152 2024-04-20T10:19:07.756 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T10:19:07.756 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /sys 2024-04-20T10:19:07.756 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /proc 2024-04-20T10:19:07.756 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /mnt 2024-04-20T10:19:07.756 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /var/tmp 2024-04-20T10:19:07.756 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /home 2024-04-20T10:19:07.756 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /root 2024-04-20T10:19:07.756 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /tmp 2024-04-20T10:19:07.756 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:07.981 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T10:19:08.034 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T10:19:08.035 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:08.035 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-20T10:19:08.035 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:09.668 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T10:19:09.746 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T10:19:09.747 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:09.747 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-20T10:19:09.747 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T10:19:09.747 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T10:19:09.747 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:10.001 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T10:19:10.067 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T10:19:10.067 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:10.067 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-20T10:19:10.067 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T10:19:10.068 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T10:19:10.068 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:10.279 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T10:19:10.330 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T10:19:10.543 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T10:19:10.613 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T10:19:10.613 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:10.613 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-20T10:19:10.613 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T10:19:10.613 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T10:19:10.613 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:10.744 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-20T10:19:10.804 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-20T10:19:10.880 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-20T10:19:10.960 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T10:19:11.017 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T10:19:11.139 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-20T10:19:11.178 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T10:19:11.178 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-20T10:19:11.178 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-20T10:19:11.178 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:11.197 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-20T10:19:11.226 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T10:19:11.283 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-20T10:19:11.383 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-20T10:19:11.546 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-20T10:19:11.588 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-20T10:19:11.629 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-20T10:19:11.654 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-20T10:19:12.273 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T10:19:12.306 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T10:19:12.358 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 122/152 2024-04-20T10:19:12.436 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 123/152 2024-04-20T10:19:12.476 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T10:19:13.738 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T10:19:13.777 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T10:19:14.871 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T10:19:14.902 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T10:19:15.048 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T10:19:15.229 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 127/152 2024-04-20T10:19:15.254 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T10:19:15.322 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T10:19:15.323 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:15.323 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-20T10:19:15.323 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T10:19:15.323 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T10:19:15.323 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:15.380 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T10:19:15.417 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T10:19:15.552 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/152 2024-04-20T10:19:15.613 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/152 2024-04-20T10:19:16.622 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-20T10:19:16.658 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T10:19:16.725 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T10:19:16.726 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:16.726 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-20T10:19:16.726 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T10:19:16.726 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T10:19:16.726 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:16.783 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T10:19:16.852 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T10:19:16.852 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:16.852 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-20T10:19:16.852 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:16.917 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T10:19:16.990 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T10:19:16.990 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:16.990 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-20T10:19:16.990 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T10:19:16.990 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T10:19:16.990 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:17.334 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T10:19:17.402 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T10:19:17.402 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:17.402 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-20T10:19:17.402 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T10:19:17.402 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T10:19:17.402 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:22.364 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 137/152 2024-04-20T10:19:22.408 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 138/152 2024-04-20T10:19:22.515 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 139/152 2024-04-20T10:19:22.561 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-20T10:19:22.601 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-20T10:19:22.855 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 142/152 2024-04-20T10:19:22.902 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-20T10:19:22.930 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 144/152 2024-04-20T10:19:23.813 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T10:19:23.862 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T10:19:23.911 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T10:19:24.126 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T10:19:24.126 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /sys 2024-04-20T10:19:24.127 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /proc 2024-04-20T10:19:24.127 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /mnt 2024-04-20T10:19:24.127 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /var/tmp 2024-04-20T10:19:24.127 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /home 2024-04-20T10:19:24.127 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /root 2024-04-20T10:19:24.127 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /tmp 2024-04-20T10:19:24.127 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:24.356 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T10:19:24.399 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T10:19:24.399 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:24.399 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-20T10:19:24.399 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:25.141 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 146/152 2024-04-20T10:19:25.242 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-20T10:19:25.316 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-20T10:19:25.486 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-20T10:19:25.523 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-20T10:19:25.523 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T10:19:25.648 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T10:19:25.648 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T10:19:26.016 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T10:19:26.115 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T10:19:26.115 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:26.115 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-20T10:19:26.115 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T10:19:26.115 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T10:19:26.116 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:26.155 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T10:19:26.155 INFO:teuthology.orchestra.run.smithi123.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-20T10:19:26.155 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:26.366 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T10:19:26.457 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T10:19:26.458 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:26.458 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-20T10:19:26.458 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T10:19:26.458 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T10:19:26.458 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:26.959 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T10:19:27.051 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T10:19:27.051 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:27.051 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-20T10:19:27.051 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T10:19:27.051 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T10:19:27.051 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:27.193 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-20T10:19:27.259 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-20T10:19:27.341 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-20T10:19:27.606 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-20T10:19:27.665 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-20T10:19:27.754 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-20T10:19:27.863 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-20T10:19:28.039 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-20T10:19:28.114 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-20T10:19:28.155 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-20T10:19:28.186 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-20T10:19:28.839 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T10:19:28.874 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T10:19:28.937 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 122/152 2024-04-20T10:19:29.059 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 123/152 2024-04-20T10:19:29.092 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T10:19:30.402 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T10:19:30.452 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T10:19:31.546 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T10:19:31.598 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T10:19:31.747 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T10:19:31.916 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 1/152 2024-04-20T10:19:31.916 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2/152 2024-04-20T10:19:31.916 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 3/152 2024-04-20T10:19:31.916 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 4/152 2024-04-20T10:19:31.916 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 5/152 2024-04-20T10:19:31.916 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T10:19:31.917 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T10:19:31.917 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 8/152 2024-04-20T10:19:31.917 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 9/152 2024-04-20T10:19:31.917 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/152 2024-04-20T10:19:31.917 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/152 2024-04-20T10:19:31.917 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T10:19:31.917 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 13/152 2024-04-20T10:19:31.917 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 14/152 2024-04-20T10:19:31.917 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 15/152 2024-04-20T10:19:31.917 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/152 2024-04-20T10:19:31.917 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 17/152 2024-04-20T10:19:31.917 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 18/152 2024-04-20T10:19:31.917 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 19/152 2024-04-20T10:19:31.918 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 20/152 2024-04-20T10:19:31.918 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 21/152 2024-04-20T10:19:31.918 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 22/152 2024-04-20T10:19:31.918 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 23/152 2024-04-20T10:19:31.918 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 24/152 2024-04-20T10:19:31.918 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 25/152 2024-04-20T10:19:31.930 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 26/152 2024-04-20T10:19:31.930 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 27/152 2024-04-20T10:19:31.930 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 28/152 2024-04-20T10:19:31.930 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 29/152 2024-04-20T10:19:31.930 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 30/152 2024-04-20T10:19:31.930 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 31/152 2024-04-20T10:19:31.930 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 32/152 2024-04-20T10:19:31.930 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 33/152 2024-04-20T10:19:31.930 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 34/152 2024-04-20T10:19:31.931 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 35/152 2024-04-20T10:19:31.931 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 36/152 2024-04-20T10:19:31.931 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 37/152 2024-04-20T10:19:31.931 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 38/152 2024-04-20T10:19:31.931 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 39/152 2024-04-20T10:19:31.931 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-20T10:19:31.931 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-20T10:19:31.931 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-20T10:19:31.931 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-20T10:19:31.931 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-20T10:19:31.931 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-20T10:19:31.931 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-20T10:19:31.932 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-20T10:19:31.932 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-20T10:19:31.932 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-20T10:19:31.932 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-20T10:19:31.932 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-20T10:19:31.932 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-20T10:19:31.932 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-20T10:19:31.932 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-20T10:19:31.932 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-20T10:19:31.932 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-20T10:19:31.932 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-20T10:19:31.932 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-20T10:19:31.933 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-20T10:19:31.933 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-20T10:19:31.933 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-20T10:19:31.933 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-20T10:19:31.933 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-20T10:19:31.933 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-20T10:19:31.933 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-20T10:19:31.933 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-20T10:19:31.933 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-20T10:19:31.933 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-20T10:19:31.933 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-20T10:19:31.933 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-20T10:19:31.934 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-20T10:19:31.934 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-20T10:19:31.934 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-20T10:19:31.934 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-20T10:19:31.934 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-20T10:19:31.934 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-20T10:19:31.934 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-20T10:19:31.934 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-20T10:19:31.934 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-20T10:19:31.934 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-20T10:19:31.935 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-20T10:19:31.935 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-20T10:19:31.935 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-20T10:19:31.935 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-20T10:19:31.935 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-20T10:19:31.935 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-20T10:19:31.935 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-20T10:19:31.935 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-20T10:19:31.935 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-20T10:19:31.935 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-20T10:19:31.935 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-20T10:19:31.935 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-20T10:19:31.935 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-20T10:19:31.935 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-20T10:19:31.936 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-20T10:19:31.936 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-20T10:19:31.936 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-20T10:19:31.936 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-20T10:19:31.936 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-20T10:19:31.936 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-20T10:19:31.936 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-20T10:19:31.936 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-20T10:19:31.936 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-20T10:19:31.936 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-20T10:19:31.936 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-20T10:19:31.936 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-20T10:19:31.936 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-20T10:19:31.936 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-20T10:19:31.936 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-20T10:19:31.937 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-20T10:19:31.937 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-20T10:19:31.937 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-20T10:19:31.937 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-20T10:19:31.937 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-20T10:19:31.937 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-20T10:19:31.937 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-20T10:19:31.937 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-20T10:19:31.937 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-20T10:19:31.937 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-20T10:19:31.937 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-20T10:19:31.937 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-20T10:19:31.937 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-20T10:19:31.937 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-20T10:19:31.938 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-20T10:19:31.938 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-20T10:19:31.938 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-20T10:19:31.938 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-20T10:19:31.938 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-20T10:19:31.938 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-20T10:19:31.938 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-20T10:19:31.938 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-20T10:19:31.938 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-20T10:19:31.938 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-20T10:19:31.938 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-20T10:19:31.938 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-20T10:19:31.938 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-20T10:19:31.939 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-20T10:19:31.939 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-20T10:19:31.939 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-20T10:19:31.939 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-20T10:19:31.939 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-20T10:19:31.939 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-20T10:19:31.939 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-20T10:19:31.939 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-20T10:19:31.939 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-20T10:19:31.939 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-20T10:19:31.939 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-20T10:19:31.939 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-20T10:19:31.940 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/152 2024-04-20T10:19:31.940 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-20T10:19:31.940 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/152 2024-04-20T10:19:31.940 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 127/152 2024-04-20T10:19:31.962 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T10:19:32.031 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T10:19:32.031 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:32.031 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-20T10:19:32.031 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T10:19:32.031 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T10:19:32.031 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:32.088 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T10:19:32.125 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T10:19:32.244 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/152 2024-04-20T10:19:32.346 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/152 2024-04-20T10:19:33.396 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-20T10:19:33.432 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T10:19:33.506 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T10:19:33.506 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:33.507 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-20T10:19:33.507 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T10:19:33.507 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T10:19:33.507 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:33.558 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T10:19:33.627 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T10:19:33.627 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:33.627 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-20T10:19:33.627 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:33.716 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T10:19:33.790 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T10:19:33.790 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:33.790 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-20T10:19:33.790 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T10:19:33.790 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T10:19:33.790 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:34.192 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T10:19:34.271 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T10:19:34.271 INFO:teuthology.orchestra.run.smithi121.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:19:34.271 INFO:teuthology.orchestra.run.smithi121.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-20T10:19:34.271 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T10:19:34.271 INFO:teuthology.orchestra.run.smithi121.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T10:19:34.272 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:34.353 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T10:19:34.353 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:34.353 INFO:teuthology.orchestra.run.smithi123.stdout:Upgraded: 2024-04-20T10:19:34.353 INFO:teuthology.orchestra.run.smithi123.stdout: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:34.353 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:34.354 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-04-20T10:19:34.354 INFO:teuthology.orchestra.run.smithi123.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-20T10:19:34.354 INFO:teuthology.orchestra.run.smithi123.stdout: automake-1.16.2-8.el9.noarch 2024-04-20T10:19:34.354 INFO:teuthology.orchestra.run.smithi123.stdout: bison-3.7.4-5.el9.x86_64 2024-04-20T10:19:34.354 INFO:teuthology.orchestra.run.smithi123.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-20T10:19:34.354 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:34.354 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:34.354 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:34.354 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:34.354 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:34.354 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:34.354 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:34.354 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:34.355 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:34.355 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:34.355 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:34.355 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:34.355 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:34.355 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:34.355 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:34.355 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:34.355 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:34.355 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:34.355 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:34.355 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:34.355 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:34.356 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:34.356 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:34.356 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:34.356 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-20T10:19:34.356 INFO:teuthology.orchestra.run.smithi123.stdout: flex-2.6.4-9.el9.x86_64 2024-04-20T10:19:34.356 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-20T10:19:34.356 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-20T10:19:34.356 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-20T10:19:34.356 INFO:teuthology.orchestra.run.smithi123.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-20T10:19:34.356 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-20T10:19:34.356 INFO:teuthology.orchestra.run.smithi123.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-20T10:19:34.356 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-20T10:19:34.356 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-20T10:19:34.356 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-20T10:19:34.357 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:34.357 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:34.357 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:34.357 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-20T10:19:34.357 INFO:teuthology.orchestra.run.smithi123.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-20T10:19:34.357 INFO:teuthology.orchestra.run.smithi123.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-20T10:19:34.357 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-20T10:19:34.357 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-20T10:19:34.357 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:34.357 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:34.357 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-20T10:19:34.357 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:34.357 INFO:teuthology.orchestra.run.smithi123.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-20T10:19:34.357 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-20T10:19:34.358 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-20T10:19:34.358 INFO:teuthology.orchestra.run.smithi123.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-20T10:19:34.358 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-20T10:19:34.358 INFO:teuthology.orchestra.run.smithi123.stdout: lua-5.4.4-4.el9.x86_64 2024-04-20T10:19:34.358 INFO:teuthology.orchestra.run.smithi123.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-20T10:19:34.358 INFO:teuthology.orchestra.run.smithi123.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-20T10:19:34.358 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-20T10:19:34.358 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-20T10:19:34.358 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-20T10:19:34.358 INFO:teuthology.orchestra.run.smithi123.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-20T10:19:34.358 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-13.14-1.el9.x86_64 2024-04-20T10:19:34.358 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-20T10:19:34.358 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-20T10:19:34.359 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-20T10:19:34.359 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-20T10:19:34.359 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-20T10:19:34.359 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-20T10:19:34.359 INFO:teuthology.orchestra.run.smithi123.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-20T10:19:34.359 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-20T10:19:34.359 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-20T10:19:34.359 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-20T10:19:34.359 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:34.359 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:34.359 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:34.359 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-20T10:19:34.359 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-20T10:19:34.359 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-20T10:19:34.360 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-20T10:19:34.360 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-20T10:19:34.360 INFO:teuthology.orchestra.run.smithi123.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-20T10:19:34.360 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-20T10:19:34.360 INFO:teuthology.orchestra.run.smithi123.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-20T10:19:34.360 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-20T10:19:34.360 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-20T10:19:34.360 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-20T10:19:34.360 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-20T10:19:34.360 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-20T10:19:34.360 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-20T10:19:34.360 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-20T10:19:34.360 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-20T10:19:34.360 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-20T10:19:34.361 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-20T10:19:34.361 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-20T10:19:34.361 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-20T10:19:34.361 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-20T10:19:34.361 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-20T10:19:34.361 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-20T10:19:34.361 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-20T10:19:34.361 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-20T10:19:34.361 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-20T10:19:34.361 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-20T10:19:34.361 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-20T10:19:34.361 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-20T10:19:34.361 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-20T10:19:34.362 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ply-3.11-14.el9.noarch 2024-04-20T10:19:34.362 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-20T10:19:34.362 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-20T10:19:34.362 INFO:teuthology.orchestra.run.smithi123.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-20T10:19:34.362 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-20T10:19:34.362 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-20T10:19:34.362 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-20T10:19:34.362 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-20T10:19:34.362 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-20T10:19:34.362 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-20T10:19:34.362 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:34.362 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:34.362 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-20T10:19:34.362 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-20T10:19:34.363 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-20T10:19:34.363 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:34.363 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-20T10:19:34.363 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-20T10:19:34.363 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-20T10:19:34.363 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-20T10:19:34.363 INFO:teuthology.orchestra.run.smithi123.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-20T10:19:34.363 INFO:teuthology.orchestra.run.smithi123.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-20T10:19:34.363 INFO:teuthology.orchestra.run.smithi123.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-20T10:19:34.363 INFO:teuthology.orchestra.run.smithi123.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-20T10:19:34.363 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-20T10:19:34.363 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-20T10:19:34.363 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-20T10:19:34.363 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-20T10:19:34.364 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-20T10:19:34.364 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-20T10:19:34.364 INFO:teuthology.orchestra.run.smithi123.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-20T10:19:34.364 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:34.364 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:34.364 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:34.364 INFO:teuthology.orchestra.run.smithi123.stdout: re2-1:20211101-3.el9.x86_64 2024-04-20T10:19:34.364 INFO:teuthology.orchestra.run.smithi123.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-20T10:19:34.364 INFO:teuthology.orchestra.run.smithi123.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-20T10:19:34.364 INFO:teuthology.orchestra.run.smithi123.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-20T10:19:34.364 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-20T10:19:34.364 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-20T10:19:34.365 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-20T10:19:34.365 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:34.365 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-04-20T10:19:34.989 DEBUG:teuthology.parallel:result is None 2024-04-20T10:19:39.293 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 137/152 2024-04-20T10:19:39.376 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 138/152 2024-04-20T10:19:39.468 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 139/152 2024-04-20T10:19:39.514 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-20T10:19:39.563 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-20T10:19:39.886 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 142/152 2024-04-20T10:19:39.938 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-20T10:19:39.974 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 144/152 2024-04-20T10:19:40.894 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T10:19:40.956 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T10:19:41.013 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T10:19:42.254 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 146/152 2024-04-20T10:19:42.354 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-20T10:19:42.436 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-20T10:19:42.591 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-20T10:19:42.643 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-20T10:19:42.643 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T10:19:42.777 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T10:19:42.777 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T10:19:43.315 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T10:19:43.315 INFO:teuthology.orchestra.run.smithi121.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-20T10:19:43.315 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:48.992 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 1/152 2024-04-20T10:19:48.992 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2/152 2024-04-20T10:19:48.992 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 3/152 2024-04-20T10:19:48.992 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 4/152 2024-04-20T10:19:48.992 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 5/152 2024-04-20T10:19:48.992 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T10:19:48.992 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T10:19:48.992 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 8/152 2024-04-20T10:19:48.993 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 9/152 2024-04-20T10:19:48.993 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/152 2024-04-20T10:19:48.993 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/152 2024-04-20T10:19:48.993 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T10:19:48.993 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 13/152 2024-04-20T10:19:48.993 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 14/152 2024-04-20T10:19:48.993 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 15/152 2024-04-20T10:19:48.993 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/152 2024-04-20T10:19:48.993 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 17/152 2024-04-20T10:19:48.993 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 18/152 2024-04-20T10:19:48.993 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 19/152 2024-04-20T10:19:48.993 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 20/152 2024-04-20T10:19:48.993 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 21/152 2024-04-20T10:19:48.993 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 22/152 2024-04-20T10:19:48.994 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 23/152 2024-04-20T10:19:48.994 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 24/152 2024-04-20T10:19:48.994 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 25/152 2024-04-20T10:19:48.994 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 26/152 2024-04-20T10:19:48.994 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 27/152 2024-04-20T10:19:48.994 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 28/152 2024-04-20T10:19:48.994 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 29/152 2024-04-20T10:19:48.994 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 30/152 2024-04-20T10:19:48.994 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 31/152 2024-04-20T10:19:49.005 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 32/152 2024-04-20T10:19:49.005 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 33/152 2024-04-20T10:19:49.005 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 34/152 2024-04-20T10:19:49.005 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 35/152 2024-04-20T10:19:49.005 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 36/152 2024-04-20T10:19:49.006 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 37/152 2024-04-20T10:19:49.006 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 38/152 2024-04-20T10:19:49.006 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 39/152 2024-04-20T10:19:49.006 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-20T10:19:49.006 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-20T10:19:49.006 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-20T10:19:49.006 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-20T10:19:49.006 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-20T10:19:49.006 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-20T10:19:49.006 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-20T10:19:49.006 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-20T10:19:49.006 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-20T10:19:49.006 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-20T10:19:49.007 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-20T10:19:49.007 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-20T10:19:49.007 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-20T10:19:49.007 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-20T10:19:49.007 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-20T10:19:49.007 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-20T10:19:49.007 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-20T10:19:49.007 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-20T10:19:49.007 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-20T10:19:49.007 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-20T10:19:49.007 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-20T10:19:49.007 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-20T10:19:49.007 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-20T10:19:49.008 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-20T10:19:49.008 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-20T10:19:49.008 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-20T10:19:49.008 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-20T10:19:49.008 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-20T10:19:49.008 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-20T10:19:49.008 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-20T10:19:49.008 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-20T10:19:49.008 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-20T10:19:49.008 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-20T10:19:49.008 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-20T10:19:49.008 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-20T10:19:49.008 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-20T10:19:49.008 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-20T10:19:49.009 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-20T10:19:49.009 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-20T10:19:49.009 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-20T10:19:49.009 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-20T10:19:49.009 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-20T10:19:49.009 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-20T10:19:49.009 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-20T10:19:49.009 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-20T10:19:49.009 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-20T10:19:49.009 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-20T10:19:49.009 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-20T10:19:49.009 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-20T10:19:49.009 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-20T10:19:49.009 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-20T10:19:49.010 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-20T10:19:49.010 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-20T10:19:49.010 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-20T10:19:49.010 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-20T10:19:49.010 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-20T10:19:49.010 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-20T10:19:49.010 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-20T10:19:49.010 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-20T10:19:49.010 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-20T10:19:49.010 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-20T10:19:49.010 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-20T10:19:49.010 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-20T10:19:49.010 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-20T10:19:49.010 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-20T10:19:49.011 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-20T10:19:49.011 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-20T10:19:49.011 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-20T10:19:49.011 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-20T10:19:49.011 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-20T10:19:49.011 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-20T10:19:49.011 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-20T10:19:49.011 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-20T10:19:49.011 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-20T10:19:49.011 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-20T10:19:49.011 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-20T10:19:49.011 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-20T10:19:49.011 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-20T10:19:49.011 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-20T10:19:49.012 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-20T10:19:49.012 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-20T10:19:49.012 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-20T10:19:49.012 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-20T10:19:49.012 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-20T10:19:49.012 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-20T10:19:49.012 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-20T10:19:49.012 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-20T10:19:49.012 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-20T10:19:49.012 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-20T10:19:49.012 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-20T10:19:49.012 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-20T10:19:49.012 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-20T10:19:49.012 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-20T10:19:49.012 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-20T10:19:49.013 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-20T10:19:49.013 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-20T10:19:49.013 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-20T10:19:49.013 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-20T10:19:49.013 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-20T10:19:49.013 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-20T10:19:49.013 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-20T10:19:49.013 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-20T10:19:49.013 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-20T10:19:49.013 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-20T10:19:49.013 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-20T10:19:49.013 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-20T10:19:49.013 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-20T10:19:49.013 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-20T10:19:49.014 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-20T10:19:49.014 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/152 2024-04-20T10:19:49.014 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-20T10:19:49.014 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/152 2024-04-20T10:19:51.508 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T10:19:51.508 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:51.508 INFO:teuthology.orchestra.run.smithi121.stdout:Upgraded: 2024-04-20T10:19:51.508 INFO:teuthology.orchestra.run.smithi121.stdout: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:51.508 INFO:teuthology.orchestra.run.smithi121.stdout: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:51.508 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-04-20T10:19:51.508 INFO:teuthology.orchestra.run.smithi121.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-20T10:19:51.508 INFO:teuthology.orchestra.run.smithi121.stdout: automake-1.16.2-8.el9.noarch 2024-04-20T10:19:51.508 INFO:teuthology.orchestra.run.smithi121.stdout: bison-3.7.4-5.el9.x86_64 2024-04-20T10:19:51.509 INFO:teuthology.orchestra.run.smithi121.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-20T10:19:51.509 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:51.509 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:51.509 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:51.509 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:51.509 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:51.509 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:51.509 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:51.509 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:51.509 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:51.509 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:51.510 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:51.510 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:51.510 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:51.510 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:51.510 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:51.510 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:51.510 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:51.510 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:51.510 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:51.510 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:51.510 INFO:teuthology.orchestra.run.smithi121.stdout: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:51.510 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:51.511 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:51.511 INFO:teuthology.orchestra.run.smithi121.stdout: cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:19:51.511 INFO:teuthology.orchestra.run.smithi121.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-20T10:19:51.511 INFO:teuthology.orchestra.run.smithi121.stdout: flex-2.6.4-9.el9.x86_64 2024-04-20T10:19:51.511 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-20T10:19:51.511 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-20T10:19:51.511 INFO:teuthology.orchestra.run.smithi121.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-20T10:19:51.511 INFO:teuthology.orchestra.run.smithi121.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-20T10:19:51.511 INFO:teuthology.orchestra.run.smithi121.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-20T10:19:51.511 INFO:teuthology.orchestra.run.smithi121.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-20T10:19:51.512 INFO:teuthology.orchestra.run.smithi121.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-20T10:19:51.512 INFO:teuthology.orchestra.run.smithi121.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-20T10:19:51.512 INFO:teuthology.orchestra.run.smithi121.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-20T10:19:51.512 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:51.512 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:51.512 INFO:teuthology.orchestra.run.smithi121.stdout: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:51.512 INFO:teuthology.orchestra.run.smithi121.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-20T10:19:51.512 INFO:teuthology.orchestra.run.smithi121.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-20T10:19:51.512 INFO:teuthology.orchestra.run.smithi121.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-20T10:19:51.512 INFO:teuthology.orchestra.run.smithi121.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-20T10:19:51.512 INFO:teuthology.orchestra.run.smithi121.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-20T10:19:51.513 INFO:teuthology.orchestra.run.smithi121.stdout: librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:51.513 INFO:teuthology.orchestra.run.smithi121.stdout: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:51.513 INFO:teuthology.orchestra.run.smithi121.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-20T10:19:51.513 INFO:teuthology.orchestra.run.smithi121.stdout: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:51.513 INFO:teuthology.orchestra.run.smithi121.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-20T10:19:51.513 INFO:teuthology.orchestra.run.smithi121.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-20T10:19:51.513 INFO:teuthology.orchestra.run.smithi121.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-20T10:19:51.513 INFO:teuthology.orchestra.run.smithi121.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-20T10:19:51.513 INFO:teuthology.orchestra.run.smithi121.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-20T10:19:51.513 INFO:teuthology.orchestra.run.smithi121.stdout: lua-5.4.4-4.el9.x86_64 2024-04-20T10:19:51.513 INFO:teuthology.orchestra.run.smithi121.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-20T10:19:51.513 INFO:teuthology.orchestra.run.smithi121.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-20T10:19:51.513 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-20T10:19:51.513 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-20T10:19:51.514 INFO:teuthology.orchestra.run.smithi121.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-20T10:19:51.514 INFO:teuthology.orchestra.run.smithi121.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-20T10:19:51.514 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-13.14-1.el9.x86_64 2024-04-20T10:19:51.514 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-20T10:19:51.514 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-20T10:19:51.514 INFO:teuthology.orchestra.run.smithi121.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-20T10:19:51.514 INFO:teuthology.orchestra.run.smithi121.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-20T10:19:51.514 INFO:teuthology.orchestra.run.smithi121.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-20T10:19:51.514 INFO:teuthology.orchestra.run.smithi121.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-20T10:19:51.514 INFO:teuthology.orchestra.run.smithi121.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-20T10:19:51.514 INFO:teuthology.orchestra.run.smithi121.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-20T10:19:51.514 INFO:teuthology.orchestra.run.smithi121.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-20T10:19:51.514 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-20T10:19:51.515 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:51.515 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:51.515 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:51.515 INFO:teuthology.orchestra.run.smithi121.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-20T10:19:51.515 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-20T10:19:51.515 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-20T10:19:51.515 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-20T10:19:51.515 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-20T10:19:51.515 INFO:teuthology.orchestra.run.smithi121.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-20T10:19:51.515 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-20T10:19:51.515 INFO:teuthology.orchestra.run.smithi121.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-20T10:19:51.515 INFO:teuthology.orchestra.run.smithi121.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-20T10:19:51.515 INFO:teuthology.orchestra.run.smithi121.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-20T10:19:51.516 INFO:teuthology.orchestra.run.smithi121.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-20T10:19:51.516 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-20T10:19:51.516 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-20T10:19:51.516 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-20T10:19:51.516 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-20T10:19:51.516 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-20T10:19:51.516 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-20T10:19:51.516 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-20T10:19:51.516 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-20T10:19:51.516 INFO:teuthology.orchestra.run.smithi121.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-20T10:19:51.516 INFO:teuthology.orchestra.run.smithi121.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-20T10:19:51.516 INFO:teuthology.orchestra.run.smithi121.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-20T10:19:51.516 INFO:teuthology.orchestra.run.smithi121.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-20T10:19:51.517 INFO:teuthology.orchestra.run.smithi121.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-20T10:19:51.517 INFO:teuthology.orchestra.run.smithi121.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-20T10:19:51.517 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-20T10:19:51.517 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-20T10:19:51.517 INFO:teuthology.orchestra.run.smithi121.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-20T10:19:51.517 INFO:teuthology.orchestra.run.smithi121.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-20T10:19:51.517 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-20T10:19:51.517 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ply-3.11-14.el9.noarch 2024-04-20T10:19:51.517 INFO:teuthology.orchestra.run.smithi121.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-20T10:19:51.517 INFO:teuthology.orchestra.run.smithi121.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-20T10:19:51.517 INFO:teuthology.orchestra.run.smithi121.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-20T10:19:51.517 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-20T10:19:51.517 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-20T10:19:51.517 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-20T10:19:51.517 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-20T10:19:51.518 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-20T10:19:51.518 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-20T10:19:51.518 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:51.518 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:51.518 INFO:teuthology.orchestra.run.smithi121.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-20T10:19:51.518 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-20T10:19:51.518 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-20T10:19:51.518 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:51.518 INFO:teuthology.orchestra.run.smithi121.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-20T10:19:51.518 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-20T10:19:51.518 INFO:teuthology.orchestra.run.smithi121.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-20T10:19:51.518 INFO:teuthology.orchestra.run.smithi121.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-20T10:19:51.518 INFO:teuthology.orchestra.run.smithi121.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-20T10:19:51.519 INFO:teuthology.orchestra.run.smithi121.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-20T10:19:51.519 INFO:teuthology.orchestra.run.smithi121.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-20T10:19:51.519 INFO:teuthology.orchestra.run.smithi121.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-20T10:19:51.519 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-20T10:19:51.519 INFO:teuthology.orchestra.run.smithi121.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-20T10:19:51.519 INFO:teuthology.orchestra.run.smithi121.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-20T10:19:51.519 INFO:teuthology.orchestra.run.smithi121.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-20T10:19:51.519 INFO:teuthology.orchestra.run.smithi121.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-20T10:19:51.519 INFO:teuthology.orchestra.run.smithi121.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-20T10:19:51.519 INFO:teuthology.orchestra.run.smithi121.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-20T10:19:51.519 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:51.519 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:51.519 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:19:51.519 INFO:teuthology.orchestra.run.smithi121.stdout: re2-1:20211101-3.el9.x86_64 2024-04-20T10:19:51.519 INFO:teuthology.orchestra.run.smithi121.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-20T10:19:51.519 INFO:teuthology.orchestra.run.smithi121.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-20T10:19:51.520 INFO:teuthology.orchestra.run.smithi121.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-20T10:19:51.520 INFO:teuthology.orchestra.run.smithi121.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-20T10:19:51.520 INFO:teuthology.orchestra.run.smithi121.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-20T10:19:51.520 INFO:teuthology.orchestra.run.smithi121.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-20T10:19:51.520 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:51.520 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-04-20T10:19:52.133 DEBUG:teuthology.parallel:result is None 2024-04-20T10:19:52.134 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T10:19:52.359 DEBUG:teuthology.orchestra.run.smithi121:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-20T10:19:52.417 INFO:teuthology.orchestra.run.smithi121.stdout:19.0.0-3133.ga8c84f59.el9 2024-04-20T10:19:52.418 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3133.ga8c84f59.el9 2024-04-20T10:19:52.418 INFO:teuthology.task.install:The correct ceph version 19.0.0-3133.ga8c84f59 is installed. 2024-04-20T10:19:52.420 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T10:19:52.631 DEBUG:teuthology.orchestra.run.smithi123:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-20T10:19:52.704 INFO:teuthology.orchestra.run.smithi123.stdout:19.0.0-3133.ga8c84f59.el9 2024-04-20T10:19:52.705 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3133.ga8c84f59.el9 2024-04-20T10:19:52.705 INFO:teuthology.task.install:The correct ceph version 19.0.0-3133.ga8c84f59 is installed. 2024-04-20T10:19:52.710 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-20T10:19:52.710 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T10:19:52.710 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-20T10:19:52.785 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T10:19:52.786 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-20T10:19:52.879 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-20T10:19:52.880 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T10:19:52.880 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/daemon-helper 2024-04-20T10:19:52.954 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-20T10:19:53.059 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T10:19:53.060 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/daemon-helper 2024-04-20T10:19:53.151 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-20T10:19:53.233 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-20T10:19:53.234 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T10:19:53.235 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-20T10:19:53.311 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-20T10:19:53.416 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T10:19:53.416 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-20T10:19:53.493 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-20T10:19:53.609 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-20T10:19:53.610 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T10:19:53.610 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/stdin-killer 2024-04-20T10:19:53.674 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-20T10:19:53.778 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T10:19:53.779 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/stdin-killer 2024-04-20T10:19:53.855 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-20T10:19:53.970 INFO:teuthology.run_tasks:Running task ceph... 2024-04-20T10:19:54.060 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-20T10:19:54.060 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 777 /var/log/ceph 2024-04-20T10:19:54.065 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 777 /var/log/ceph 2024-04-20T10:19:54.147 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-20T10:19:54.148 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-20T10:19:54.152 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-20T10:19:54.216 INFO:tasks.ceph:Creating extra log directories... 2024-04-20T10:19:54.216 DEBUG:teuthology.orchestra.run.smithi121:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-20T10:19:54.220 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-20T10:19:54.290 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-20T10:19:54.290 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-20T10:19:54.290 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-20T10:19:54.290 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T10:19:54.291 DEBUG:teuthology.orchestra.run.smithi121:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-20T10:19:54.331 DEBUG:teuthology.orchestra.run.smithi121:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-20T10:19:54.517 DEBUG:teuthology.orchestra.run.smithi121:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-20T10:19:54.581 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T10:19:54.582 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-20T10:19:54.633 DEBUG:teuthology.orchestra.run.smithi123:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-20T10:19:54.849 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-20T10:19:54.941 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-20T10:19:54.942 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-20T10:19:54.942 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664420', 'branch': 'wip-rishabh-testing-20240416.193735', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/xfstests-dev}', 'email': None, 'first_in_suite': False, 'job_id': '7664420', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240416.193735'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240416.193735', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 7954, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'sleep_before_teardown': 0, 'subset': '364/512', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240416.193735', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi121.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDwe06AVl2/32bDUjOa4CZ3632mEG9zF+J0zWvWIBKREkq+95M61hZTmM7tTEDHkajzjNb/g3ICfn2BJO3iRSno=', 'smithi123.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL4H76MxIbiuUrrg5fWvZxX2p/H8QEzrAnJWPuUM0ZDIde5fB3OeGVfnDBwnxJu/suKVEmGmC37T1rQ/TOw3cI8='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.tests_from_xfstests_dev']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-20_09:50:15', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-20T10:19:54.943 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-20T10:19:54.947 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-20T10:19:55.003 DEBUG:teuthology.orchestra.run.smithi121:> sudo install -d -m0777 -- /var/run/ceph 2024-04-20T10:19:55.006 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m0777 -- /var/run/ceph 2024-04-20T10:19:55.083 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T10:19:55.083 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/scratch_devs of=/dev/stdout 2024-04-20T10:19:55.123 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-20T10:19:55.123 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_1 2024-04-20T10:19:55.204 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-20T10:19:55.205 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:19:55.205 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 6h/6d Inode: 631 Links: 1 2024-04-20T10:19:55.205 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:19:55.205 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:19:55.205 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2024-04-20 10:19:43.836731898 +0000 2024-04-20T10:19:55.205 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-04-20 10:14:30.655243979 +0000 2024-04-20T10:19:55.205 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-04-20 10:14:30.655243979 +0000 2024-04-20T10:19:55.206 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: 2024-04-20 10:14:30.655243979 +0000 2024-04-20T10:19:55.206 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-20T10:19:55.311 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2024-04-20T10:19:55.311 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2024-04-20T10:19:55.311 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000368238 s, 1.4 MB/s 2024-04-20T10:19:55.313 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-20T10:19:55.352 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_2 2024-04-20T10:19:55.431 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-20T10:19:55.431 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:19:55.432 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-04-20T10:19:55.432 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:19:55.432 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:19:55.432 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2024-04-20 10:19:43.837731885 +0000 2024-04-20T10:19:55.432 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-04-20 10:14:31.251236219 +0000 2024-04-20T10:19:55.432 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-04-20 10:14:31.251236219 +0000 2024-04-20T10:19:55.432 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: 2024-04-20 10:14:31.251236219 +0000 2024-04-20T10:19:55.433 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-20T10:19:55.530 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2024-04-20T10:19:55.531 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2024-04-20T10:19:55.531 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000283364 s, 1.8 MB/s 2024-04-20T10:19:55.533 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-20T10:19:55.574 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_3 2024-04-20T10:19:55.653 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-20T10:19:55.653 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:19:55.653 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-04-20T10:19:55.653 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:19:55.653 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:19:55.653 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2024-04-20 10:19:43.837731885 +0000 2024-04-20T10:19:55.653 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-04-20 10:14:31.844228497 +0000 2024-04-20T10:19:55.653 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-04-20 10:14:31.844228497 +0000 2024-04-20T10:19:55.653 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: 2024-04-20 10:14:31.844228497 +0000 2024-04-20T10:19:55.654 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-20T10:19:55.751 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2024-04-20T10:19:55.751 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2024-04-20T10:19:55.751 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000296781 s, 1.7 MB/s 2024-04-20T10:19:55.754 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-20T10:19:55.795 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_4 2024-04-20T10:19:55.874 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-20T10:19:55.874 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:19:55.875 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-04-20T10:19:55.875 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:19:55.875 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:19:55.875 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2024-04-20 10:19:43.838731872 +0000 2024-04-20T10:19:55.875 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2024-04-20 10:14:32.458220502 +0000 2024-04-20T10:19:55.875 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2024-04-20 10:14:32.458220502 +0000 2024-04-20T10:19:55.875 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: 2024-04-20 10:14:32.458220502 +0000 2024-04-20T10:19:55.875 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-20T10:19:55.971 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2024-04-20T10:19:55.972 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2024-04-20T10:19:55.972 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000365334 s, 1.4 MB/s 2024-04-20T10:19:55.974 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-20T10:19:56.012 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-20T10:19:56.012 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T10:19:56.012 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/scratch_devs of=/dev/stdout 2024-04-20T10:19:56.070 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-20T10:19:56.070 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_1 2024-04-20T10:19:56.123 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-20T10:19:56.123 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:19:56.123 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 631 Links: 1 2024-04-20T10:19:56.123 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:19:56.123 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:19:56.123 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-04-20 10:19:26.621557925 +0000 2024-04-20T10:19:56.123 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-04-20 10:14:20.853643063 +0000 2024-04-20T10:19:56.123 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-04-20 10:14:20.853643063 +0000 2024-04-20T10:19:56.123 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-04-20 10:14:20.853643063 +0000 2024-04-20T10:19:56.124 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-20T10:19:56.205 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-04-20T10:19:56.205 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-04-20T10:19:56.205 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000362866 s, 1.4 MB/s 2024-04-20T10:19:56.208 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-20T10:19:56.262 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_2 2024-04-20T10:19:56.311 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-20T10:19:56.311 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:19:56.311 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-04-20T10:19:56.311 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:19:56.311 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:19:56.311 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-04-20 10:19:26.621557925 +0000 2024-04-20T10:19:56.312 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-04-20 10:14:21.469628544 +0000 2024-04-20T10:19:56.312 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-04-20 10:14:21.469628544 +0000 2024-04-20T10:19:56.312 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-04-20 10:14:21.469628544 +0000 2024-04-20T10:19:56.312 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-20T10:19:56.386 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-04-20T10:19:56.386 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-04-20T10:19:56.386 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000471679 s, 1.1 MB/s 2024-04-20T10:19:56.390 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-20T10:19:56.449 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_3 2024-04-20T10:19:56.495 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-20T10:19:56.495 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:19:56.495 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-04-20T10:19:56.495 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:19:56.495 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:19:56.495 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-04-20 10:19:26.622557902 +0000 2024-04-20T10:19:56.496 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-04-20 10:14:22.055614733 +0000 2024-04-20T10:19:56.496 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-04-20 10:14:22.055614733 +0000 2024-04-20T10:19:56.496 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-04-20 10:14:22.055614733 +0000 2024-04-20T10:19:56.496 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-20T10:19:56.615 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-04-20T10:19:56.615 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-04-20T10:19:56.615 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000346654 s, 1.5 MB/s 2024-04-20T10:19:56.618 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-20T10:19:56.674 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_4 2024-04-20T10:19:56.725 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-20T10:19:56.725 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:19:56.726 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-04-20T10:19:56.726 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:19:56.726 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:19:56.726 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-04-20 10:19:26.623557879 +0000 2024-04-20T10:19:56.726 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-04-20 10:14:22.661600450 +0000 2024-04-20T10:19:56.726 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-04-20 10:14:22.661600450 +0000 2024-04-20T10:19:56.726 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-04-20 10:14:22.661600450 +0000 2024-04-20T10:19:56.726 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-20T10:19:56.804 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-04-20T10:19:56.804 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-04-20T10:19:56.804 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000327983 s, 1.6 MB/s 2024-04-20T10:19:56.808 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-20T10:19:56.864 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-20T10:19:56.864 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi121.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi123.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-20T10:19:56.864 INFO:tasks.ceph:Generating config... 2024-04-20T10:19:56.865 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-20T10:19:56.866 INFO:tasks.ceph:[client] debug client = 20 2024-04-20T10:19:56.866 INFO:tasks.ceph:[client] debug ms = 1 2024-04-20T10:19:56.866 INFO:tasks.ceph:[client] pid file = 2024-04-20T10:19:56.866 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-20T10:19:56.866 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-20T10:19:56.866 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-20T10:19:56.866 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-20T10:19:56.866 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-20T10:19:56.866 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-20T10:19:56.866 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-20T10:19:56.866 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-20T10:19:56.866 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-20T10:19:56.867 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-20T10:19:56.867 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-20T10:19:56.867 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-20T10:19:56.867 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-20T10:19:56.867 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-20T10:19:56.867 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-20T10:19:56.867 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-20T10:19:56.867 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-20T10:19:56.867 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-20T10:19:56.867 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-20T10:19:56.867 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-20T10:19:56.867 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-20T10:19:56.867 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-20T10:19:56.868 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-20T10:19:56.868 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-20T10:19:56.868 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-20T10:19:56.868 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-20T10:19:56.868 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-20T10:19:56.868 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-20T10:19:56.868 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-20T10:19:56.868 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-20T10:19:56.868 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-20T10:19:56.868 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-20T10:19:56.868 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-20T10:19:56.868 INFO:tasks.ceph:Setting up mon.a... 2024-04-20T10:19:56.869 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-20T10:19:56.970 INFO:teuthology.orchestra.run.smithi121.stdout:creating /etc/ceph/ceph.keyring 2024-04-20T10:19:56.978 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-20T10:19:57.073 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-20T10:19:57.134 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.121'), ('mon.b', '172.21.15.123'), ('mon.c', '[v2:172.21.15.123:3301,v1:172.21.15.123:6790]')] 2024-04-20T10:19:57.134 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.121,172.21.15.123,[v2:172.21.15.123:3301,v1:172.21.15.123:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-20T10:19:57.135 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T10:19:57.135 DEBUG:teuthology.orchestra.run.smithi121:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-20T10:19:57.174 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.121 --add b 172.21.15.123 --addv c '[v2:172.21.15.123:3301,v1:172.21.15.123:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:19:57.301 INFO:teuthology.orchestra.run.smithi121.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:19:57.301 INFO:teuthology.orchestra.run.smithi121.stdout:monmaptool: generated fsid ceaac016-4ab2-472d-a36e-7bfe9497c762 2024-04-20T10:19:57.301 INFO:teuthology.orchestra.run.smithi121.stdout:setting min_mon_release = quincy 2024-04-20T10:19:57.301 INFO:teuthology.orchestra.run.smithi121.stdout:epoch 0 2024-04-20T10:19:57.301 INFO:teuthology.orchestra.run.smithi121.stdout:fsid ceaac016-4ab2-472d-a36e-7bfe9497c762 2024-04-20T10:19:57.301 INFO:teuthology.orchestra.run.smithi121.stdout:last_changed 2024-04-20T10:19:57.302223+0000 2024-04-20T10:19:57.302 INFO:teuthology.orchestra.run.smithi121.stdout:created 2024-04-20T10:19:57.302223+0000 2024-04-20T10:19:57.302 INFO:teuthology.orchestra.run.smithi121.stdout:min_mon_release 17 (quincy) 2024-04-20T10:19:57.302 INFO:teuthology.orchestra.run.smithi121.stdout:election_strategy: 1 2024-04-20T10:19:57.302 INFO:teuthology.orchestra.run.smithi121.stdout:0: [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] mon.a 2024-04-20T10:19:57.302 INFO:teuthology.orchestra.run.smithi121.stdout:1: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.b 2024-04-20T10:19:57.302 INFO:teuthology.orchestra.run.smithi121.stdout:2: [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] mon.c 2024-04-20T10:19:57.302 INFO:teuthology.orchestra.run.smithi121.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-20T10:19:57.304 DEBUG:teuthology.orchestra.run.smithi121:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-20T10:19:57.342 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID ceaac016-4ab2-472d-a36e-7bfe9497c762... 2024-04-20T10:19:57.343 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-20T10:19:57.388 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-20T10:19:57.501 INFO:teuthology.orchestra.run.smithi121.stdout:[global] 2024-04-20T10:19:57.501 INFO:teuthology.orchestra.run.smithi121.stdout: chdir = "" 2024-04-20T10:19:57.501 INFO:teuthology.orchestra.run.smithi121.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-20T10:19:57.501 INFO:teuthology.orchestra.run.smithi121.stdout: auth supported = cephx 2024-04-20T10:19:57.501 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:57.501 INFO:teuthology.orchestra.run.smithi121.stdout: filestore xattr use omap = true 2024-04-20T10:19:57.501 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:57.502 INFO:teuthology.orchestra.run.smithi121.stdout: mon clock drift allowed = 1.000 2024-04-20T10:19:57.502 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:57.502 INFO:teuthology.orchestra.run.smithi121.stdout: osd crush chooseleaf type = 0 2024-04-20T10:19:57.502 INFO:teuthology.orchestra.run.smithi121.stdout: auth debug = true 2024-04-20T10:19:57.502 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:57.502 INFO:teuthology.orchestra.run.smithi121.stdout: ms die on old message = true 2024-04-20T10:19:57.502 INFO:teuthology.orchestra.run.smithi121.stdout: ms die on bug = true 2024-04-20T10:19:57.502 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:57.502 INFO:teuthology.orchestra.run.smithi121.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-20T10:19:57.502 INFO:teuthology.orchestra.run.smithi121.stdout: mon pg warn max object skew = 0 2024-04-20T10:19:57.502 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:57.502 INFO:teuthology.orchestra.run.smithi121.stdout: # disable pg_autoscaler by default for new pools 2024-04-20T10:19:57.503 INFO:teuthology.orchestra.run.smithi121.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-20T10:19:57.503 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:57.503 INFO:teuthology.orchestra.run.smithi121.stdout: osd pool default size = 2 2024-04-20T10:19:57.503 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:57.503 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd allow primary affinity = true 2024-04-20T10:19:57.503 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd allow pg remap = true 2024-04-20T10:19:57.503 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on legacy crush tunables = false 2024-04-20T10:19:57.503 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on crush straw calc version zero = false 2024-04-20T10:19:57.503 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on no sortbitwise = false 2024-04-20T10:19:57.503 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on osd down out interval zero = false 2024-04-20T10:19:57.503 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on too few osds = false 2024-04-20T10:19:57.503 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-20T10:19:57.503 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-20T10:19:57.503 INFO:teuthology.orchestra.run.smithi121.stdout: mon_allow_pool_size_one = true 2024-04-20T10:19:57.503 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:57.504 INFO:teuthology.orchestra.run.smithi121.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-20T10:19:57.504 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:57.504 INFO:teuthology.orchestra.run.smithi121.stdout: osd default data pool replay window = 5 2024-04-20T10:19:57.504 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:57.504 INFO:teuthology.orchestra.run.smithi121.stdout: mon allow pool delete = true 2024-04-20T10:19:57.504 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:57.504 INFO:teuthology.orchestra.run.smithi121.stdout: mon cluster log file level = debug 2024-04-20T10:19:57.504 INFO:teuthology.orchestra.run.smithi121.stdout: debug asserts on shutdown = true 2024-04-20T10:19:57.504 INFO:teuthology.orchestra.run.smithi121.stdout: mon health detail to clog = false 2024-04-20T10:19:57.504 INFO:teuthology.orchestra.run.smithi121.stdout: mon host = "172.21.15.121,172.21.15.123,[v2:172.21.15.123:3301,v1:172.21.15.123:6790]" 2024-04-20T10:19:57.504 INFO:teuthology.orchestra.run.smithi121.stdout: fsid = ceaac016-4ab2-472d-a36e-7bfe9497c762 2024-04-20T10:19:57.504 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:57.505 INFO:teuthology.orchestra.run.smithi121.stdout:[osd] 2024-04-20T10:19:57.505 INFO:teuthology.orchestra.run.smithi121.stdout: osd journal size = 100 2024-04-20T10:19:57.505 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:57.505 INFO:teuthology.orchestra.run.smithi121.stdout: osd scrub load threshold = 5.0 2024-04-20T10:19:57.505 INFO:teuthology.orchestra.run.smithi121.stdout: osd scrub max interval = 600 2024-04-20T10:19:57.505 INFO:teuthology.orchestra.run.smithi121.stdout: osd mclock profile = high_recovery_ops 2024-04-20T10:19:57.505 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:57.505 INFO:teuthology.orchestra.run.smithi121.stdout: osd recover clone overlap = true 2024-04-20T10:19:57.505 INFO:teuthology.orchestra.run.smithi121.stdout: osd recovery max chunk = 1048576 2024-04-20T10:19:57.505 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:57.505 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug shutdown = true 2024-04-20T10:19:57.505 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug op order = true 2024-04-20T10:19:57.505 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug verify stray on activate = true 2024-04-20T10:19:57.506 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug trim objects = true 2024-04-20T10:19:57.506 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:57.506 INFO:teuthology.orchestra.run.smithi121.stdout: osd open classes on start = true 2024-04-20T10:19:57.506 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug pg log writeout = true 2024-04-20T10:19:57.506 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:57.506 INFO:teuthology.orchestra.run.smithi121.stdout: osd deep scrub update digest min age = 30 2024-04-20T10:19:57.506 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:57.506 INFO:teuthology.orchestra.run.smithi121.stdout: osd map max advance = 10 2024-04-20T10:19:57.506 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:57.506 INFO:teuthology.orchestra.run.smithi121.stdout: journal zero on create = true 2024-04-20T10:19:57.506 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:57.506 INFO:teuthology.orchestra.run.smithi121.stdout: filestore ondisk finisher threads = 3 2024-04-20T10:19:57.506 INFO:teuthology.orchestra.run.smithi121.stdout: filestore apply finisher threads = 3 2024-04-20T10:19:57.506 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:57.507 INFO:teuthology.orchestra.run.smithi121.stdout: bdev debug aio = true 2024-04-20T10:19:57.507 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug misdirected ops = true 2024-04-20T10:19:57.507 INFO:teuthology.orchestra.run.smithi121.stdout: bluestore block size = 96636764160 2024-04-20T10:19:57.507 INFO:teuthology.orchestra.run.smithi121.stdout: bluestore fsck on mount = True 2024-04-20T10:19:57.507 INFO:teuthology.orchestra.run.smithi121.stdout: debug bluefs = 20 2024-04-20T10:19:57.507 INFO:teuthology.orchestra.run.smithi121.stdout: debug bluestore = 20 2024-04-20T10:19:57.507 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2024-04-20T10:19:57.507 INFO:teuthology.orchestra.run.smithi121.stdout: debug osd = 20 2024-04-20T10:19:57.507 INFO:teuthology.orchestra.run.smithi121.stdout: debug rocksdb = 10 2024-04-20T10:19:57.507 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd backfillfull_ratio = 0.85 2024-04-20T10:19:57.507 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd full ratio = 0.9 2024-04-20T10:19:57.507 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd nearfull ratio = 0.8 2024-04-20T10:19:57.507 INFO:teuthology.orchestra.run.smithi121.stdout: osd failsafe full ratio = 0.95 2024-04-20T10:19:57.507 INFO:teuthology.orchestra.run.smithi121.stdout: osd objectstore = bluestore 2024-04-20T10:19:57.507 INFO:teuthology.orchestra.run.smithi121.stdout: osd op complaint time = 180 2024-04-20T10:19:57.508 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:57.508 INFO:teuthology.orchestra.run.smithi121.stdout:[mgr] 2024-04-20T10:19:57.508 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2024-04-20T10:19:57.508 INFO:teuthology.orchestra.run.smithi121.stdout: debug mgr = 20 2024-04-20T10:19:57.508 INFO:teuthology.orchestra.run.smithi121.stdout: debug mon = 20 2024-04-20T10:19:57.508 INFO:teuthology.orchestra.run.smithi121.stdout: debug auth = 20 2024-04-20T10:19:57.508 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min pgs per osd = 4 2024-04-20T10:19:57.508 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min bytes per osd = 10 2024-04-20T10:19:57.508 INFO:teuthology.orchestra.run.smithi121.stdout: mgr/telemetry/nag = false 2024-04-20T10:19:57.508 INFO:teuthology.orchestra.run.smithi121.stdout: debug client = 20 2024-04-20T10:19:57.508 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:57.508 INFO:teuthology.orchestra.run.smithi121.stdout:[mon] 2024-04-20T10:19:57.508 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2024-04-20T10:19:57.508 INFO:teuthology.orchestra.run.smithi121.stdout: debug mon = 20 2024-04-20T10:19:57.509 INFO:teuthology.orchestra.run.smithi121.stdout: debug paxos = 20 2024-04-20T10:19:57.509 INFO:teuthology.orchestra.run.smithi121.stdout: debug auth = 20 2024-04-20T10:19:57.509 INFO:teuthology.orchestra.run.smithi121.stdout: mon data avail warn = 5 2024-04-20T10:19:57.509 INFO:teuthology.orchestra.run.smithi121.stdout: mon mgr mkfs grace = 240 2024-04-20T10:19:57.509 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min pgs per osd = 4 2024-04-20T10:19:57.509 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd reporter subtree level = osd 2024-04-20T10:19:57.509 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd prime pg temp = true 2024-04-20T10:19:57.509 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min bytes per osd = 10 2024-04-20T10:19:57.509 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:57.509 INFO:teuthology.orchestra.run.smithi121.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-20T10:19:57.509 INFO:teuthology.orchestra.run.smithi121.stdout: auth mon ticket ttl = 660 # 11m 2024-04-20T10:19:57.509 INFO:teuthology.orchestra.run.smithi121.stdout: auth service ticket ttl = 240 # 4m 2024-04-20T10:19:57.509 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:57.509 INFO:teuthology.orchestra.run.smithi121.stdout: # don't complain about insecure global_id in the test suite 2024-04-20T10:19:57.509 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-20T10:19:57.510 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-20T10:19:57.510 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:57.510 INFO:teuthology.orchestra.run.smithi121.stdout: # 1m isn't quite enough 2024-04-20T10:19:57.510 INFO:teuthology.orchestra.run.smithi121.stdout: mon_down_mkfs_grace = 2m 2024-04-20T10:19:57.510 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:57.510 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_filestore_osds = false 2024-04-20T10:19:57.510 INFO:teuthology.orchestra.run.smithi121.stdout: mon down mkfs grace = 300 2024-04-20T10:19:57.510 INFO:teuthology.orchestra.run.smithi121.stdout: mon op complaint time = 120 2024-04-20T10:19:57.510 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:19:57.510 INFO:teuthology.orchestra.run.smithi121.stdout:[client] 2024-04-20T10:19:57.510 INFO:teuthology.orchestra.run.smithi121.stdout: rgw cache enabled = true 2024-04-20T10:19:57.510 INFO:teuthology.orchestra.run.smithi121.stdout: rgw enable ops log = true 2024-04-20T10:19:57.510 INFO:teuthology.orchestra.run.smithi121.stdout: rgw enable usage log = true 2024-04-20T10:19:57.511 INFO:teuthology.orchestra.run.smithi121.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-20T10:19:57.511 INFO:teuthology.orchestra.run.smithi121.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-20T10:19:57.511 INFO:teuthology.orchestra.run.smithi121.stdout: client mount timeout = 600 2024-04-20T10:19:57.511 INFO:teuthology.orchestra.run.smithi121.stdout: debug client = 20 2024-04-20T10:19:57.511 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2024-04-20T10:19:57.511 INFO:teuthology.orchestra.run.smithi121.stdout: pid file = "" 2024-04-20T10:19:57.511 INFO:teuthology.orchestra.run.smithi121.stdout: rados mon op timeout = 900 2024-04-20T10:19:57.511 INFO:teuthology.orchestra.run.smithi121.stdout: rados osd op timeout = 900 2024-04-20T10:19:57.511 INFO:teuthology.orchestra.run.smithi121.stdout:[mon.a] 2024-04-20T10:19:57.511 INFO:teuthology.orchestra.run.smithi121.stdout:[mon.b] 2024-04-20T10:19:57.511 INFO:teuthology.orchestra.run.smithi121.stdout:[mon.c] 2024-04-20T10:19:57.511 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.a] 2024-04-20T10:19:57.511 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.c] 2024-04-20T10:19:57.511 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.b] 2024-04-20T10:19:57.512 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.d] 2024-04-20T10:19:57.512 INFO:teuthology.orchestra.run.smithi121.stdout:[mds] 2024-04-20T10:19:57.512 INFO:teuthology.orchestra.run.smithi121.stdout: debug mds = 20 2024-04-20T10:19:57.512 INFO:teuthology.orchestra.run.smithi121.stdout: debug mds balancer = 20 2024-04-20T10:19:57.512 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2024-04-20T10:19:57.512 INFO:teuthology.orchestra.run.smithi121.stdout: mds debug frag = True 2024-04-20T10:19:57.512 INFO:teuthology.orchestra.run.smithi121.stdout: mds debug scatterstat = True 2024-04-20T10:19:57.512 INFO:teuthology.orchestra.run.smithi121.stdout: mds op complaint time = 180 2024-04-20T10:19:57.512 INFO:teuthology.orchestra.run.smithi121.stdout: mds verify scatter = True 2024-04-20T10:19:57.512 INFO:teuthology.orchestra.run.smithi121.stdout: osd op complaint time = 180 2024-04-20T10:19:57.512 INFO:teuthology.orchestra.run.smithi121.stdout: rados mon op timeout = 900 2024-04-20T10:19:57.512 INFO:teuthology.orchestra.run.smithi121.stdout: rados osd op timeout = 900 2024-04-20T10:19:57.531 INFO:teuthology.orchestra.run.smithi123.stdout:[global] 2024-04-20T10:19:57.531 INFO:teuthology.orchestra.run.smithi123.stdout: chdir = "" 2024-04-20T10:19:57.531 INFO:teuthology.orchestra.run.smithi123.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-20T10:19:57.531 INFO:teuthology.orchestra.run.smithi123.stdout: auth supported = cephx 2024-04-20T10:19:57.531 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:57.531 INFO:teuthology.orchestra.run.smithi123.stdout: filestore xattr use omap = true 2024-04-20T10:19:57.531 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:57.531 INFO:teuthology.orchestra.run.smithi123.stdout: mon clock drift allowed = 1.000 2024-04-20T10:19:57.531 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:57.532 INFO:teuthology.orchestra.run.smithi123.stdout: osd crush chooseleaf type = 0 2024-04-20T10:19:57.532 INFO:teuthology.orchestra.run.smithi123.stdout: auth debug = true 2024-04-20T10:19:57.532 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:57.532 INFO:teuthology.orchestra.run.smithi123.stdout: ms die on old message = true 2024-04-20T10:19:57.532 INFO:teuthology.orchestra.run.smithi123.stdout: ms die on bug = true 2024-04-20T10:19:57.532 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:57.532 INFO:teuthology.orchestra.run.smithi123.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-20T10:19:57.532 INFO:teuthology.orchestra.run.smithi123.stdout: mon pg warn max object skew = 0 2024-04-20T10:19:57.532 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:57.532 INFO:teuthology.orchestra.run.smithi123.stdout: # disable pg_autoscaler by default for new pools 2024-04-20T10:19:57.532 INFO:teuthology.orchestra.run.smithi123.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-20T10:19:57.532 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:57.532 INFO:teuthology.orchestra.run.smithi123.stdout: osd pool default size = 2 2024-04-20T10:19:57.533 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:57.533 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd allow primary affinity = true 2024-04-20T10:19:57.533 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd allow pg remap = true 2024-04-20T10:19:57.533 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on legacy crush tunables = false 2024-04-20T10:19:57.533 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on crush straw calc version zero = false 2024-04-20T10:19:57.533 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on no sortbitwise = false 2024-04-20T10:19:57.533 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on osd down out interval zero = false 2024-04-20T10:19:57.533 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on too few osds = false 2024-04-20T10:19:57.533 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-20T10:19:57.533 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-20T10:19:57.533 INFO:teuthology.orchestra.run.smithi123.stdout: mon_allow_pool_size_one = true 2024-04-20T10:19:57.533 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:57.533 INFO:teuthology.orchestra.run.smithi123.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-20T10:19:57.533 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:57.533 INFO:teuthology.orchestra.run.smithi123.stdout: osd default data pool replay window = 5 2024-04-20T10:19:57.534 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:57.534 INFO:teuthology.orchestra.run.smithi123.stdout: mon allow pool delete = true 2024-04-20T10:19:57.534 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:57.534 INFO:teuthology.orchestra.run.smithi123.stdout: mon cluster log file level = debug 2024-04-20T10:19:57.534 INFO:teuthology.orchestra.run.smithi123.stdout: debug asserts on shutdown = true 2024-04-20T10:19:57.534 INFO:teuthology.orchestra.run.smithi123.stdout: mon health detail to clog = false 2024-04-20T10:19:57.534 INFO:teuthology.orchestra.run.smithi123.stdout: mon host = "172.21.15.121,172.21.15.123,[v2:172.21.15.123:3301,v1:172.21.15.123:6790]" 2024-04-20T10:19:57.534 INFO:teuthology.orchestra.run.smithi123.stdout: fsid = ceaac016-4ab2-472d-a36e-7bfe9497c762 2024-04-20T10:19:57.534 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:57.534 INFO:teuthology.orchestra.run.smithi123.stdout:[osd] 2024-04-20T10:19:57.534 INFO:teuthology.orchestra.run.smithi123.stdout: osd journal size = 100 2024-04-20T10:19:57.534 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:57.534 INFO:teuthology.orchestra.run.smithi123.stdout: osd scrub load threshold = 5.0 2024-04-20T10:19:57.534 INFO:teuthology.orchestra.run.smithi123.stdout: osd scrub max interval = 600 2024-04-20T10:19:57.535 INFO:teuthology.orchestra.run.smithi123.stdout: osd mclock profile = high_recovery_ops 2024-04-20T10:19:57.535 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:57.535 INFO:teuthology.orchestra.run.smithi123.stdout: osd recover clone overlap = true 2024-04-20T10:19:57.535 INFO:teuthology.orchestra.run.smithi123.stdout: osd recovery max chunk = 1048576 2024-04-20T10:19:57.535 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:57.535 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug shutdown = true 2024-04-20T10:19:57.535 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug op order = true 2024-04-20T10:19:57.535 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug verify stray on activate = true 2024-04-20T10:19:57.535 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug trim objects = true 2024-04-20T10:19:57.535 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:57.535 INFO:teuthology.orchestra.run.smithi123.stdout: osd open classes on start = true 2024-04-20T10:19:57.535 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug pg log writeout = true 2024-04-20T10:19:57.535 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:57.535 INFO:teuthology.orchestra.run.smithi123.stdout: osd deep scrub update digest min age = 30 2024-04-20T10:19:57.535 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:57.536 INFO:teuthology.orchestra.run.smithi123.stdout: osd map max advance = 10 2024-04-20T10:19:57.536 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:57.536 INFO:teuthology.orchestra.run.smithi123.stdout: journal zero on create = true 2024-04-20T10:19:57.536 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:57.536 INFO:teuthology.orchestra.run.smithi123.stdout: filestore ondisk finisher threads = 3 2024-04-20T10:19:57.536 INFO:teuthology.orchestra.run.smithi123.stdout: filestore apply finisher threads = 3 2024-04-20T10:19:57.536 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:57.536 INFO:teuthology.orchestra.run.smithi123.stdout: bdev debug aio = true 2024-04-20T10:19:57.536 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug misdirected ops = true 2024-04-20T10:19:57.536 INFO:teuthology.orchestra.run.smithi123.stdout: bluestore block size = 96636764160 2024-04-20T10:19:57.536 INFO:teuthology.orchestra.run.smithi123.stdout: bluestore fsck on mount = True 2024-04-20T10:19:57.536 INFO:teuthology.orchestra.run.smithi123.stdout: debug bluefs = 20 2024-04-20T10:19:57.536 INFO:teuthology.orchestra.run.smithi123.stdout: debug bluestore = 20 2024-04-20T10:19:57.536 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2024-04-20T10:19:57.537 INFO:teuthology.orchestra.run.smithi123.stdout: debug osd = 20 2024-04-20T10:19:57.537 INFO:teuthology.orchestra.run.smithi123.stdout: debug rocksdb = 10 2024-04-20T10:19:57.537 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd backfillfull_ratio = 0.85 2024-04-20T10:19:57.537 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd full ratio = 0.9 2024-04-20T10:19:57.537 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd nearfull ratio = 0.8 2024-04-20T10:19:57.537 INFO:teuthology.orchestra.run.smithi123.stdout: osd failsafe full ratio = 0.95 2024-04-20T10:19:57.537 INFO:teuthology.orchestra.run.smithi123.stdout: osd objectstore = bluestore 2024-04-20T10:19:57.537 INFO:teuthology.orchestra.run.smithi123.stdout: osd op complaint time = 180 2024-04-20T10:19:57.537 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:57.537 INFO:teuthology.orchestra.run.smithi123.stdout:[mgr] 2024-04-20T10:19:57.537 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2024-04-20T10:19:57.537 INFO:teuthology.orchestra.run.smithi123.stdout: debug mgr = 20 2024-04-20T10:19:57.537 INFO:teuthology.orchestra.run.smithi123.stdout: debug mon = 20 2024-04-20T10:19:57.537 INFO:teuthology.orchestra.run.smithi123.stdout: debug auth = 20 2024-04-20T10:19:57.538 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min pgs per osd = 4 2024-04-20T10:19:57.538 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min bytes per osd = 10 2024-04-20T10:19:57.538 INFO:teuthology.orchestra.run.smithi123.stdout: mgr/telemetry/nag = false 2024-04-20T10:19:57.538 INFO:teuthology.orchestra.run.smithi123.stdout: debug client = 20 2024-04-20T10:19:57.538 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:57.538 INFO:teuthology.orchestra.run.smithi123.stdout:[mon] 2024-04-20T10:19:57.538 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2024-04-20T10:19:57.538 INFO:teuthology.orchestra.run.smithi123.stdout: debug mon = 20 2024-04-20T10:19:57.538 INFO:teuthology.orchestra.run.smithi123.stdout: debug paxos = 20 2024-04-20T10:19:57.538 INFO:teuthology.orchestra.run.smithi123.stdout: debug auth = 20 2024-04-20T10:19:57.538 INFO:teuthology.orchestra.run.smithi123.stdout: mon data avail warn = 5 2024-04-20T10:19:57.538 INFO:teuthology.orchestra.run.smithi123.stdout: mon mgr mkfs grace = 240 2024-04-20T10:19:57.538 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min pgs per osd = 4 2024-04-20T10:19:57.538 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd reporter subtree level = osd 2024-04-20T10:19:57.539 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd prime pg temp = true 2024-04-20T10:19:57.539 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min bytes per osd = 10 2024-04-20T10:19:57.539 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:57.539 INFO:teuthology.orchestra.run.smithi123.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-20T10:19:57.539 INFO:teuthology.orchestra.run.smithi123.stdout: auth mon ticket ttl = 660 # 11m 2024-04-20T10:19:57.539 INFO:teuthology.orchestra.run.smithi123.stdout: auth service ticket ttl = 240 # 4m 2024-04-20T10:19:57.539 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:57.539 INFO:teuthology.orchestra.run.smithi123.stdout: # don't complain about insecure global_id in the test suite 2024-04-20T10:19:57.539 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-20T10:19:57.539 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-20T10:19:57.539 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:57.539 INFO:teuthology.orchestra.run.smithi123.stdout: # 1m isn't quite enough 2024-04-20T10:19:57.539 INFO:teuthology.orchestra.run.smithi123.stdout: mon_down_mkfs_grace = 2m 2024-04-20T10:19:57.539 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:57.540 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_filestore_osds = false 2024-04-20T10:19:57.540 INFO:teuthology.orchestra.run.smithi123.stdout: mon down mkfs grace = 300 2024-04-20T10:19:57.540 INFO:teuthology.orchestra.run.smithi123.stdout: mon op complaint time = 120 2024-04-20T10:19:57.540 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-04-20T10:19:57.540 INFO:teuthology.orchestra.run.smithi123.stdout:[client] 2024-04-20T10:19:57.540 INFO:teuthology.orchestra.run.smithi123.stdout: rgw cache enabled = true 2024-04-20T10:19:57.540 INFO:teuthology.orchestra.run.smithi123.stdout: rgw enable ops log = true 2024-04-20T10:19:57.540 INFO:teuthology.orchestra.run.smithi123.stdout: rgw enable usage log = true 2024-04-20T10:19:57.540 INFO:teuthology.orchestra.run.smithi123.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-20T10:19:57.540 INFO:teuthology.orchestra.run.smithi123.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-20T10:19:57.540 INFO:teuthology.orchestra.run.smithi123.stdout: client mount timeout = 600 2024-04-20T10:19:57.540 INFO:teuthology.orchestra.run.smithi123.stdout: debug client = 20 2024-04-20T10:19:57.540 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2024-04-20T10:19:57.541 INFO:teuthology.orchestra.run.smithi123.stdout: pid file = "" 2024-04-20T10:19:57.541 INFO:teuthology.orchestra.run.smithi123.stdout: rados mon op timeout = 900 2024-04-20T10:19:57.541 INFO:teuthology.orchestra.run.smithi123.stdout: rados osd op timeout = 900 2024-04-20T10:19:57.541 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.a] 2024-04-20T10:19:57.541 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.b] 2024-04-20T10:19:57.541 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.c] 2024-04-20T10:19:57.541 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.a] 2024-04-20T10:19:57.541 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.c] 2024-04-20T10:19:57.541 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.b] 2024-04-20T10:19:57.541 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.d] 2024-04-20T10:19:57.541 INFO:teuthology.orchestra.run.smithi123.stdout:[mds] 2024-04-20T10:19:57.541 INFO:teuthology.orchestra.run.smithi123.stdout: debug mds = 20 2024-04-20T10:19:57.541 INFO:teuthology.orchestra.run.smithi123.stdout: debug mds balancer = 20 2024-04-20T10:19:57.541 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2024-04-20T10:19:57.542 INFO:teuthology.orchestra.run.smithi123.stdout: mds debug frag = True 2024-04-20T10:19:57.542 INFO:teuthology.orchestra.run.smithi123.stdout: mds debug scatterstat = True 2024-04-20T10:19:57.542 INFO:teuthology.orchestra.run.smithi123.stdout: mds op complaint time = 180 2024-04-20T10:19:57.542 INFO:teuthology.orchestra.run.smithi123.stdout: mds verify scatter = True 2024-04-20T10:19:57.542 INFO:teuthology.orchestra.run.smithi123.stdout: osd op complaint time = 180 2024-04-20T10:19:57.542 INFO:teuthology.orchestra.run.smithi123.stdout: rados mon op timeout = 900 2024-04-20T10:19:57.542 INFO:teuthology.orchestra.run.smithi123.stdout: rados osd op timeout = 900 2024-04-20T10:19:57.564 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-20T10:19:57.564 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-20T10:19:57.674 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-20T10:19:57.675 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T10:19:57.675 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-20T10:19:57.712 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T10:19:57.712 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-20T10:19:57.790 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi121.front.sepia.ceph.com 2024-04-20T10:19:57.790 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T10:19:57.790 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-20T10:19:57.791 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-20T10:19:57.921 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T10:19:57.921 DEBUG:teuthology.orchestra.run.smithi121:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:19:57.997 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi123.front.sepia.ceph.com 2024-04-20T10:19:57.998 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T10:19:57.998 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-20T10:19:57.998 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-20T10:19:58.118 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T10:19:58.118 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:19:58.213 INFO:tasks.ceph:Setting up mon nodes... 2024-04-20T10:19:58.214 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-20T10:19:58.214 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-20T10:19:58.329 INFO:teuthology.orchestra.run.smithi121.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-20T10:19:58.336 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-20T10:19:58.512 INFO:teuthology.orchestra.run.smithi123.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-20T10:19:58.523 INFO:tasks.ceph:Setting up mds nodes... 2024-04-20T10:19:58.523 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-20T10:19:58.655 INFO:teuthology.orchestra.run.smithi121.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-20T10:19:58.662 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-20T10:19:58.736 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-20T10:19:58.858 INFO:teuthology.orchestra.run.smithi121.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-20T10:19:58.865 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-20T10:19:58.927 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-20T10:19:59.086 INFO:teuthology.orchestra.run.smithi123.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-20T10:19:59.095 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-20T10:19:59.177 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-20T10:19:59.321 INFO:teuthology.orchestra.run.smithi123.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-20T10:19:59.329 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-20T10:19:59.408 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-20T10:19:59.409 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-20T10:19:59.526 INFO:teuthology.orchestra.run.smithi121.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-20T10:19:59.564 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-20T10:19:59.671 INFO:teuthology.orchestra.run.smithi121.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-20T10:19:59.714 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-20T10:19:59.834 INFO:teuthology.orchestra.run.smithi123.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-20T10:19:59.875 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-20T10:19:59.989 INFO:teuthology.orchestra.run.smithi123.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-20T10:20:00.043 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-20T10:20:00.043 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi121.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi123.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-20T10:20:00.043 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-20T10:20:00.120 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-20T10:20:00.120 INFO:tasks.ceph:role: osd.0 2024-04-20T10:20:00.121 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi121.front.sepia.ceph.com 2024-04-20T10:20:00.121 DEBUG:teuthology.orchestra.run.smithi121:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-20T10:20:00.200 INFO:teuthology.orchestra.run.smithi121.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T10:20:00.200 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T10:20:00.200 INFO:teuthology.orchestra.run.smithi121.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T10:20:00.200 INFO:teuthology.orchestra.run.smithi121.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T10:20:00.200 INFO:teuthology.orchestra.run.smithi121.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T10:20:00.200 INFO:teuthology.orchestra.run.smithi121.stdout: = sunit=0 swidth=0 blks 2024-04-20T10:20:00.200 INFO:teuthology.orchestra.run.smithi121.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T10:20:00.200 INFO:teuthology.orchestra.run.smithi121.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T10:20:00.200 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T10:20:00.200 INFO:teuthology.orchestra.run.smithi121.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T10:20:00.614 INFO:teuthology.orchestra.run.smithi121.stdout:Discarding blocks...Done. 2024-04-20T10:20:00.620 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi121.front.sepia.ceph.com -o noatime 2024-04-20T10:20:00.620 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-20T10:20:00.770 DEBUG:teuthology.orchestra.run.smithi121:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-20T10:20:00.848 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-20T10:20:00.917 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-20T10:20:00.917 INFO:tasks.ceph:role: osd.1 2024-04-20T10:20:00.917 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi121.front.sepia.ceph.com 2024-04-20T10:20:00.917 DEBUG:teuthology.orchestra.run.smithi121:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-20T10:20:00.989 INFO:teuthology.orchestra.run.smithi121.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T10:20:00.989 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T10:20:00.989 INFO:teuthology.orchestra.run.smithi121.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T10:20:00.989 INFO:teuthology.orchestra.run.smithi121.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T10:20:00.989 INFO:teuthology.orchestra.run.smithi121.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T10:20:00.989 INFO:teuthology.orchestra.run.smithi121.stdout: = sunit=0 swidth=0 blks 2024-04-20T10:20:00.989 INFO:teuthology.orchestra.run.smithi121.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T10:20:00.989 INFO:teuthology.orchestra.run.smithi121.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T10:20:00.989 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T10:20:00.989 INFO:teuthology.orchestra.run.smithi121.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T10:20:01.346 INFO:teuthology.orchestra.run.smithi121.stdout:Discarding blocks...Done. 2024-04-20T10:20:01.353 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi121.front.sepia.ceph.com -o noatime 2024-04-20T10:20:01.353 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-20T10:20:01.466 DEBUG:teuthology.orchestra.run.smithi121:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-20T10:20:01.541 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-20T10:20:01.606 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-20T10:20:01.606 INFO:tasks.ceph:role: osd.2 2024-04-20T10:20:01.606 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi121.front.sepia.ceph.com 2024-04-20T10:20:01.606 DEBUG:teuthology.orchestra.run.smithi121:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-20T10:20:01.686 INFO:teuthology.orchestra.run.smithi121.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T10:20:01.686 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T10:20:01.686 INFO:teuthology.orchestra.run.smithi121.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T10:20:01.686 INFO:teuthology.orchestra.run.smithi121.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T10:20:01.686 INFO:teuthology.orchestra.run.smithi121.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T10:20:01.687 INFO:teuthology.orchestra.run.smithi121.stdout: = sunit=0 swidth=0 blks 2024-04-20T10:20:01.687 INFO:teuthology.orchestra.run.smithi121.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T10:20:01.687 INFO:teuthology.orchestra.run.smithi121.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T10:20:01.687 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T10:20:01.687 INFO:teuthology.orchestra.run.smithi121.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T10:20:02.087 INFO:teuthology.orchestra.run.smithi121.stdout:Discarding blocks...Done. 2024-04-20T10:20:02.092 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi121.front.sepia.ceph.com -o noatime 2024-04-20T10:20:02.093 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-20T10:20:02.225 DEBUG:teuthology.orchestra.run.smithi121:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-20T10:20:02.309 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-20T10:20:02.390 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-20T10:20:02.390 INFO:tasks.ceph:role: osd.3 2024-04-20T10:20:02.390 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi121.front.sepia.ceph.com 2024-04-20T10:20:02.390 DEBUG:teuthology.orchestra.run.smithi121:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-20T10:20:02.479 INFO:teuthology.orchestra.run.smithi121.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T10:20:02.479 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T10:20:02.479 INFO:teuthology.orchestra.run.smithi121.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T10:20:02.479 INFO:teuthology.orchestra.run.smithi121.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T10:20:02.479 INFO:teuthology.orchestra.run.smithi121.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T10:20:02.479 INFO:teuthology.orchestra.run.smithi121.stdout: = sunit=0 swidth=0 blks 2024-04-20T10:20:02.479 INFO:teuthology.orchestra.run.smithi121.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T10:20:02.479 INFO:teuthology.orchestra.run.smithi121.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T10:20:02.479 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T10:20:02.479 INFO:teuthology.orchestra.run.smithi121.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T10:20:02.895 INFO:teuthology.orchestra.run.smithi121.stdout:Discarding blocks...Done. 2024-04-20T10:20:02.902 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi121.front.sepia.ceph.com -o noatime 2024-04-20T10:20:02.902 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-20T10:20:03.064 DEBUG:teuthology.orchestra.run.smithi121:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-20T10:20:03.162 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:20:03.319 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:03.318+0000 7fc7759dc640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-20T10:20:03.320 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:03.319+0000 7fc7759dc640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-20T10:20:03.320 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:03.319+0000 7fc7759dc640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-20T10:20:03.321 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:03.319+0000 7fc7759dc640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-20T10:20:03.321 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:03.319+0000 7fc7759dc640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-20T10:20:03.323 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:03.323+0000 7fc7759dc640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-20T10:20:06.545 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-20T10:20:06.632 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:20:06.761 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:06.761+0000 7f46f09dc640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-20T10:20:06.761 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:06.761+0000 7f46f09dc640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-20T10:20:06.762 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:06.761+0000 7f46f09dc640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-20T10:20:06.762 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:06.761+0000 7f46f09dc640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-20T10:20:06.763 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:06.761+0000 7f46f09dc640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-20T10:20:06.764 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:06.763+0000 7f46f09dc640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-20T10:20:09.747 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-20T10:20:09.832 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:20:09.946 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:09.946+0000 7efd00e0c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-20T10:20:09.946 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:09.946+0000 7efd00e0c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-20T10:20:09.946 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:09.946+0000 7efd00e0c640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-20T10:20:09.946 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:09.946+0000 7efd00e0c640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-20T10:20:09.946 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:09.946+0000 7efd00e0c640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-20T10:20:09.948 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:09.949+0000 7efd00e0c640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-20T10:20:13.173 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-20T10:20:13.261 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:20:13.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:13.391+0000 7f91c7022640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-20T10:20:13.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:13.391+0000 7f91c7022640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-20T10:20:13.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:13.391+0000 7f91c7022640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-20T10:20:13.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:13.391+0000 7f91c7022640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-20T10:20:13.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:13.391+0000 7f91c7022640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-20T10:20:13.393 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:13.394+0000 7f91c7022640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-20T10:20:16.370 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-20T10:20:16.455 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-20T10:20:16.549 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-20T10:20:16.549 INFO:tasks.ceph:role: osd.4 2024-04-20T10:20:16.549 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi123.front.sepia.ceph.com 2024-04-20T10:20:16.549 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-20T10:20:16.638 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T10:20:16.639 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T10:20:16.639 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T10:20:16.639 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T10:20:16.639 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T10:20:16.639 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2024-04-20T10:20:16.639 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T10:20:16.639 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T10:20:16.639 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T10:20:16.639 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T10:20:17.051 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2024-04-20T10:20:17.058 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2024-04-20T10:20:17.058 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-20T10:20:17.207 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-20T10:20:17.294 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-20T10:20:17.375 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-20T10:20:17.375 INFO:tasks.ceph:role: osd.5 2024-04-20T10:20:17.375 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi123.front.sepia.ceph.com 2024-04-20T10:20:17.375 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-20T10:20:17.458 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T10:20:17.458 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T10:20:17.458 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T10:20:17.458 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T10:20:17.459 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T10:20:17.459 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2024-04-20T10:20:17.459 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T10:20:17.459 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T10:20:17.459 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T10:20:17.459 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T10:20:17.838 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2024-04-20T10:20:17.843 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2024-04-20T10:20:17.843 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-20T10:20:17.961 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-20T10:20:18.052 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-20T10:20:18.115 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-20T10:20:18.115 INFO:tasks.ceph:role: osd.6 2024-04-20T10:20:18.115 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi123.front.sepia.ceph.com 2024-04-20T10:20:18.115 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-20T10:20:18.193 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T10:20:18.193 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T10:20:18.193 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T10:20:18.193 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T10:20:18.193 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T10:20:18.193 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2024-04-20T10:20:18.193 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T10:20:18.193 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T10:20:18.194 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T10:20:18.194 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T10:20:18.605 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2024-04-20T10:20:18.613 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2024-04-20T10:20:18.614 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-20T10:20:18.745 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-20T10:20:18.820 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-20T10:20:18.888 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-20T10:20:18.888 INFO:tasks.ceph:role: osd.7 2024-04-20T10:20:18.888 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi123.front.sepia.ceph.com 2024-04-20T10:20:18.888 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-20T10:20:18.956 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T10:20:18.956 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T10:20:18.956 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T10:20:18.956 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T10:20:18.956 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T10:20:18.956 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2024-04-20T10:20:18.956 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T10:20:18.956 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T10:20:18.956 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T10:20:18.957 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T10:20:19.369 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2024-04-20T10:20:19.375 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2024-04-20T10:20:19.375 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-20T10:20:19.508 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-20T10:20:19.586 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:20:19.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:19.729+0000 7f69e6811640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-20T10:20:19.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:19.730+0000 7f69e6811640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-20T10:20:19.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:19.730+0000 7f69e6811640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-20T10:20:19.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:19.730+0000 7f69e6811640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-20T10:20:19.730 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:19.730+0000 7f69e6811640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-20T10:20:19.733 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:19.733+0000 7f69e6811640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-20T10:20:22.731 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-20T10:20:22.818 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:20:22.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:22.949+0000 7f37ede1c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-20T10:20:22.949 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:22.949+0000 7f37ede1c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-20T10:20:22.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:22.949+0000 7f37ede1c640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-20T10:20:22.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:22.949+0000 7f37ede1c640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-20T10:20:22.950 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:22.949+0000 7f37ede1c640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-20T10:20:22.952 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:22.952+0000 7f37ede1c640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-20T10:20:24.944 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:20:24.952 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:20:25.914 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-20T10:20:25.998 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:20:26.129 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:26.128+0000 7fd127e13640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-20T10:20:26.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:26.129+0000 7fd127e13640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-20T10:20:26.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:26.130+0000 7fd127e13640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-20T10:20:26.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:26.130+0000 7fd127e13640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-20T10:20:26.130 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:26.130+0000 7fd127e13640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-20T10:20:26.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:26.132+0000 7fd127e13640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-20T10:20:29.371 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-20T10:20:29.456 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:20:29.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:29.585+0000 7fd9e4706640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-20T10:20:29.586 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:29.586+0000 7fd9e4706640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-20T10:20:29.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:29.586+0000 7fd9e4706640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-20T10:20:29.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:29.586+0000 7fd9e4706640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-20T10:20:29.587 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:29.586+0000 7fd9e4706640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-20T10:20:29.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:29.588+0000 7fd9e4706640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-20T10:20:32.557 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-20T10:20:32.643 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-20T10:20:32.643 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T10:20:32.644 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-20T10:20:32.730 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T10:20:32.730 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-20T10:20:32.810 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T10:20:32.811 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-20T10:20:32.891 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T10:20:32.891 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-20T10:20:32.969 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T10:20:32.969 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-20T10:20:33.051 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T10:20:33.051 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-20T10:20:33.179 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T10:20:33.179 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-20T10:20:33.261 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T10:20:33.261 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-20T10:20:33.345 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T10:20:33.345 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-20T10:20:33.422 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T10:20:33.422 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-20T10:20:33.504 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T10:20:33.505 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-20T10:20:33.634 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T10:20:33.634 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-20T10:20:33.719 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T10:20:33.719 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-20T10:20:33.806 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T10:20:33.806 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-20T10:20:33.890 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T10:20:33.890 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-20T10:20:33.947 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T10:20:33.947 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-20T10:20:33.999 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T10:20:33.999 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-20T10:20:34.058 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T10:20:34.058 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-20T10:20:34.110 INFO:tasks.ceph:Adding keys to all mons... 2024-04-20T10:20:34.110 DEBUG:teuthology.orchestra.run.smithi121:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-20T10:20:34.117 DEBUG:teuthology.orchestra.run.smithi123:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-20T10:20:34.185 INFO:teuthology.orchestra.run.smithi121.stdout:[mgr.y] 2024-04-20T10:20:34.186 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDOliNmokq5ExAAZUOG88huC1qC8Jq0L377ew== 2024-04-20T10:20:34.186 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.a] 2024-04-20T10:20:34.186 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDOliNmpcYbJxAAt+TxajxrS1sHNHD7ciN9Sg== 2024-04-20T10:20:34.186 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.c] 2024-04-20T10:20:34.186 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDOliNmnDJBMxAA0cFAA1boV1mn/BUzaMxU7g== 2024-04-20T10:20:34.186 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.0] 2024-04-20T10:20:34.186 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDTliNmJWIXExAAK2RnVtCRRoJ+B1T9D+wOLg== 2024-04-20T10:20:34.186 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.1] 2024-04-20T10:20:34.186 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDWliNmaNFtLRAArWQ2gjzzaKKM0EH93+fL0w== 2024-04-20T10:20:34.186 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.2] 2024-04-20T10:20:34.186 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDZliNmd3VvOBAADO8HDWYGf6eGBT3/9tNOBg== 2024-04-20T10:20:34.186 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.3] 2024-04-20T10:20:34.187 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDdliNmfMFZFxAADSklm9tSKlQcw93sDLWJpw== 2024-04-20T10:20:34.187 INFO:teuthology.orchestra.run.smithi121.stdout:[mgr.x] 2024-04-20T10:20:34.187 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDOliNmTemVHhAAV0A7vk9k1brOe/Pb/TwWWg== 2024-04-20T10:20:34.187 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.b] 2024-04-20T10:20:34.187 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDPliNmcsE6BRAA2kx+YMZFBMJGQ8ulIdFW8Q== 2024-04-20T10:20:34.187 INFO:teuthology.orchestra.run.smithi121.stdout:[mds.d] 2024-04-20T10:20:34.187 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDPliNmjLI0ExAA8AMkCBO3j9STy8+ylXRJew== 2024-04-20T10:20:34.187 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.4] 2024-04-20T10:20:34.187 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDjliNmru+LKxAAtIKi2DpRkATWzvsGUdxlkQ== 2024-04-20T10:20:34.187 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.5] 2024-04-20T10:20:34.187 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDmliNmglWkOBAAONYjMcLSddLPq32d5xa5eA== 2024-04-20T10:20:34.187 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.6] 2024-04-20T10:20:34.187 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDqliNmhz3BBxAAOokJT1XFqXHTnDRJKuobDg== 2024-04-20T10:20:34.187 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.7] 2024-04-20T10:20:34.188 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDtliNmovn8IhAAFubL/COYWIO1a4mGZ1VIwA== 2024-04-20T10:20:34.188 INFO:teuthology.orchestra.run.smithi121.stdout:[client.0] 2024-04-20T10:20:34.188 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDPliNmP7t4HxAACvwRBNvbdSP0BmDmaoo32g== 2024-04-20T10:20:34.188 INFO:teuthology.orchestra.run.smithi121.stdout:[client.2] 2024-04-20T10:20:34.188 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDPliNm3TMTKBAAXwbwz8I3RwfoFqLMMvqFhw== 2024-04-20T10:20:34.188 INFO:teuthology.orchestra.run.smithi121.stdout:[client.1] 2024-04-20T10:20:34.188 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDPliNmVjLRMRAAE2gUeLSlsGol8FjxI3UzMg== 2024-04-20T10:20:34.188 INFO:teuthology.orchestra.run.smithi121.stdout:[client.3] 2024-04-20T10:20:34.188 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDPliNm3WkGOxAAPGD1K7WlKziDTqKU/xfqDQ== 2024-04-20T10:20:34.189 INFO:teuthology.orchestra.run.smithi123.stdout:[mgr.y] 2024-04-20T10:20:34.189 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDOliNmokq5ExAAZUOG88huC1qC8Jq0L377ew== 2024-04-20T10:20:34.190 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.a] 2024-04-20T10:20:34.190 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDOliNmpcYbJxAAt+TxajxrS1sHNHD7ciN9Sg== 2024-04-20T10:20:34.190 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.c] 2024-04-20T10:20:34.190 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDOliNmnDJBMxAA0cFAA1boV1mn/BUzaMxU7g== 2024-04-20T10:20:34.190 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.0] 2024-04-20T10:20:34.190 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDTliNmJWIXExAAK2RnVtCRRoJ+B1T9D+wOLg== 2024-04-20T10:20:34.190 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.1] 2024-04-20T10:20:34.190 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDWliNmaNFtLRAArWQ2gjzzaKKM0EH93+fL0w== 2024-04-20T10:20:34.190 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.2] 2024-04-20T10:20:34.190 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDZliNmd3VvOBAADO8HDWYGf6eGBT3/9tNOBg== 2024-04-20T10:20:34.190 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.3] 2024-04-20T10:20:34.190 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDdliNmfMFZFxAADSklm9tSKlQcw93sDLWJpw== 2024-04-20T10:20:34.190 INFO:teuthology.orchestra.run.smithi123.stdout:[mgr.x] 2024-04-20T10:20:34.191 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDOliNmTemVHhAAV0A7vk9k1brOe/Pb/TwWWg== 2024-04-20T10:20:34.191 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.b] 2024-04-20T10:20:34.191 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDPliNmcsE6BRAA2kx+YMZFBMJGQ8ulIdFW8Q== 2024-04-20T10:20:34.191 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.d] 2024-04-20T10:20:34.191 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDPliNmjLI0ExAA8AMkCBO3j9STy8+ylXRJew== 2024-04-20T10:20:34.191 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.4] 2024-04-20T10:20:34.191 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDjliNmru+LKxAAtIKi2DpRkATWzvsGUdxlkQ== 2024-04-20T10:20:34.191 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.5] 2024-04-20T10:20:34.191 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDmliNmglWkOBAAONYjMcLSddLPq32d5xa5eA== 2024-04-20T10:20:34.191 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.6] 2024-04-20T10:20:34.191 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDqliNmhz3BBxAAOokJT1XFqXHTnDRJKuobDg== 2024-04-20T10:20:34.191 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.7] 2024-04-20T10:20:34.191 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDtliNmovn8IhAAFubL/COYWIO1a4mGZ1VIwA== 2024-04-20T10:20:34.191 INFO:teuthology.orchestra.run.smithi123.stdout:[client.0] 2024-04-20T10:20:34.191 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDPliNmP7t4HxAACvwRBNvbdSP0BmDmaoo32g== 2024-04-20T10:20:34.192 INFO:teuthology.orchestra.run.smithi123.stdout:[client.2] 2024-04-20T10:20:34.192 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDPliNm3TMTKBAAXwbwz8I3RwfoFqLMMvqFhw== 2024-04-20T10:20:34.192 INFO:teuthology.orchestra.run.smithi123.stdout:[client.1] 2024-04-20T10:20:34.192 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDPliNmVjLRMRAAE2gUeLSlsGol8FjxI3UzMg== 2024-04-20T10:20:34.192 INFO:teuthology.orchestra.run.smithi123.stdout:[client.3] 2024-04-20T10:20:34.192 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQDPliNm3WkGOxAAPGD1K7WlKziDTqKU/xfqDQ== 2024-04-20T10:20:34.193 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-20T10:20:34.238 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-20T10:20:34.375 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T10:20:34.382 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T10:20:34.519 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T10:20:34.523 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T10:20:34.649 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:20:34.654 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:20:34.785 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:20:34.792 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:20:34.927 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:20:34.932 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:20:35.067 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:20:35.074 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:20:35.208 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-20T10:20:35.213 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-20T10:20:35.345 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T10:20:35.353 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T10:20:35.488 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T10:20:35.495 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T10:20:35.623 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:20:35.628 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:20:35.753 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:20:35.758 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:20:35.885 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:20:35.890 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:20:36.018 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:20:36.025 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:20:36.151 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T10:20:36.155 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T10:20:36.287 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T10:20:36.292 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T10:20:36.425 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T10:20:36.430 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T10:20:36.565 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T10:20:36.569 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T10:20:36.704 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-20T10:20:36.704 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-20T10:20:36.790 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-20T10:20:36.935 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-20T10:20:37.005 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-20T10:20:37.094 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-20T10:20:37.242 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-20T10:20:37.305 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-20T10:20:37.385 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-20T10:20:37.523 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-20T10:20:37.597 DEBUG:teuthology.orchestra.run.smithi121:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:20:37.605 DEBUG:teuthology.orchestra.run.smithi123:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:20:37.657 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-20T10:20:37.657 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-20T10:20:37.658 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-20T10:20:37.664 INFO:tasks.ceph.mon.a:Started 2024-04-20T10:20:37.664 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-20T10:20:37.664 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-20T10:20:37.671 INFO:tasks.ceph.mon.b:Started 2024-04-20T10:20:37.671 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-20T10:20:37.671 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-20T10:20:37.677 INFO:tasks.ceph.mon.c:Started 2024-04-20T10:20:37.677 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-20T10:20:37.678 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-20T10:20:37.678 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-20T10:20:37.684 INFO:tasks.ceph.mgr.y:Started 2024-04-20T10:20:37.684 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-20T10:20:37.684 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-20T10:20:37.688 INFO:tasks.ceph.mgr.x:Started 2024-04-20T10:20:37.688 INFO:tasks.ceph:Setting crush tunables to default 2024-04-20T10:20:37.689 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd crush tunables default 2024-04-20T10:20:37.859 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:37.859+0000 7f82d7000640 1 Processor -- start 2024-04-20T10:20:37.859 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:37.859+0000 7f82d7000640 1 -- start start 2024-04-20T10:20:37.861 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:37.860+0000 7f82d7000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f82d0161a50 0x7f82d00b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:37.862 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:37.860+0000 7f82d7000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f82d00d3b20 0x7f82d006b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:37.862 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:37.860+0000 7f82d7000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82d00d0980 con 0x7f82d00d0480 2024-04-20T10:20:37.862 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:37.860+0000 7f82d7000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f82d00d0c60 con 0x7f82d0161a50 2024-04-20T10:20:37.862 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:37.860+0000 7f82d7000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f82d0162ae0 con 0x7f82d00d3b20 2024-04-20T10:20:37.862 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:37.860+0000 7f82d5c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f82d0161a50 0x7f82d00b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:37.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:37.860+0000 7f82d5c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f82d0161a50 0x7f82d00b3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36520/0 (socket says 172.21.15.121:36520) 2024-04-20T10:20:37.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:37.861+0000 7f82d5c00640 1 -- 172.21.15.121:0/3504355103 learned_addr learned my addr 172.21.15.121:0/3504355103 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:20:37.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:37.861+0000 7f82d5c00640 1 -- 172.21.15.121:0/3504355103 >> v2:172.21.15.121:3300/0 conn(0x7f82d0161a50 msgr2=0x7f82d00b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T10:20:37.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:37.861+0000 7f82d5c00640 1 -- 172.21.15.121:0/3504355103 >> v2:172.21.15.121:3300/0 conn(0x7f82d0161a50 msgr2=0x7f82d00b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T10:20:37.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:37.861+0000 7f82d5c00640 1 --2- 172.21.15.121:0/3504355103 >> v2:172.21.15.121:3300/0 conn(0x7f82d0161a50 0x7f82d00b3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T10:20:37.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:37.861+0000 7f82d5200640 1 --2- 172.21.15.121:0/3504355103 >> v2:172.21.15.123:3300/0 conn(0x7f82d00d3b20 0x7f82d006b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:37.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:37.861+0000 7f82d5c00640 1 --2- 172.21.15.121:0/3504355103 >> v2:172.21.15.121:3300/0 conn(0x7f82d0161a50 0x7f82d00b3180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T10:20:37.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:37.861+0000 7f82d5200640 1 -- 172.21.15.121:0/3504355103 >> v2:172.21.15.123:3300/0 conn(0x7f82d00d3b20 msgr2=0x7f82d006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T10:20:37.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:37.861+0000 7f82d5200640 1 -- 172.21.15.121:0/3504355103 >> v2:172.21.15.123:3300/0 conn(0x7f82d00d3b20 msgr2=0x7f82d006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T10:20:37.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:37.861+0000 7f82d5200640 1 --2- 172.21.15.121:0/3504355103 >> v2:172.21.15.123:3300/0 conn(0x7f82d00d3b20 0x7f82d006b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T10:20:37.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:37.861+0000 7f82d5200640 1 --2- 172.21.15.121:0/3504355103 >> v2:172.21.15.123:3300/0 conn(0x7f82d00d3b20 0x7f82d006b900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T10:20:38.062 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.062+0000 7f82d5c00640 1 --2- 172.21.15.121:0/3504355103 >> v2:172.21.15.121:3300/0 conn(0x7f82d0161a50 0x7f82d00b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:38.062 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.062+0000 7f82d5200640 1 --2- 172.21.15.121:0/3504355103 >> v2:172.21.15.123:3300/0 conn(0x7f82d00d3b20 0x7f82d006b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:38.062 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.063+0000 7f82d5c00640 1 -- 172.21.15.121:0/3504355103 >> v2:172.21.15.121:3300/0 conn(0x7f82d0161a50 msgr2=0x7f82d00b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-20T10:20:38.062 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.063+0000 7f82d5c00640 1 -- 172.21.15.121:0/3504355103 >> v2:172.21.15.121:3300/0 conn(0x7f82d0161a50 msgr2=0x7f82d00b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T10:20:38.062 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.063+0000 7f82d5c00640 1 --2- 172.21.15.121:0/3504355103 >> v2:172.21.15.121:3300/0 conn(0x7f82d0161a50 0x7f82d00b3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T10:20:38.063 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.063+0000 7f82d5c00640 1 --2- 172.21.15.121:0/3504355103 >> v2:172.21.15.121:3300/0 conn(0x7f82d0161a50 0x7f82d00b3180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T10:20:38.063 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.063+0000 7f82d5200640 1 -- 172.21.15.121:0/3504355103 >> v2:172.21.15.123:3300/0 conn(0x7f82d00d3b20 msgr2=0x7f82d006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T10:20:38.063 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.063+0000 7f82d5200640 1 -- 172.21.15.121:0/3504355103 >> v2:172.21.15.123:3300/0 conn(0x7f82d00d3b20 msgr2=0x7f82d006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T10:20:38.063 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.063+0000 7f82d5200640 1 --2- 172.21.15.121:0/3504355103 >> v2:172.21.15.123:3300/0 conn(0x7f82d00d3b20 0x7f82d006b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T10:20:38.063 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.063+0000 7f82d5200640 1 --2- 172.21.15.121:0/3504355103 >> v2:172.21.15.123:3300/0 conn(0x7f82d00d3b20 0x7f82d006b900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-20T10:20:38.110 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.111+0000 7f82cfe00640 1 -- 172.21.15.121:0/3504355103 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 617132828 0 0) 0x7f82d00d0980 con 0x7f82d00d0480 2024-04-20T10:20:38.111 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.111+0000 7f82cfe00640 1 -- 172.21.15.121:0/3504355103 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f82b8003520 con 0x7f82d00d0480 2024-04-20T10:20:38.118 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.117+0000 7f82cfe00640 1 -- 172.21.15.121:0/3504355103 <== mon.1 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2222314457 0 0) 0x7f82b8003520 con 0x7f82d00d0480 2024-04-20T10:20:38.118 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.117+0000 7f82cfe00640 1 -- 172.21.15.121:0/3504355103 >> v2:172.21.15.123:3300/0 conn(0x7f82d00d3b20 msgr2=0x7f82d006b900 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T10:20:38.118 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.117+0000 7f82cfe00640 1 --2- 172.21.15.121:0/3504355103 >> v2:172.21.15.123:3300/0 conn(0x7f82d00d3b20 0x7f82d006b900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:38.118 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.117+0000 7f82cfe00640 1 -- 172.21.15.121:0/3504355103 >> v2:172.21.15.121:3300/0 conn(0x7f82d0161a50 msgr2=0x7f82d00b3180 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T10:20:38.118 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.117+0000 7f82cfe00640 1 --2- 172.21.15.121:0/3504355103 >> v2:172.21.15.121:3300/0 conn(0x7f82d0161a50 0x7f82d00b3180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:38.118 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.118+0000 7f82cfe00640 1 -- 172.21.15.121:0/3504355103 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d0162490 con 0x7f82d00d0480 2024-04-20T10:20:38.119 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.118+0000 7f82cfe00640 1 -- 172.21.15.121:0/3504355103 <== mon.1 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f82c80019f0 con 0x7f82d00d0480 2024-04-20T10:20:38.119 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.118+0000 7f82cfe00640 1 -- 172.21.15.121:0/3504355103 >> v1:172.21.15.123:6789/0 conn(0x7f82d00d0480 legacy=0x7f82d006c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:38.119 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.118+0000 7f82cfe00640 1 --2- 172.21.15.121:0/3504355103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82b80039b0 0x7f82b8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:38.119 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.118+0000 7f82cfe00640 1 --2- 172.21.15.121:0/3504355103 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82b8004800 0x7f82b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:38.119 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.118+0000 7f82cfe00640 1 --2- 172.21.15.121:0/3504355103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82b80051c0 0x7f82b800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:38.119 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.118+0000 7f82cfe00640 1 -- 172.21.15.121:0/3504355103 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f82d00d0c60 con 0x7f82b80039b0 2024-04-20T10:20:38.119 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.118+0000 7f82cfe00640 1 -- 172.21.15.121:0/3504355103 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f82d0162ae0 con 0x7f82b80051c0 2024-04-20T10:20:38.119 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.118+0000 7f82cfe00640 1 -- 172.21.15.121:0/3504355103 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f82b800c0c0 con 0x7f82b8004800 2024-04-20T10:20:38.122 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.119+0000 7f82d5200640 1 --2- 172.21.15.121:0/3504355103 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82b8004800 0x7f82b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:38.122 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.119+0000 7f82d6600640 1 --2- 172.21.15.121:0/3504355103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82b80051c0 0x7f82b800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:38.122 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.119+0000 7f82d5c00640 1 --2- 172.21.15.121:0/3504355103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82b80039b0 0x7f82b8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:38.122 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.120+0000 7f82d5200640 1 -- 172.21.15.121:0/3504355103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82b80051c0 msgr2=0x7f82b800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:20:38.122 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.120+0000 7f82d5200640 1 --2- 172.21.15.121:0/3504355103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82b80051c0 0x7f82b800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:38.122 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.120+0000 7f82d5200640 1 -- 172.21.15.121:0/3504355103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82b80039b0 msgr2=0x7f82b8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:20:38.122 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.120+0000 7f82d5200640 1 --2- 172.21.15.121:0/3504355103 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82b80039b0 0x7f82b8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:38.122 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.120+0000 7f82d5200640 1 -- 172.21.15.121:0/3504355103 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d0162490 con 0x7f82b8004800 2024-04-20T10:20:38.123 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.121+0000 7f82d6600640 1 -- 172.21.15.121:0/3504355103 reap_dead start 2024-04-20T10:20:38.123 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.123+0000 7f82d5200640 1 --2- 172.21.15.121:0/3504355103 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82b8004800 0x7f82b8004c80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f82bc006fd0 tx=0x7f82b8005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a1501e78a0dc8e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:38.123 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.124+0000 7f82cfe00640 1 -- 172.21.15.121:0/3504355103 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc032070 con 0x7f82b8004800 2024-04-20T10:20:38.123 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.124+0000 7f82cfe00640 1 -- 172.21.15.121:0/3504355103 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82bc02c770 con 0x7f82b8004800 2024-04-20T10:20:38.123 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.124+0000 7f82cfe00640 1 -- 172.21.15.121:0/3504355103 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc00d690 con 0x7f82b8004800 2024-04-20T10:20:38.125 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.125+0000 7f82d7000640 1 -- 172.21.15.121:0/3504355103 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82b8004800 msgr2=0x7f82b8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:38.125 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.125+0000 7f82d7000640 1 --2- 172.21.15.121:0/3504355103 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82b8004800 0x7f82b8004c80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f82bc006fd0 tx=0x7f82b8005830 comp rx=0 tx=0).stop 2024-04-20T10:20:38.125 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.125+0000 7f82d7000640 1 -- 172.21.15.121:0/3504355103 shutdown_connections 2024-04-20T10:20:38.125 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.125+0000 7f82d7000640 1 --2- 172.21.15.121:0/3504355103 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82b8004800 0x7f82b8004c80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:38.125 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.125+0000 7f82d7000640 1 -- 172.21.15.121:0/3504355103 >> 172.21.15.121:0/3504355103 conn(0x7f82d00d0ed0 msgr2=0x7f82d00d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:20:38.125 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.126+0000 7f82d7000640 1 -- 172.21.15.121:0/3504355103 shutdown_connections 2024-04-20T10:20:38.125 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.126+0000 7f82d7000640 1 -- 172.21.15.121:0/3504355103 wait complete. 2024-04-20T10:20:38.129 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.130+0000 7f82d7000640 1 Processor -- start 2024-04-20T10:20:38.130 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.130+0000 7f82d7000640 1 -- start start 2024-04-20T10:20:38.132 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.132+0000 7f82d7000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82b8004800 0x7f82d00cf430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:38.132 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.132+0000 7f82d7000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82d00cae80 0x7f82d015a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:38.132 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.132+0000 7f82d7000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82d015a880 0x7f82d006c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:38.132 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.132+0000 7f82d7000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f82d0161fb0 con 0x7f82d015a880 2024-04-20T10:20:38.132 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.132+0000 7f82d7000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f82d0161e30 con 0x7f82d00cae80 2024-04-20T10:20:38.132 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.132+0000 7f82d7000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f82d0162130 con 0x7f82b8004800 2024-04-20T10:20:38.132 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.132+0000 7f82d5c00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82b8004800 0x7f82d00cf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:38.132 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.132+0000 7f82d5200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82d00cae80 0x7f82d015a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:38.132 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.132+0000 7f82d6600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82d015a880 0x7f82d006c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:38.133 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.132+0000 7f82d5c00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82b8004800 0x7f82d00cf430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.121:42306/0 (socket says 172.21.15.121:42306) 2024-04-20T10:20:38.133 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.132+0000 7f82d5c00640 1 -- 172.21.15.121:0/2955936743 learned_addr learned my addr 172.21.15.121:0/2955936743 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:20:38.134 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.133+0000 7f82d5200640 1 -- 172.21.15.121:0/2955936743 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82b8004800 msgr2=0x7f82d00cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:38.134 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.133+0000 7f82d5200640 1 --2- 172.21.15.121:0/2955936743 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82b8004800 0x7f82d00cf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:38.134 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.133+0000 7f82d5200640 1 -- 172.21.15.121:0/2955936743 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82d015a880 msgr2=0x7f82d006c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:38.134 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.133+0000 7f82d5200640 1 --2- 172.21.15.121:0/2955936743 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82d015a880 0x7f82d006c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:38.134 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.133+0000 7f82d5200640 1 -- 172.21.15.121:0/2955936743 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d00ce3d0 con 0x7f82d00cae80 2024-04-20T10:20:38.134 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.133+0000 7f82d6600640 1 --2- 172.21.15.121:0/2955936743 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82d015a880 0x7f82d006c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:20:38.134 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.133+0000 7f82d5200640 1 --2- 172.21.15.121:0/2955936743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82d00cae80 0x7f82d015a340 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f82b800c290 tx=0x7f82bc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:38.135 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.134+0000 7f82cd600640 1 -- 172.21.15.121:0/2955936743 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc044020 con 0x7f82d00cae80 2024-04-20T10:20:38.135 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.134+0000 7f82cd600640 1 -- 172.21.15.121:0/2955936743 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82bc02c570 con 0x7f82d00cae80 2024-04-20T10:20:38.135 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.134+0000 7f82cd600640 1 -- 172.21.15.121:0/2955936743 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82bc00d690 con 0x7f82d00cae80 2024-04-20T10:20:38.135 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.134+0000 7f82d7000640 1 -- 172.21.15.121:0/2955936743 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82d00ce670 con 0x7f82d00cae80 2024-04-20T10:20:38.135 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.134+0000 7f82d7000640 1 -- 172.21.15.121:0/2955936743 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82d00cec30 con 0x7f82d00cae80 2024-04-20T10:20:38.135 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.134+0000 7f82cd600640 1 -- 172.21.15.121:0/2955936743 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f82bc047020 con 0x7f82d00cae80 2024-04-20T10:20:38.135 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.134+0000 7f82cd600640 1 -- 172.21.15.121:0/2955936743 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f82bc049400 con 0x7f82d00cae80 2024-04-20T10:20:38.135 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.135+0000 7f82d7000640 1 -- 172.21.15.121:0/2955936743 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82d0075b30 con 0x7f82d00cae80 2024-04-20T10:20:38.138 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.139+0000 7f82cd600640 1 -- 172.21.15.121:0/2955936743 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f82bc04e030 con 0x7f82d00cae80 2024-04-20T10:20:38.213 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.213+0000 7f82d7000640 1 -- 172.21.15.121:0/2955936743 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f82d00cb300 con 0x7f82d00cae80 2024-04-20T10:20:38.220 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.221+0000 7f82cd600640 1 -- 172.21.15.121:0/2955936743 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f82bc02c720 con 0x7f82d00cae80 2024-04-20T10:20:38.220 INFO:teuthology.orchestra.run.smithi121.stderr:adjusted tunables profile to default 2024-04-20T10:20:38.222 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.222+0000 7f82d7000640 1 -- 172.21.15.121:0/2955936743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82d00cae80 msgr2=0x7f82d015a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:38.222 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.222+0000 7f82d7000640 1 --2- 172.21.15.121:0/2955936743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82d00cae80 0x7f82d015a340 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f82b800c290 tx=0x7f82bc0046c0 comp rx=0 tx=0).stop 2024-04-20T10:20:38.222 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.222+0000 7f82d7000640 1 -- 172.21.15.121:0/2955936743 shutdown_connections 2024-04-20T10:20:38.222 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.222+0000 7f82d7000640 1 --2- 172.21.15.121:0/2955936743 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82d015a880 0x7f82d006c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:38.222 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.222+0000 7f82d7000640 1 --2- 172.21.15.121:0/2955936743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82d00cae80 0x7f82d015a340 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:38.222 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.222+0000 7f82d7000640 1 --2- 172.21.15.121:0/2955936743 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82b8004800 0x7f82d00cf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:38.222 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.222+0000 7f82d7000640 1 -- 172.21.15.121:0/2955936743 >> 172.21.15.121:0/2955936743 conn(0x7f82d00d0ed0 msgr2=0x7f82d00b7c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:20:38.222 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.222+0000 7f82d7000640 1 -- 172.21.15.121:0/2955936743 shutdown_connections 2024-04-20T10:20:38.223 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:38.222+0000 7f82d7000640 1 -- 172.21.15.121:0/2955936743 wait complete. 2024-04-20T10:20:38.239 INFO:tasks.ceph:check_enable_crimson: False 2024-04-20T10:20:38.240 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-20T10:20:38.240 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T10:20:38.240 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-20T10:20:38.277 INFO:tasks.ceph.mgr.y.smithi121.stderr:2024-04-20T10:20:38.277+0000 7f985f644040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T10:20:38.299 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-04-20T10:20:38.299+0000 7f0707812040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T10:20:38.309 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T10:20:38.309 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-20T10:20:38.423 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T10:20:38.424 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-20T10:20:38.471 INFO:tasks.ceph.mgr.y.smithi121.stderr:2024-04-20T10:20:38.469+0000 7f985f644040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T10:20:38.490 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-04-20T10:20:38.489+0000 7f0707812040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T10:20:38.535 INFO:tasks.ceph.mgr.y.smithi121.stderr:2024-04-20T10:20:38.536+0000 7f985f644040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T10:20:38.540 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T10:20:38.540 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-20T10:20:38.556 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-04-20T10:20:38.556+0000 7f0707812040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T10:20:38.653 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T10:20:38.654 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-20T10:20:38.729 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T10:20:38.729 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-20T10:20:38.838 INFO:tasks.ceph.mgr.y.smithi121.stderr:2024-04-20T10:20:38.838+0000 7f985f644040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T10:20:38.844 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T10:20:38.844 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-20T10:20:38.884 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-04-20T10:20:38.884+0000 7f0707812040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T10:20:38.891 INFO:tasks.ceph.mgr.y.smithi121.stderr:2024-04-20T10:20:38.891+0000 7f985f644040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T10:20:38.943 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-04-20T10:20:38.943+0000 7f0707812040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T10:20:38.966 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T10:20:38.966 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-20T10:20:39.036 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new 629a7b8c-a40a-4a2a-a3cc-d9f357774310 0 2024-04-20T10:20:39.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.262+0000 7fe327e00640 1 Processor -- start 2024-04-20T10:20:39.263 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.263+0000 7fe327e00640 1 -- start start 2024-04-20T10:20:39.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.264+0000 7fe327e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe3280d46f0 0x7fe3280d4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:39.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.264+0000 7fe327e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3280d15b0 con 0x7fe32815e8c0 2024-04-20T10:20:39.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.264+0000 7fe327e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3280d1890 con 0x7fe3280d1010 2024-04-20T10:20:39.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.264+0000 7fe327e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fe3280b6d10 con 0x7fe3280d46f0 2024-04-20T10:20:39.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.264+0000 7fe326000640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fe3280d1010 0x7fe3280b6600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56068/0 (socket says 172.21.15.123:56068) 2024-04-20T10:20:39.264 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.264+0000 7fe326000640 1 -- 172.21.15.123:0/330589800 learned_addr learned my addr 172.21.15.123:0/330589800 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-20T10:20:39.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.265+0000 7fe327400640 1 --2- 172.21.15.123:0/330589800 >> v2:172.21.15.121:3300/0 conn(0x7fe3280d46f0 0x7fe3280d4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:39.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.265+0000 7fe325600640 1 -- 172.21.15.123:0/330589800 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3197464364 0 0) 0x7fe3280d1890 con 0x7fe3280d1010 2024-04-20T10:20:39.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.265+0000 7fe325600640 1 -- 172.21.15.123:0/330589800 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe30c003520 con 0x7fe3280d1010 2024-04-20T10:20:39.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.265+0000 7fe325600640 1 -- 172.21.15.123:0/330589800 <== mon.1 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3639443923 0 0) 0x7fe30c003520 con 0x7fe3280d1010 2024-04-20T10:20:39.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.265+0000 7fe325600640 1 -- 172.21.15.123:0/330589800 >> v2:172.21.15.121:3300/0 conn(0x7fe3280d46f0 msgr2=0x7fe3280d4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:20:39.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.265+0000 7fe325600640 1 --2- 172.21.15.123:0/330589800 >> v2:172.21.15.121:3300/0 conn(0x7fe3280d46f0 0x7fe3280d4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:39.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.265+0000 7fe325600640 1 -- 172.21.15.123:0/330589800 >> v1:172.21.15.121:6789/0 conn(0x7fe32815e8c0 legacy=0x7fe32806ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:39.267 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.265+0000 7fe325600640 1 -- 172.21.15.123:0/330589800 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3280d1b70 con 0x7fe3280d1010 2024-04-20T10:20:39.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.265+0000 7fe325600640 1 -- 172.21.15.123:0/330589800 <== mon.1 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7fe314001670 con 0x7fe3280d1010 2024-04-20T10:20:39.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.265+0000 7fe325600640 1 -- 172.21.15.123:0/330589800 >> v1:172.21.15.123:6789/0 conn(0x7fe3280d1010 legacy=0x7fe3280b6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:39.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.265+0000 7fe325600640 1 --2- 172.21.15.123:0/330589800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe30c0039b0 0x7fe30c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:39.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.265+0000 7fe325600640 1 --2- 172.21.15.123:0/330589800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe30c004800 0x7fe30c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:39.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.265+0000 7fe325600640 1 --2- 172.21.15.123:0/330589800 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe30c0051c0 0x7fe30c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:39.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.265+0000 7fe325600640 1 -- 172.21.15.123:0/330589800 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe3280b6d10 con 0x7fe30c0039b0 2024-04-20T10:20:39.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.265+0000 7fe325600640 1 -- 172.21.15.123:0/330589800 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe30c00c060 con 0x7fe30c004800 2024-04-20T10:20:39.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.265+0000 7fe325600640 1 -- 172.21.15.123:0/330589800 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe30c00c260 con 0x7fe30c0051c0 2024-04-20T10:20:39.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.266+0000 7fe326000640 1 --2- 172.21.15.123:0/330589800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe30c004800 0x7fe30c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:39.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.266+0000 7fe327400640 1 --2- 172.21.15.123:0/330589800 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe30c0051c0 0x7fe30c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:39.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.266+0000 7fe326a00640 1 --2- 172.21.15.123:0/330589800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe30c0039b0 0x7fe30c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:39.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.266+0000 7fe326000640 1 -- 172.21.15.123:0/330589800 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe30c0051c0 msgr2=0x7fe30c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:20:39.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.266+0000 7fe326000640 1 --2- 172.21.15.123:0/330589800 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe30c0051c0 0x7fe30c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:39.268 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.266+0000 7fe326000640 1 -- 172.21.15.123:0/330589800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe30c0039b0 msgr2=0x7fe30c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:20:39.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.266+0000 7fe326000640 1 --2- 172.21.15.123:0/330589800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe30c0039b0 0x7fe30c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:39.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.266+0000 7fe326000640 1 -- 172.21.15.123:0/330589800 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe30c003520 con 0x7fe30c004800 2024-04-20T10:20:39.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.266+0000 7fe326a00640 1 --2- 172.21.15.123:0/330589800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe30c0039b0 0x7fe30c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:20:39.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.266+0000 7fe327400640 1 -- 172.21.15.123:0/330589800 reap_dead start 2024-04-20T10:20:39.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.267+0000 7fe326000640 1 --2- 172.21.15.123:0/330589800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe30c004800 0x7fe30c004c80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe31400c240 tx=0x7fe30c005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee109228cb99012f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:39.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.267+0000 7fe325600640 1 -- 172.21.15.123:0/330589800 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe314007d10 con 0x7fe30c004800 2024-04-20T10:20:39.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.267+0000 7fe325600640 1 -- 172.21.15.123:0/330589800 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe314030070 con 0x7fe30c004800 2024-04-20T10:20:39.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.267+0000 7fe325600640 1 -- 172.21.15.123:0/330589800 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe314005d90 con 0x7fe30c004800 2024-04-20T10:20:39.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.268+0000 7fe327e00640 1 -- 172.21.15.123:0/330589800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe30c004800 msgr2=0x7fe30c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:39.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.268+0000 7fe327e00640 1 --2- 172.21.15.123:0/330589800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe30c004800 0x7fe30c004c80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe31400c240 tx=0x7fe30c005830 comp rx=0 tx=0).stop 2024-04-20T10:20:39.270 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.268+0000 7fe327e00640 1 -- 172.21.15.123:0/330589800 shutdown_connections 2024-04-20T10:20:39.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.268+0000 7fe327e00640 1 --2- 172.21.15.123:0/330589800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe30c004800 0x7fe30c004c80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:39.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.268+0000 7fe327e00640 1 -- 172.21.15.123:0/330589800 >> 172.21.15.123:0/330589800 conn(0x7fe32809d6f0 msgr2=0x7fe3280b9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:20:39.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.268+0000 7fe327e00640 1 -- 172.21.15.123:0/330589800 shutdown_connections 2024-04-20T10:20:39.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.268+0000 7fe327e00640 1 -- 172.21.15.123:0/330589800 wait complete. 2024-04-20T10:20:39.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.270+0000 7fe327e00640 1 Processor -- start 2024-04-20T10:20:39.271 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.270+0000 7fe327e00640 1 -- start start 2024-04-20T10:20:39.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.271+0000 7fe327e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe30c004800 0x7fe32806b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:39.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.271+0000 7fe327e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe32815ba30 0x7fe328152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:39.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.271+0000 7fe327e00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe3281f4490 0x7fe3280cd460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:39.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.271+0000 7fe327e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe3280d4b40 con 0x7fe32815ba30 2024-04-20T10:20:39.273 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.271+0000 7fe327e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe3280b6170 con 0x7fe30c004800 2024-04-20T10:20:39.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.271+0000 7fe327e00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe3280d4cc0 con 0x7fe3281f4490 2024-04-20T10:20:39.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.271+0000 7fe326a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe30c004800 0x7fe32806b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:39.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.271+0000 7fe326a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe30c004800 0x7fe32806b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52458/0 (socket says 172.21.15.123:52458) 2024-04-20T10:20:39.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.271+0000 7fe327400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe3281f4490 0x7fe3280cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:39.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.271+0000 7fe326a00640 1 -- 172.21.15.123:0/2111311508 learned_addr learned my addr 172.21.15.123:0/2111311508 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-20T10:20:39.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.271+0000 7fe326a00640 1 -- 172.21.15.123:0/2111311508 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe3281f4490 msgr2=0x7fe3280cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:39.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.271+0000 7fe326a00640 1 --2- 172.21.15.123:0/2111311508 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe3281f4490 0x7fe3280cd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:39.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.271+0000 7fe326a00640 1 -- 172.21.15.123:0/2111311508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe32815ba30 msgr2=0x7fe328152870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:39.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.271+0000 7fe326a00640 1 --2- 172.21.15.123:0/2111311508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe32815ba30 0x7fe328152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:39.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.272+0000 7fe326a00640 1 -- 172.21.15.123:0/2111311508 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe32806eb30 con 0x7fe30c004800 2024-04-20T10:20:39.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.272+0000 7fe327400640 1 --2- 172.21.15.123:0/2111311508 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe3281f4490 0x7fe3280cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:20:39.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.272+0000 7fe326a00640 1 --2- 172.21.15.123:0/2111311508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe30c004800 0x7fe32806b900 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe31800e410 tx=0x7fe31800e8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:39.274 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.272+0000 7fe31ea00640 1 -- 172.21.15.123:0/2111311508 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe318019070 con 0x7fe30c004800 2024-04-20T10:20:39.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.272+0000 7fe31ea00640 1 -- 172.21.15.123:0/2111311508 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3180052e0 con 0x7fe30c004800 2024-04-20T10:20:39.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.272+0000 7fe327e00640 1 -- 172.21.15.123:0/2111311508 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3281603b0 con 0x7fe30c004800 2024-04-20T10:20:39.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.272+0000 7fe31ea00640 1 -- 172.21.15.123:0/2111311508 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe318005490 con 0x7fe30c004800 2024-04-20T10:20:39.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.272+0000 7fe327e00640 1 -- 172.21.15.123:0/2111311508 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe328160930 con 0x7fe30c004800 2024-04-20T10:20:39.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.272+0000 7fe31ea00640 1 -- 172.21.15.123:0/2111311508 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe31801c070 con 0x7fe30c004800 2024-04-20T10:20:39.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.273+0000 7fe31ea00640 1 -- 172.21.15.123:0/2111311508 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fe318014ce0 con 0x7fe30c004800 2024-04-20T10:20:39.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.273+0000 7fe327e00640 1 -- 172.21.15.123:0/2111311508 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe328075b30 con 0x7fe30c004800 2024-04-20T10:20:39.275 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.276+0000 7fe31ea00640 1 -- 172.21.15.123:0/2111311508 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7fe318014480 con 0x7fe30c004800 2024-04-20T10:20:39.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.354+0000 7fe327e00640 1 -- 172.21.15.123:0/2111311508 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd new", "uuid": "629a7b8c-a40a-4a2a-a3cc-d9f357774310", "id": 0} v 0) -- 0x7fe3280d3880 con 0x7fe30c004800 2024-04-20T10:20:39.362 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.362+0000 7fe31ea00640 1 -- 172.21.15.123:0/2111311508 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "629a7b8c-a40a-4a2a-a3cc-d9f357774310", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fe318003830 con 0x7fe30c004800 2024-04-20T10:20:39.362 INFO:teuthology.orchestra.run.smithi123.stdout:0 2024-04-20T10:20:39.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.363+0000 7fe327e00640 1 -- 172.21.15.123:0/2111311508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe30c004800 msgr2=0x7fe32806b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:39.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.363+0000 7fe327e00640 1 --2- 172.21.15.123:0/2111311508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe30c004800 0x7fe32806b900 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe31800e410 tx=0x7fe31800e8d0 comp rx=0 tx=0).stop 2024-04-20T10:20:39.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.363+0000 7fe327e00640 1 -- 172.21.15.123:0/2111311508 shutdown_connections 2024-04-20T10:20:39.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.363+0000 7fe327e00640 1 --2- 172.21.15.123:0/2111311508 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe3281f4490 0x7fe3280cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:39.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.363+0000 7fe327e00640 1 --2- 172.21.15.123:0/2111311508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe32815ba30 0x7fe328152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:39.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.363+0000 7fe327e00640 1 --2- 172.21.15.123:0/2111311508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe30c004800 0x7fe32806b900 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:39.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.363+0000 7fe327e00640 1 -- 172.21.15.123:0/2111311508 >> 172.21.15.123:0/2111311508 conn(0x7fe32809d6f0 msgr2=0x7fe3280b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:20:39.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.364+0000 7fe327e00640 1 -- 172.21.15.123:0/2111311508 shutdown_connections 2024-04-20T10:20:39.364 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.364+0000 7fe327e00640 1 -- 172.21.15.123:0/2111311508 wait complete. 2024-04-20T10:20:39.380 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new f2570399-aa50-4ff3-bf5d-3d09207290db 1 2024-04-20T10:20:39.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.545+0000 7f3b76000640 1 Processor -- start 2024-04-20T10:20:39.545 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.545+0000 7f3b76000640 1 -- start start 2024-04-20T10:20:39.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.546+0000 7f3b76000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3b700d4270 0x7f3b700d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:39.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.546+0000 7f3b76000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b700d0380 0x7f3b700d0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:39.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.546+0000 7f3b76000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b700d0d50 con 0x7f3b700d4650 2024-04-20T10:20:39.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.546+0000 7f3b76000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f3b700d40c0 con 0x7f3b700d4270 2024-04-20T10:20:39.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.546+0000 7f3b76000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f3b700d3820 con 0x7f3b700d0380 2024-04-20T10:20:39.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.547+0000 7f3b6fe00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b700d0380 0x7f3b700d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:39.547 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.547+0000 7f3b6fe00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b700d0380 0x7f3b700d0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.123:54368/0 (socket says 172.21.15.123:54368) 2024-04-20T10:20:39.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.547+0000 7f3b6fe00640 1 -- 172.21.15.123:0/1365282095 learned_addr learned my addr 172.21.15.123:0/1365282095 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-20T10:20:39.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.547+0000 7f3b6fe00640 1 -- 172.21.15.123:0/1365282095 >> v2:172.21.15.121:3300/0 conn(0x7f3b700d4270 msgr2=0x7f3b700d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:20:39.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.547+0000 7f3b6f400640 1 --2- 172.21.15.123:0/1365282095 >> v2:172.21.15.121:3300/0 conn(0x7f3b700d4270 0x7f3b700d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:39.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.547+0000 7f3b6fe00640 1 --2- 172.21.15.123:0/1365282095 >> v2:172.21.15.121:3300/0 conn(0x7f3b700d4270 0x7f3b700d1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:39.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.547+0000 7f3b6fe00640 1 -- 172.21.15.123:0/1365282095 >> v1:172.21.15.121:6789/0 conn(0x7f3b700d4650 legacy=0x7f3b700b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:20:39.548 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.547+0000 7f3b6fe00640 1 -- 172.21.15.123:0/1365282095 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b700d3b00 con 0x7f3b700d0380 2024-04-20T10:20:39.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.548+0000 7f3b6fe00640 1 --2- 172.21.15.123:0/1365282095 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b700d0380 0x7f3b700d0810 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3b68009870 tx=0x7f3b6802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=115632f59efeaee6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:39.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.548+0000 7f3b6e000640 1 -- 172.21.15.123:0/1365282095 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b68035020 con 0x7f3b700d0380 2024-04-20T10:20:39.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.548+0000 7f3b6e000640 1 -- 172.21.15.123:0/1365282095 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b68031640 con 0x7f3b700d0380 2024-04-20T10:20:39.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.548+0000 7f3b6e000640 1 -- 172.21.15.123:0/1365282095 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6802d040 con 0x7f3b700d0380 2024-04-20T10:20:39.549 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.549+0000 7f3b76000640 1 -- 172.21.15.123:0/1365282095 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b700d0380 msgr2=0x7f3b700d0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:39.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.549+0000 7f3b76000640 1 --2- 172.21.15.123:0/1365282095 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b700d0380 0x7f3b700d0810 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3b68009870 tx=0x7f3b6802b8f0 comp rx=0 tx=0).stop 2024-04-20T10:20:39.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.549+0000 7f3b76000640 1 -- 172.21.15.123:0/1365282095 shutdown_connections 2024-04-20T10:20:39.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.549+0000 7f3b76000640 1 --2- 172.21.15.123:0/1365282095 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b700d0380 0x7f3b700d0810 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:39.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.549+0000 7f3b76000640 1 --2- 172.21.15.123:0/1365282095 >> v2:172.21.15.121:3300/0 conn(0x7f3b700d4270 0x7f3b700d1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:39.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.549+0000 7f3b76000640 1 -- 172.21.15.123:0/1365282095 >> 172.21.15.123:0/1365282095 conn(0x7f3b700b99c0 msgr2=0x7f3b700b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:20:39.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.549+0000 7f3b76000640 1 -- 172.21.15.123:0/1365282095 shutdown_connections 2024-04-20T10:20:39.550 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.549+0000 7f3b76000640 1 -- 172.21.15.123:0/1365282095 wait complete. 2024-04-20T10:20:39.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.551+0000 7f3b76000640 1 Processor -- start 2024-04-20T10:20:39.551 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.551+0000 7f3b76000640 1 -- start start 2024-04-20T10:20:39.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.552+0000 7f3b76000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b700d0380 0x7f3b701f45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:39.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.552+0000 7f3b76000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b700d4270 0x7f3b70155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:39.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.552+0000 7f3b76000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b700d4650 0x7f3b700c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:39.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.552+0000 7f3b76000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3b700b19a0 con 0x7f3b700d4270 2024-04-20T10:20:39.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.552+0000 7f3b76000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3b700cfe60 con 0x7f3b700d0380 2024-04-20T10:20:39.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.552+0000 7f3b76000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f3b700d53b0 con 0x7f3b700d4650 2024-04-20T10:20:39.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.552+0000 7f3b6f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b700d0380 0x7f3b701f45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:39.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.552+0000 7f3b6fe00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b700d4650 0x7f3b700c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:39.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.552+0000 7f3b6f400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b700d0380 0x7f3b701f45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52472/0 (socket says 172.21.15.123:52472) 2024-04-20T10:20:39.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.552+0000 7f3b6f400640 1 -- 172.21.15.123:0/951651070 learned_addr learned my addr 172.21.15.123:0/951651070 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-20T10:20:39.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.552+0000 7f3b6ea00640 1 --2- 172.21.15.123:0/951651070 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b700d4270 0x7f3b70155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:39.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.552+0000 7f3b6f400640 1 -- 172.21.15.123:0/951651070 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b700d4650 msgr2=0x7f3b700c7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:39.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.552+0000 7f3b6f400640 1 --2- 172.21.15.123:0/951651070 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b700d4650 0x7f3b700c7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:39.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.552+0000 7f3b6f400640 1 -- 172.21.15.123:0/951651070 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b700d4270 msgr2=0x7f3b70155640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:39.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.552+0000 7f3b6f400640 1 --2- 172.21.15.123:0/951651070 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b700d4270 0x7f3b70155640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:39.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.552+0000 7f3b6f400640 1 -- 172.21.15.123:0/951651070 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b700cb210 con 0x7f3b700d0380 2024-04-20T10:20:39.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.552+0000 7f3b6fe00640 1 --2- 172.21.15.123:0/951651070 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b700d4650 0x7f3b700c7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:20:39.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.553+0000 7f3b6f400640 1 --2- 172.21.15.123:0/951651070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b700d0380 0x7f3b701f45d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3b6000c820 tx=0x7f3b6000cce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:39.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.553+0000 7f3b67400640 1 -- 172.21.15.123:0/951651070 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b60015070 con 0x7f3b700d0380 2024-04-20T10:20:39.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.553+0000 7f3b67400640 1 -- 172.21.15.123:0/951651070 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b60004240 con 0x7f3b700d0380 2024-04-20T10:20:39.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.553+0000 7f3b67400640 1 -- 172.21.15.123:0/951651070 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b6000ce10 con 0x7f3b700d0380 2024-04-20T10:20:39.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.553+0000 7f3b76000640 1 -- 172.21.15.123:0/951651070 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b7006d210 con 0x7f3b700d0380 2024-04-20T10:20:39.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.553+0000 7f3b76000640 1 -- 172.21.15.123:0/951651070 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b700c8360 con 0x7f3b700d0380 2024-04-20T10:20:39.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.553+0000 7f3b67400640 1 -- 172.21.15.123:0/951651070 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3b60007830 con 0x7f3b700d0380 2024-04-20T10:20:39.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.553+0000 7f3b67400640 1 -- 172.21.15.123:0/951651070 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f3b600197b0 con 0x7f3b700d0380 2024-04-20T10:20:39.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.554+0000 7f3b76000640 1 -- 172.21.15.123:0/951651070 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b70068980 con 0x7f3b700d0380 2024-04-20T10:20:39.558 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.557+0000 7f3b67400640 1 -- 172.21.15.123:0/951651070 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f3b60007d00 con 0x7f3b700d0380 2024-04-20T10:20:39.635 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.635+0000 7f3b76000640 1 -- 172.21.15.123:0/951651070 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f2570399-aa50-4ff3-bf5d-3d09207290db", "id": 1} v 0) -- 0x7f3b700b6b30 con 0x7f3b700d0380 2024-04-20T10:20:39.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.646+0000 7f3b67400640 1 -- 172.21.15.123:0/951651070 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f2570399-aa50-4ff3-bf5d-3d09207290db", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f3b60003610 con 0x7f3b700d0380 2024-04-20T10:20:39.646 INFO:teuthology.orchestra.run.smithi123.stdout:1 2024-04-20T10:20:39.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.647+0000 7f3b76000640 1 -- 172.21.15.123:0/951651070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b700d0380 msgr2=0x7f3b701f45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:39.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.647+0000 7f3b76000640 1 --2- 172.21.15.123:0/951651070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b700d0380 0x7f3b701f45d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3b6000c820 tx=0x7f3b6000cce0 comp rx=0 tx=0).stop 2024-04-20T10:20:39.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.647+0000 7f3b76000640 1 -- 172.21.15.123:0/951651070 shutdown_connections 2024-04-20T10:20:39.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.647+0000 7f3b76000640 1 --2- 172.21.15.123:0/951651070 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b700d4650 0x7f3b700c7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:39.647 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.647+0000 7f3b76000640 1 --2- 172.21.15.123:0/951651070 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b700d4270 0x7f3b70155640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:39.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.647+0000 7f3b76000640 1 --2- 172.21.15.123:0/951651070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b700d0380 0x7f3b701f45d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:39.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.647+0000 7f3b76000640 1 -- 172.21.15.123:0/951651070 >> 172.21.15.123:0/951651070 conn(0x7f3b700b99c0 msgr2=0x7f3b700b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:20:39.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.647+0000 7f3b76000640 1 -- 172.21.15.123:0/951651070 shutdown_connections 2024-04-20T10:20:39.648 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.647+0000 7f3b76000640 1 -- 172.21.15.123:0/951651070 wait complete. 2024-04-20T10:20:39.662 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new 50893807-dc30-49f4-b41f-739366626658 2 2024-04-20T10:20:39.829 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.829+0000 7ffb92a00640 1 Processor -- start 2024-04-20T10:20:39.830 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.829+0000 7ffb92a00640 1 -- start start 2024-04-20T10:20:39.833 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.831+0000 7ffb92a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ffb8c0d3b20 0x7ffb8c0b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:39.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.831+0000 7ffb92a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb8c0d0980 con 0x7ffb8c161a50 2024-04-20T10:20:39.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.831+0000 7ffb92a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb8c162490 con 0x7ffb8c06c010 2024-04-20T10:20:39.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.831+0000 7ffb92a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7ffb8c0d0c60 con 0x7ffb8c0d3b20 2024-04-20T10:20:39.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.831+0000 7ffb92000640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7ffb8c06c010 0x7ffb8c0ceaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56080/0 (socket says 172.21.15.123:56080) 2024-04-20T10:20:39.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.831+0000 7ffb92000640 1 -- 172.21.15.123:0/2013103244 learned_addr learned my addr 172.21.15.123:0/2013103244 (peer_addr_for_me v1:172.21.15.123:0/0) 2024-04-20T10:20:39.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.831+0000 7ffb90c00640 1 --2- 172.21.15.123:0/2013103244 >> v2:172.21.15.121:3300/0 conn(0x7ffb8c0d3b20 0x7ffb8c0b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:39.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.832+0000 7ffb8be00640 1 -- 172.21.15.123:0/2013103244 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3497160076 0 0) 0x7ffb8c162490 con 0x7ffb8c06c010 2024-04-20T10:20:39.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.832+0000 7ffb8be00640 1 -- 172.21.15.123:0/2013103244 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb74003520 con 0x7ffb8c06c010 2024-04-20T10:20:39.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.832+0000 7ffb8be00640 1 -- 172.21.15.123:0/2013103244 <== mon.1 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1351834573 0 0) 0x7ffb74003520 con 0x7ffb8c06c010 2024-04-20T10:20:39.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.832+0000 7ffb8be00640 1 -- 172.21.15.123:0/2013103244 >> v2:172.21.15.121:3300/0 conn(0x7ffb8c0d3b20 msgr2=0x7ffb8c0b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:20:39.834 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.832+0000 7ffb8be00640 1 --2- 172.21.15.123:0/2013103244 >> v2:172.21.15.121:3300/0 conn(0x7ffb8c0d3b20 0x7ffb8c0b3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:39.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.832+0000 7ffb8be00640 1 -- 172.21.15.123:0/2013103244 >> v1:172.21.15.121:6789/0 conn(0x7ffb8c161a50 legacy=0x7ffb8c06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:39.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.832+0000 7ffb8be00640 1 -- 172.21.15.123:0/2013103244 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb8c162770 con 0x7ffb8c06c010 2024-04-20T10:20:39.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.832+0000 7ffb8be00640 1 -- 172.21.15.123:0/2013103244 <== mon.1 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7ffb840019f0 con 0x7ffb8c06c010 2024-04-20T10:20:39.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.832+0000 7ffb8be00640 1 -- 172.21.15.123:0/2013103244 >> v1:172.21.15.123:6789/0 conn(0x7ffb8c06c010 legacy=0x7ffb8c0ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:39.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.832+0000 7ffb8be00640 1 --2- 172.21.15.123:0/2013103244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb740039b0 0x7ffb74003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:39.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.832+0000 7ffb8be00640 1 --2- 172.21.15.123:0/2013103244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb74004800 0x7ffb74004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:39.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.832+0000 7ffb8be00640 1 --2- 172.21.15.123:0/2013103244 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb740051c0 0x7ffb7400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:39.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.832+0000 7ffb8be00640 1 -- 172.21.15.123:0/2013103244 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c0d0c60 con 0x7ffb740039b0 2024-04-20T10:20:39.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.832+0000 7ffb8be00640 1 -- 172.21.15.123:0/2013103244 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7400c090 con 0x7ffb74004800 2024-04-20T10:20:39.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.832+0000 7ffb8be00640 1 -- 172.21.15.123:0/2013103244 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ffb7400c290 con 0x7ffb740051c0 2024-04-20T10:20:39.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.833+0000 7ffb90c00640 1 --2- 172.21.15.123:0/2013103244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb74004800 0x7ffb74004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:39.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.833+0000 7ffb92000640 1 --2- 172.21.15.123:0/2013103244 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb740051c0 0x7ffb7400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:39.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.833+0000 7ffb91600640 1 --2- 172.21.15.123:0/2013103244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb740039b0 0x7ffb74003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:39.835 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.833+0000 7ffb90c00640 1 -- 172.21.15.123:0/2013103244 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb740051c0 msgr2=0x7ffb7400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:20:39.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.833+0000 7ffb90c00640 1 --2- 172.21.15.123:0/2013103244 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb740051c0 0x7ffb7400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:39.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.833+0000 7ffb90c00640 1 -- 172.21.15.123:0/2013103244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb740039b0 msgr2=0x7ffb74003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:20:39.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.833+0000 7ffb90c00640 1 --2- 172.21.15.123:0/2013103244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb740039b0 0x7ffb74003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:39.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.833+0000 7ffb90c00640 1 -- 172.21.15.123:0/2013103244 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb74003520 con 0x7ffb74004800 2024-04-20T10:20:39.836 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.833+0000 7ffb92000640 1 -- 172.21.15.123:0/2013103244 reap_dead start 2024-04-20T10:20:39.839 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.834+0000 7ffb90c00640 1 --2- 172.21.15.123:0/2013103244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb74004800 0x7ffb74004c80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ffb78009ac0 tx=0x7ffb74005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1ec99e00124de72 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:39.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.834+0000 7ffb8be00640 1 -- 172.21.15.123:0/2013103244 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb78039070 con 0x7ffb74004800 2024-04-20T10:20:39.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.834+0000 7ffb8be00640 1 -- 172.21.15.123:0/2013103244 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb7802c680 con 0x7ffb74004800 2024-04-20T10:20:39.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.834+0000 7ffb8be00640 1 -- 172.21.15.123:0/2013103244 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb78009690 con 0x7ffb74004800 2024-04-20T10:20:39.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.835+0000 7ffb92a00640 1 -- 172.21.15.123:0/2013103244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb74004800 msgr2=0x7ffb74004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:39.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.835+0000 7ffb92a00640 1 --2- 172.21.15.123:0/2013103244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb74004800 0x7ffb74004c80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ffb78009ac0 tx=0x7ffb74005830 comp rx=0 tx=0).stop 2024-04-20T10:20:39.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.835+0000 7ffb92a00640 1 -- 172.21.15.123:0/2013103244 shutdown_connections 2024-04-20T10:20:39.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.835+0000 7ffb92a00640 1 --2- 172.21.15.123:0/2013103244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb74004800 0x7ffb74004c80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:39.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.835+0000 7ffb92a00640 1 -- 172.21.15.123:0/2013103244 >> 172.21.15.123:0/2013103244 conn(0x7ffb8c0d0ed0 msgr2=0x7ffb8c0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:20:39.840 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.835+0000 7ffb92a00640 1 -- 172.21.15.123:0/2013103244 shutdown_connections 2024-04-20T10:20:39.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.835+0000 7ffb92a00640 1 -- 172.21.15.123:0/2013103244 wait complete. 2024-04-20T10:20:39.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.837+0000 7ffb92a00640 1 Processor -- start 2024-04-20T10:20:39.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.837+0000 7ffb92a00640 1 -- start start 2024-04-20T10:20:39.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.838+0000 7ffb92a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb74004800 0x7ffb8c0ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:39.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.838+0000 7ffb92a00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb8c0ce370 0x7ffb8c0ce7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:39.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.838+0000 7ffb92a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb8c0aad90 0x7ffb8c0ab210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:39.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.838+0000 7ffb92a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c162ae0 con 0x7ffb74004800 2024-04-20T10:20:39.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.838+0000 7ffb92a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c0d0690 con 0x7ffb8c0aad90 2024-04-20T10:20:39.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.838+0000 7ffb92a00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ffb8c161e30 con 0x7ffb8c0ce370 2024-04-20T10:20:39.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.838+0000 7ffb92000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb8c0aad90 0x7ffb8c0ab210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:39.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.838+0000 7ffb90c00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb8c0ce370 0x7ffb8c0ce7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:39.841 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.838+0000 7ffb92000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb8c0aad90 0x7ffb8c0ab210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52480/0 (socket says 172.21.15.123:52480) 2024-04-20T10:20:39.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.838+0000 7ffb92000640 1 -- 172.21.15.123:0/4258090816 learned_addr learned my addr 172.21.15.123:0/4258090816 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-20T10:20:39.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.838+0000 7ffb91600640 1 --2- 172.21.15.123:0/4258090816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb74004800 0x7ffb8c0ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:39.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.839+0000 7ffb90c00640 1 -- 172.21.15.123:0/4258090816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb8c0aad90 msgr2=0x7ffb8c0ab210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:39.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.839+0000 7ffb90c00640 1 --2- 172.21.15.123:0/4258090816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb8c0aad90 0x7ffb8c0ab210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:39.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.839+0000 7ffb90c00640 1 -- 172.21.15.123:0/4258090816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb74004800 msgr2=0x7ffb8c0ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:39.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.839+0000 7ffb90c00640 1 --2- 172.21.15.123:0/4258090816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb74004800 0x7ffb8c0ca0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:39.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.839+0000 7ffb90c00640 1 -- 172.21.15.123:0/4258090816 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb8c0aba30 con 0x7ffb8c0ce370 2024-04-20T10:20:39.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.839+0000 7ffb90c00640 1 --2- 172.21.15.123:0/4258090816 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb8c0ce370 0x7ffb8c0ce7f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ffb78033000 tx=0x7ffb78004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:39.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.839+0000 7ffb89600640 1 -- 172.21.15.123:0/4258090816 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb78039050 con 0x7ffb8c0ce370 2024-04-20T10:20:39.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.839+0000 7ffb89600640 1 -- 172.21.15.123:0/4258090816 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb7803a040 con 0x7ffb8c0ce370 2024-04-20T10:20:39.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.839+0000 7ffb92a00640 1 -- 172.21.15.123:0/4258090816 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb8c0abd10 con 0x7ffb8c0ce370 2024-04-20T10:20:39.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.839+0000 7ffb89600640 1 -- 172.21.15.123:0/4258090816 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb78003610 con 0x7ffb8c0ce370 2024-04-20T10:20:39.842 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.840+0000 7ffb92a00640 1 -- 172.21.15.123:0/4258090816 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb8c15ab90 con 0x7ffb8c0ce370 2024-04-20T10:20:39.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.840+0000 7ffb89600640 1 -- 172.21.15.123:0/4258090816 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ffb78047020 con 0x7ffb8c0ce370 2024-04-20T10:20:39.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.840+0000 7ffb89600640 1 -- 172.21.15.123:0/4258090816 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7ffb780037b0 con 0x7ffb8c0ce370 2024-04-20T10:20:39.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.841+0000 7ffb92a00640 1 -- 172.21.15.123:0/4258090816 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb8c075b30 con 0x7ffb8c0ce370 2024-04-20T10:20:39.843 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.843+0000 7ffb89600640 1 -- 172.21.15.123:0/4258090816 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7ffb78003c10 con 0x7ffb8c0ce370 2024-04-20T10:20:39.848 INFO:tasks.ceph.mgr.y.smithi121.stderr:2024-04-20T10:20:39.848+0000 7f985f644040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T10:20:39.923 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.923+0000 7ffb92a00640 1 -- 172.21.15.123:0/4258090816 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd new", "uuid": "50893807-dc30-49f4-b41f-739366626658", "id": 2} v 0) -- 0x7ffb8c0d3f10 con 0x7ffb8c0ce370 2024-04-20T10:20:39.929 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.929+0000 7ffb89600640 1 -- 172.21.15.123:0/4258090816 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "50893807-dc30-49f4-b41f-739366626658", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7ffb78005990 con 0x7ffb8c0ce370 2024-04-20T10:20:39.930 INFO:teuthology.orchestra.run.smithi123.stdout:2 2024-04-20T10:20:39.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.931+0000 7ffb92a00640 1 -- 172.21.15.123:0/4258090816 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb8c0ce370 msgr2=0x7ffb8c0ce7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:39.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.931+0000 7ffb92a00640 1 --2- 172.21.15.123:0/4258090816 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb8c0ce370 0x7ffb8c0ce7f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ffb78033000 tx=0x7ffb78004730 comp rx=0 tx=0).stop 2024-04-20T10:20:39.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.931+0000 7ffb92a00640 1 -- 172.21.15.123:0/4258090816 shutdown_connections 2024-04-20T10:20:39.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.931+0000 7ffb92a00640 1 --2- 172.21.15.123:0/4258090816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb8c0aad90 0x7ffb8c0ab210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:39.931 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.931+0000 7ffb92a00640 1 --2- 172.21.15.123:0/4258090816 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffb8c0ce370 0x7ffb8c0ce7f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:39.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.931+0000 7ffb92a00640 1 --2- 172.21.15.123:0/4258090816 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffb74004800 0x7ffb8c0ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:39.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.931+0000 7ffb92a00640 1 -- 172.21.15.123:0/4258090816 >> 172.21.15.123:0/4258090816 conn(0x7ffb8c0d0ed0 msgr2=0x7ffb8c0cb520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:20:39.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.931+0000 7ffb92a00640 1 -- 172.21.15.123:0/4258090816 shutdown_connections 2024-04-20T10:20:39.932 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:39.931+0000 7ffb92a00640 1 -- 172.21.15.123:0/4258090816 wait complete. 2024-04-20T10:20:39.949 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new dadb01bc-5491-4ffc-8824-d48a057240b5 3 2024-04-20T10:20:39.952 INFO:tasks.ceph.mgr.y.smithi121.stderr:2024-04-20T10:20:39.953+0000 7f985f644040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T10:20:39.963 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-04-20T10:20:39.963+0000 7f0707812040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T10:20:40.002 INFO:tasks.ceph.mgr.y.smithi121.stderr:2024-04-20T10:20:40.003+0000 7f985f644040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T10:20:40.077 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-04-20T10:20:40.077+0000 7f0707812040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T10:20:40.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.128+0000 7f7100800640 1 Processor -- start 2024-04-20T10:20:40.128 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.128+0000 7f7100800640 1 -- start start 2024-04-20T10:20:40.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.130+0000 7f7100800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f70f80d4270 0x7f70f80d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:40.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.130+0000 7f7100800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70f80d4650 0x7f70f80b1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:40.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.130+0000 7f7100800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f70f80b17d0 0x7f70f80d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:40.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.130+0000 7f7100800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f70f80d40c0 con 0x7f70f80d4270 2024-04-20T10:20:40.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.130+0000 7f7100800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f70f80d3820 con 0x7f70f80b17d0 2024-04-20T10:20:40.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.130+0000 7f7100800640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f70f80d3b00 con 0x7f70f80d4650 2024-04-20T10:20:40.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.130+0000 7f70fd400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70f80d4650 0x7f70f80b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:40.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.130+0000 7f70fd400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70f80d4650 0x7f70f80b1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.123:48862/0 (socket says 172.21.15.123:48862) 2024-04-20T10:20:40.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.130+0000 7f70fd400640 1 -- 172.21.15.123:0/1516063097 learned_addr learned my addr 172.21.15.123:0/1516063097 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-20T10:20:40.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.130+0000 7f70fe800640 1 --2- 172.21.15.123:0/1516063097 >> v2:172.21.15.123:3300/0 conn(0x7f70f80b17d0 0x7f70f80d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:40.131 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.130+0000 7f70fde00640 1 --2- 172.21.15.123:0/1516063097 >> v2:172.21.15.121:3300/0 conn(0x7f70f80d4270 0x7f70f80d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:40.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.131+0000 7f70fd400640 1 -- 172.21.15.123:0/1516063097 >> v2:172.21.15.123:3300/0 conn(0x7f70f80b17d0 msgr2=0x7f70f80d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:20:40.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.131+0000 7f70fd400640 1 --2- 172.21.15.123:0/1516063097 >> v2:172.21.15.123:3300/0 conn(0x7f70f80b17d0 0x7f70f80d0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:40.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.131+0000 7f70fd400640 1 -- 172.21.15.123:0/1516063097 >> v2:172.21.15.121:3300/0 conn(0x7f70f80d4270 msgr2=0x7f70f80d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:20:40.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.131+0000 7f70fd400640 1 --2- 172.21.15.123:0/1516063097 >> v2:172.21.15.121:3300/0 conn(0x7f70f80d4270 0x7f70f80d1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:40.132 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.131+0000 7f70fd400640 1 -- 172.21.15.123:0/1516063097 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70f80d3de0 con 0x7f70f80d4650 2024-04-20T10:20:40.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.132+0000 7f70fd400640 1 --2- 172.21.15.123:0/1516063097 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70f80d4650 0x7f70f80b1290 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f70f808edb0 tx=0x7f70e802b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7511c6716fb131a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:40.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.132+0000 7f70fca00640 1 -- 172.21.15.123:0/1516063097 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e8035020 con 0x7f70f80d4650 2024-04-20T10:20:40.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.132+0000 7f70fca00640 1 -- 172.21.15.123:0/1516063097 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70e8031870 con 0x7f70f80d4650 2024-04-20T10:20:40.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.132+0000 7f70fca00640 1 -- 172.21.15.123:0/1516063097 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70e8031c40 con 0x7f70f80d4650 2024-04-20T10:20:40.133 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.133+0000 7f7100800640 1 -- 172.21.15.123:0/1516063097 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70f80d4650 msgr2=0x7f70f80b1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:40.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.133+0000 7f7100800640 1 --2- 172.21.15.123:0/1516063097 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70f80d4650 0x7f70f80b1290 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f70f808edb0 tx=0x7f70e802b910 comp rx=0 tx=0).stop 2024-04-20T10:20:40.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.133+0000 7f7100800640 1 -- 172.21.15.123:0/1516063097 shutdown_connections 2024-04-20T10:20:40.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.133+0000 7f7100800640 1 --2- 172.21.15.123:0/1516063097 >> v2:172.21.15.123:3300/0 conn(0x7f70f80b17d0 0x7f70f80d0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:40.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.133+0000 7f7100800640 1 --2- 172.21.15.123:0/1516063097 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70f80d4650 0x7f70f80b1290 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:40.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.133+0000 7f7100800640 1 --2- 172.21.15.123:0/1516063097 >> v2:172.21.15.121:3300/0 conn(0x7f70f80d4270 0x7f70f80d1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:40.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.133+0000 7f7100800640 1 -- 172.21.15.123:0/1516063097 >> 172.21.15.123:0/1516063097 conn(0x7f70f80b99c0 msgr2=0x7f70f80b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:20:40.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.133+0000 7f7100800640 1 -- 172.21.15.123:0/1516063097 shutdown_connections 2024-04-20T10:20:40.134 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.133+0000 7f7100800640 1 -- 172.21.15.123:0/1516063097 wait complete. 2024-04-20T10:20:40.134 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-04-20T10:20:40.133+0000 7f0707812040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T10:20:40.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.135+0000 7f7100800640 1 Processor -- start 2024-04-20T10:20:40.135 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.135+0000 7f7100800640 1 -- start start 2024-04-20T10:20:40.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.136+0000 7f7100800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70f80b17d0 0x7f70f8150810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:40.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.136+0000 7f7100800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70f80d4270 0x7f70f815f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:40.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.136+0000 7f7100800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70f80d4650 0x7f70f815f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:40.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.136+0000 7f7100800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f70f8161f70 con 0x7f70f80d4650 2024-04-20T10:20:40.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.136+0000 7f7100800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f70f80d53b0 con 0x7f70f80b17d0 2024-04-20T10:20:40.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.136+0000 7f7100800640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f70f81620f0 con 0x7f70f80d4270 2024-04-20T10:20:40.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.136+0000 7f70fde00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70f80b17d0 0x7f70f8150810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:40.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.136+0000 7f70fde00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70f80b17d0 0x7f70f8150810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50976/0 (socket says 172.21.15.123:50976) 2024-04-20T10:20:40.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.136+0000 7f70fde00640 1 -- 172.21.15.123:0/3457167243 learned_addr learned my addr 172.21.15.123:0/3457167243 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-20T10:20:40.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.136+0000 7f70fd400640 1 --2- 172.21.15.123:0/3457167243 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70f80d4270 0x7f70f815f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:40.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.136+0000 7f70fe800640 1 --2- 172.21.15.123:0/3457167243 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70f80d4650 0x7f70f815f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:40.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.136+0000 7f70fde00640 1 -- 172.21.15.123:0/3457167243 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70f80d4270 msgr2=0x7f70f815f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:40.137 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.136+0000 7f70fde00640 1 --2- 172.21.15.123:0/3457167243 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70f80d4270 0x7f70f815f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:40.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.137+0000 7f70fde00640 1 -- 172.21.15.123:0/3457167243 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70f80d4650 msgr2=0x7f70f815f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:40.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.137+0000 7f70fde00640 1 --2- 172.21.15.123:0/3457167243 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70f80d4650 0x7f70f815f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:40.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.137+0000 7f70fde00640 1 -- 172.21.15.123:0/3457167243 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70f80ce4b0 con 0x7f70f80b17d0 2024-04-20T10:20:40.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.137+0000 7f70fd400640 1 --2- 172.21.15.123:0/3457167243 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70f80d4270 0x7f70f815f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:20:40.138 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.137+0000 7f70fde00640 1 --2- 172.21.15.123:0/3457167243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70f80b17d0 0x7f70f8150810 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f70ec00d830 tx=0x7f70ec00dcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:40.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.137+0000 7f70f2000640 1 -- 172.21.15.123:0/3457167243 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ec00b6c0 con 0x7f70f80b17d0 2024-04-20T10:20:40.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.137+0000 7f70f2000640 1 -- 172.21.15.123:0/3457167243 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f70ec019070 con 0x7f70f80b17d0 2024-04-20T10:20:40.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.137+0000 7f70f2000640 1 -- 172.21.15.123:0/3457167243 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70ec00ba80 con 0x7f70f80b17d0 2024-04-20T10:20:40.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.137+0000 7f7100800640 1 -- 172.21.15.123:0/3457167243 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70f80cc7a0 con 0x7f70f80b17d0 2024-04-20T10:20:40.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.137+0000 7f7100800640 1 -- 172.21.15.123:0/3457167243 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70f80ccd60 con 0x7f70f80b17d0 2024-04-20T10:20:40.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.138+0000 7f70f2000640 1 -- 172.21.15.123:0/3457167243 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f70ec01e460 con 0x7f70f80b17d0 2024-04-20T10:20:40.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.138+0000 7f70f2000640 1 -- 172.21.15.123:0/3457167243 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f70ec014b40 con 0x7f70f80b17d0 2024-04-20T10:20:40.139 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.138+0000 7f7100800640 1 -- 172.21.15.123:0/3457167243 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70f8068980 con 0x7f70f80b17d0 2024-04-20T10:20:40.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.141+0000 7f70f2000640 1 -- 172.21.15.123:0/3457167243 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f70ec014d40 con 0x7f70f80b17d0 2024-04-20T10:20:40.194 INFO:tasks.ceph.mgr.y.smithi121.stderr:2024-04-20T10:20:40.194+0000 7f985f644040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T10:20:40.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.216+0000 7f7100800640 1 -- 172.21.15.123:0/3457167243 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd new", "uuid": "dadb01bc-5491-4ffc-8824-d48a057240b5", "id": 3} v 0) -- 0x7f70f80c97d0 con 0x7f70f80b17d0 2024-04-20T10:20:40.224 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.224+0000 7f70f2000640 1 -- 172.21.15.123:0/3457167243 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dadb01bc-5491-4ffc-8824-d48a057240b5", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f70ec0146b0 con 0x7f70f80b17d0 2024-04-20T10:20:40.224 INFO:teuthology.orchestra.run.smithi123.stdout:3 2024-04-20T10:20:40.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.225+0000 7f7100800640 1 -- 172.21.15.123:0/3457167243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70f80b17d0 msgr2=0x7f70f8150810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:40.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.225+0000 7f7100800640 1 --2- 172.21.15.123:0/3457167243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70f80b17d0 0x7f70f8150810 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f70ec00d830 tx=0x7f70ec00dcf0 comp rx=0 tx=0).stop 2024-04-20T10:20:40.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.226+0000 7f7100800640 1 -- 172.21.15.123:0/3457167243 shutdown_connections 2024-04-20T10:20:40.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.226+0000 7f7100800640 1 --2- 172.21.15.123:0/3457167243 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70f80d4650 0x7f70f815f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:40.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.226+0000 7f7100800640 1 --2- 172.21.15.123:0/3457167243 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70f80d4270 0x7f70f815f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:40.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.226+0000 7f7100800640 1 --2- 172.21.15.123:0/3457167243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70f80b17d0 0x7f70f8150810 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:40.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.226+0000 7f7100800640 1 -- 172.21.15.123:0/3457167243 >> 172.21.15.123:0/3457167243 conn(0x7f70f80b99c0 msgr2=0x7f70f8160570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:20:40.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.226+0000 7f7100800640 1 -- 172.21.15.123:0/3457167243 shutdown_connections 2024-04-20T10:20:40.227 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.226+0000 7f7100800640 1 -- 172.21.15.123:0/3457167243 wait complete. 2024-04-20T10:20:40.242 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new b3f3b4a1-b375-4526-b69e-344ab24c2da1 4 2024-04-20T10:20:40.259 INFO:tasks.ceph.mgr.y.smithi121.stderr:2024-04-20T10:20:40.259+0000 7f985f644040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T10:20:40.319 INFO:tasks.ceph.mgr.y.smithi121.stderr:2024-04-20T10:20:40.320+0000 7f985f644040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T10:20:40.340 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-04-20T10:20:40.340+0000 7f0707812040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T10:20:40.422 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-04-20T10:20:40.422+0000 7f0707812040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T10:20:40.447 INFO:tasks.ceph.mgr.y.smithi121.stderr:2024-04-20T10:20:40.447+0000 7f985f644040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T10:20:40.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.462+0000 7f4a99600640 1 Processor -- start 2024-04-20T10:20:40.463 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.463+0000 7f4a99600640 1 -- start start 2024-04-20T10:20:40.465 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.464+0000 7f4a99600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4a94161a50 0x7f4a940b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:40.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.464+0000 7f4a99600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4a940d3b20 0x7f4a9406b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:40.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.464+0000 7f4a99600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a940d0980 con 0x7f4a940d0480 2024-04-20T10:20:40.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.464+0000 7f4a99600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f4a940d0c60 con 0x7f4a94161a50 2024-04-20T10:20:40.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.464+0000 7f4a99600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f4a94162ae0 con 0x7f4a940d3b20 2024-04-20T10:20:40.466 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.464+0000 7f4a93400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4a940d3b20 0x7f4a9406b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:40.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.464+0000 7f4a93400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4a940d3b20 0x7f4a9406b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50980/0 (socket says 172.21.15.123:50980) 2024-04-20T10:20:40.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.464+0000 7f4a98c00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f4a940d0480 0x7f4a9406c630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:45962/0 (socket says 172.21.15.123:45962) 2024-04-20T10:20:40.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.465+0000 7f4a93400640 1 -- 172.21.15.123:0/3194790031 learned_addr learned my addr 172.21.15.123:0/3194790031 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-20T10:20:40.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.465+0000 7f4a93e00640 1 --2- 172.21.15.123:0/3194790031 >> v2:172.21.15.121:3300/0 conn(0x7f4a94161a50 0x7f4a940b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:40.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.465+0000 7f4a93400640 1 -- 172.21.15.123:0/3194790031 >> v2:172.21.15.121:3300/0 conn(0x7f4a94161a50 msgr2=0x7f4a940b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:20:40.467 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.465+0000 7f4a93400640 1 --2- 172.21.15.123:0/3194790031 >> v2:172.21.15.121:3300/0 conn(0x7f4a94161a50 0x7f4a940b3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:40.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.465+0000 7f4a93400640 1 -- 172.21.15.123:0/3194790031 >> v1:172.21.15.123:6789/0 conn(0x7f4a940d0480 legacy=0x7f4a9406c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:40.468 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.465+0000 7f4a93400640 1 -- 172.21.15.123:0/3194790031 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a94162490 con 0x7f4a940d3b20 2024-04-20T10:20:40.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.466+0000 7f4a93400640 1 --2- 172.21.15.123:0/3194790031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a940d3b20 0x7f4a9406b900 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4a940b7590 tx=0x7f4a7c02b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7ba38fd27019f8d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:40.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.466+0000 7f4a92a00640 1 -- 172.21.15.123:0/3194790031 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c034070 con 0x7f4a940d3b20 2024-04-20T10:20:40.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.466+0000 7f4a92a00640 1 -- 172.21.15.123:0/3194790031 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a7c030860 con 0x7f4a940d3b20 2024-04-20T10:20:40.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.466+0000 7f4a92a00640 1 -- 172.21.15.123:0/3194790031 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c030c30 con 0x7f4a940d3b20 2024-04-20T10:20:40.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.467+0000 7f4a99600640 1 -- 172.21.15.123:0/3194790031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a940d3b20 msgr2=0x7f4a9406b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:40.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.467+0000 7f4a99600640 1 --2- 172.21.15.123:0/3194790031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a940d3b20 0x7f4a9406b900 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4a940b7590 tx=0x7f4a7c02b9a0 comp rx=0 tx=0).stop 2024-04-20T10:20:40.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.467+0000 7f4a99600640 1 -- 172.21.15.123:0/3194790031 shutdown_connections 2024-04-20T10:20:40.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.467+0000 7f4a99600640 1 --2- 172.21.15.123:0/3194790031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a940d3b20 0x7f4a9406b900 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:40.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.467+0000 7f4a99600640 1 --2- 172.21.15.123:0/3194790031 >> v2:172.21.15.121:3300/0 conn(0x7f4a94161a50 0x7f4a940b3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:40.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.467+0000 7f4a99600640 1 -- 172.21.15.123:0/3194790031 >> 172.21.15.123:0/3194790031 conn(0x7f4a940d0ed0 msgr2=0x7f4a940d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:20:40.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.467+0000 7f4a99600640 1 -- 172.21.15.123:0/3194790031 shutdown_connections 2024-04-20T10:20:40.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.467+0000 7f4a99600640 1 -- 172.21.15.123:0/3194790031 wait complete. 2024-04-20T10:20:40.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.469+0000 7f4a99600640 1 Processor -- start 2024-04-20T10:20:40.470 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.469+0000 7f4a99600640 1 -- start start 2024-04-20T10:20:40.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.470+0000 7f4a99600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a940d0480 0x7f4a94155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:40.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.470+0000 7f4a99600640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a94161a50 0x7f4a940aad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:40.472 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.470+0000 7f4a99600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a940cd6f0 0x7f4a940cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:40.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.470+0000 7f4a99600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4a94161fb0 con 0x7f4a940cd6f0 2024-04-20T10:20:40.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.470+0000 7f4a99600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4a94161e30 con 0x7f4a940d0480 2024-04-20T10:20:40.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.470+0000 7f4a99600640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f4a94162130 con 0x7f4a94161a50 2024-04-20T10:20:40.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.470+0000 7f4a93e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a940d0480 0x7f4a94155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:40.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.470+0000 7f4a93e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a940d0480 0x7f4a94155b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50984/0 (socket says 172.21.15.123:50984) 2024-04-20T10:20:40.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.470+0000 7f4a93e00640 1 -- 172.21.15.123:0/971562898 learned_addr learned my addr 172.21.15.123:0/971562898 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-20T10:20:40.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.470+0000 7f4a93400640 1 --2- 172.21.15.123:0/971562898 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a94161a50 0x7f4a940aad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:40.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.470+0000 7f4a98c00640 1 --2- 172.21.15.123:0/971562898 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a940cd6f0 0x7f4a940cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:40.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.470+0000 7f4a93400640 1 -- 172.21.15.123:0/971562898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a940d0480 msgr2=0x7f4a94155b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:40.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.470+0000 7f4a93400640 1 --2- 172.21.15.123:0/971562898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a940d0480 0x7f4a94155b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:40.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.470+0000 7f4a93400640 1 -- 172.21.15.123:0/971562898 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a940cd6f0 msgr2=0x7f4a940cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:40.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.470+0000 7f4a93400640 1 --2- 172.21.15.123:0/971562898 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a940cd6f0 0x7f4a940cdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:40.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.471+0000 7f4a93400640 1 -- 172.21.15.123:0/971562898 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a940b2d30 con 0x7f4a94161a50 2024-04-20T10:20:40.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.471+0000 7f4a93e00640 1 --2- 172.21.15.123:0/971562898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a940d0480 0x7f4a94155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:20:40.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.471+0000 7f4a93400640 1 --2- 172.21.15.123:0/971562898 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a94161a50 0x7f4a940aad90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4a940d0de0 tx=0x7f4a7c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:40.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.471+0000 7f4a8be00640 1 -- 172.21.15.123:0/971562898 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c048070 con 0x7f4a94161a50 2024-04-20T10:20:40.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.471+0000 7f4a8be00640 1 -- 172.21.15.123:0/971562898 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a7c030660 con 0x7f4a94161a50 2024-04-20T10:20:40.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.471+0000 7f4a8be00640 1 -- 172.21.15.123:0/971562898 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c034040 con 0x7f4a94161a50 2024-04-20T10:20:40.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.471+0000 7f4a99600640 1 -- 172.21.15.123:0/971562898 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a940b2770 con 0x7f4a94161a50 2024-04-20T10:20:40.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.471+0000 7f4a99600640 1 -- 172.21.15.123:0/971562898 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a9415aef0 con 0x7f4a94161a50 2024-04-20T10:20:40.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.472+0000 7f4a8be00640 1 -- 172.21.15.123:0/971562898 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4a7c04b070 con 0x7f4a94161a50 2024-04-20T10:20:40.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.472+0000 7f4a8be00640 1 -- 172.21.15.123:0/971562898 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f4a7c04d950 con 0x7f4a94161a50 2024-04-20T10:20:40.474 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.472+0000 7f4a99600640 1 -- 172.21.15.123:0/971562898 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a94075b30 con 0x7f4a94161a50 2024-04-20T10:20:40.477 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.477+0000 7f4a8be00640 1 -- 172.21.15.123:0/971562898 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f4a940b2d30 con 0x7f4a94161a50 2024-04-20T10:20:40.489 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-04-20T10:20:40.489+0000 7f0707812040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T10:20:40.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.553+0000 7f4a99600640 1 -- 172.21.15.123:0/971562898 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b3f3b4a1-b375-4526-b69e-344ab24c2da1", "id": 4} v 0) -- 0x7f4a940d25b0 con 0x7f4a94161a50 2024-04-20T10:20:40.560 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.560+0000 7f4a8be00640 1 -- 172.21.15.123:0/971562898 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b3f3b4a1-b375-4526-b69e-344ab24c2da1", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f4a7c04dbe0 con 0x7f4a94161a50 2024-04-20T10:20:40.560 INFO:teuthology.orchestra.run.smithi123.stdout:4 2024-04-20T10:20:40.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.561+0000 7f4a99600640 1 -- 172.21.15.123:0/971562898 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a94161a50 msgr2=0x7f4a940aad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:40.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.561+0000 7f4a99600640 1 --2- 172.21.15.123:0/971562898 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a94161a50 0x7f4a940aad90 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4a940d0de0 tx=0x7f4a7c004730 comp rx=0 tx=0).stop 2024-04-20T10:20:40.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.561+0000 7f4a99600640 1 -- 172.21.15.123:0/971562898 shutdown_connections 2024-04-20T10:20:40.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.561+0000 7f4a99600640 1 --2- 172.21.15.123:0/971562898 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a940cd6f0 0x7f4a940cdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:40.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.561+0000 7f4a99600640 1 --2- 172.21.15.123:0/971562898 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a94161a50 0x7f4a940aad90 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:40.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.561+0000 7f4a99600640 1 --2- 172.21.15.123:0/971562898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a940d0480 0x7f4a94155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:40.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.561+0000 7f4a99600640 1 -- 172.21.15.123:0/971562898 >> 172.21.15.123:0/971562898 conn(0x7f4a940d0ed0 msgr2=0x7f4a940b3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:20:40.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.561+0000 7f4a99600640 1 -- 172.21.15.123:0/971562898 shutdown_connections 2024-04-20T10:20:40.562 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.561+0000 7f4a99600640 1 -- 172.21.15.123:0/971562898 wait complete. 2024-04-20T10:20:40.579 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new 2ee24a08-1624-4961-9642-d7b510282484 5 2024-04-20T10:20:40.624 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-04-20T10:20:40.624+0000 7f0707812040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T10:20:40.754 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.754+0000 7fafad400640 1 Processor -- start 2024-04-20T10:20:40.755 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.755+0000 7fafad400640 1 -- start start 2024-04-20T10:20:40.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.756+0000 7fafad400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fafa80b6130 0x7fafa80b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:40.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.756+0000 7fafad400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafa81557d0 0x7fafa80ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:40.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.756+0000 7fafad400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafa80b4870 con 0x7fafa81553f0 2024-04-20T10:20:40.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.756+0000 7fafad400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fafa80b4b50 con 0x7fafa80b6130 2024-04-20T10:20:40.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.756+0000 7fafad400640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fafa80b4e30 con 0x7fafa81557d0 2024-04-20T10:20:40.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.756+0000 7faf9fe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fafa80b6130 0x7fafa80b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:40.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.756+0000 7fafaca00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafa81557d0 0x7fafa80ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:40.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.756+0000 7fafaca00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafa81557d0 0x7fafa80ced40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.123:48902/0 (socket says 172.21.15.123:48902) 2024-04-20T10:20:40.757 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.756+0000 7fafaca00640 1 -- 172.21.15.123:0/1999408878 learned_addr learned my addr 172.21.15.123:0/1999408878 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-20T10:20:40.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.756+0000 7faf9fe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fafa80b6130 0x7fafa80b41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50996/0 (socket says 172.21.15.123:50996) 2024-04-20T10:20:40.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.757+0000 7fafaca00640 1 -- 172.21.15.123:0/1999408878 >> v2:172.21.15.123:3300/0 conn(0x7fafa80b6130 msgr2=0x7fafa80b41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:20:40.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.757+0000 7fafaca00640 1 --2- 172.21.15.123:0/1999408878 >> v2:172.21.15.123:3300/0 conn(0x7fafa80b6130 0x7fafa80b41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:40.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.757+0000 7fafaca00640 1 -- 172.21.15.123:0/1999408878 >> v1:172.21.15.121:6789/0 conn(0x7fafa81553f0 legacy=0x7fafa80b5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:40.758 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.757+0000 7fafaca00640 1 -- 172.21.15.123:0/1999408878 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa80b5110 con 0x7fafa81557d0 2024-04-20T10:20:40.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.758+0000 7fafaca00640 1 --2- 172.21.15.123:0/1999408878 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafa81557d0 0x7fafa80ced40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fafa0009e50 tx=0x7fafa002bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1dad4f0e43caae2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:40.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.758+0000 7fafa7400640 1 -- 172.21.15.123:0/1999408878 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa002ca70 con 0x7fafa81557d0 2024-04-20T10:20:40.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.758+0000 7fafa7400640 1 -- 172.21.15.123:0/1999408878 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafa002cc10 con 0x7fafa81557d0 2024-04-20T10:20:40.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.758+0000 7fafa7400640 1 -- 172.21.15.123:0/1999408878 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa00343f0 con 0x7fafa81557d0 2024-04-20T10:20:40.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.759+0000 7fafad400640 1 -- 172.21.15.123:0/1999408878 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafa81557d0 msgr2=0x7fafa80ced40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:40.759 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.759+0000 7fafad400640 1 --2- 172.21.15.123:0/1999408878 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafa81557d0 0x7fafa80ced40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fafa0009e50 tx=0x7fafa002bed0 comp rx=0 tx=0).stop 2024-04-20T10:20:40.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.759+0000 7fafad400640 1 -- 172.21.15.123:0/1999408878 shutdown_connections 2024-04-20T10:20:40.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.759+0000 7fafad400640 1 --2- 172.21.15.123:0/1999408878 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafa81557d0 0x7fafa80ced40 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:40.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.759+0000 7fafad400640 1 --2- 172.21.15.123:0/1999408878 >> v2:172.21.15.123:3300/0 conn(0x7fafa80b6130 0x7fafa80b41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:40.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.759+0000 7fafad400640 1 -- 172.21.15.123:0/1999408878 >> 172.21.15.123:0/1999408878 conn(0x7fafa8070090 msgr2=0x7fafa80704a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:20:40.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.759+0000 7fafad400640 1 -- 172.21.15.123:0/1999408878 shutdown_connections 2024-04-20T10:20:40.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.759+0000 7fafad400640 1 -- 172.21.15.123:0/1999408878 wait complete. 2024-04-20T10:20:40.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.761+0000 7fafad400640 1 Processor -- start 2024-04-20T10:20:40.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.761+0000 7fafad400640 1 -- start start 2024-04-20T10:20:40.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.762+0000 7fafad400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafa80b6130 0x7fafa8154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:40.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.762+0000 7fafad400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fafa81553f0 0x7fafa806e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:40.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.762+0000 7fafad400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafa81557d0 0x7fafa806ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:40.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.762+0000 7fafad400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fafa80cf610 con 0x7fafa81553f0 2024-04-20T10:20:40.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.762+0000 7fafad400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fafa80cf490 con 0x7fafa81557d0 2024-04-20T10:20:40.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.762+0000 7fafad400640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fafa80cf790 con 0x7fafa80b6130 2024-04-20T10:20:40.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.762+0000 7fafa7e00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafa80b6130 0x7fafa8154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:40.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.762+0000 7fafaca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafa81557d0 0x7fafa806ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:40.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.762+0000 7fafa7e00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafa80b6130 0x7fafa8154950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.123:48910/0 (socket says 172.21.15.123:48910) 2024-04-20T10:20:40.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.762+0000 7fafa7e00640 1 -- 172.21.15.123:0/2039227366 learned_addr learned my addr 172.21.15.123:0/2039227366 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-20T10:20:40.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.762+0000 7faf9fe00640 1 --2- 172.21.15.123:0/2039227366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fafa81553f0 0x7fafa806e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:40.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.762+0000 7fafaca00640 1 -- 172.21.15.123:0/2039227366 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafa80b6130 msgr2=0x7fafa8154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:40.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.762+0000 7fafaca00640 1 --2- 172.21.15.123:0/2039227366 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafa80b6130 0x7fafa8154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:40.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.762+0000 7fafaca00640 1 -- 172.21.15.123:0/2039227366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fafa81553f0 msgr2=0x7fafa806e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:40.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.762+0000 7fafaca00640 1 --2- 172.21.15.123:0/2039227366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fafa81553f0 0x7fafa806e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:40.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.762+0000 7fafaca00640 1 -- 172.21.15.123:0/2039227366 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa806f290 con 0x7fafa81557d0 2024-04-20T10:20:40.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.762+0000 7fafa7e00640 1 --2- 172.21.15.123:0/2039227366 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafa80b6130 0x7fafa8154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:20:40.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.763+0000 7fafaca00640 1 --2- 172.21.15.123:0/2039227366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafa81557d0 0x7fafa806ed50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fafa0032000 tx=0x7fafa0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:40.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.763+0000 7fafa4c00640 1 -- 172.21.15.123:0/2039227366 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa0044020 con 0x7fafa81557d0 2024-04-20T10:20:40.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.763+0000 7fafa4c00640 1 -- 172.21.15.123:0/2039227366 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafa0040690 con 0x7fafa81557d0 2024-04-20T10:20:40.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.763+0000 7fafa4c00640 1 -- 172.21.15.123:0/2039227366 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafa0009510 con 0x7fafa81557d0 2024-04-20T10:20:40.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.763+0000 7fafad400640 1 -- 172.21.15.123:0/2039227366 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafa806c2c0 con 0x7fafa81557d0 2024-04-20T10:20:40.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.763+0000 7fafad400640 1 -- 172.21.15.123:0/2039227366 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafa806c880 con 0x7fafa81557d0 2024-04-20T10:20:40.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.763+0000 7fafa4c00640 1 -- 172.21.15.123:0/2039227366 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fafa0047020 con 0x7fafa81557d0 2024-04-20T10:20:40.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.763+0000 7fafa4c00640 1 -- 172.21.15.123:0/2039227366 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fafa002ca80 con 0x7fafa81557d0 2024-04-20T10:20:40.765 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.764+0000 7fafad400640 1 -- 172.21.15.123:0/2039227366 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafa8075b30 con 0x7fafa81557d0 2024-04-20T10:20:40.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.766+0000 7fafa4c00640 1 -- 172.21.15.123:0/2039227366 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7fafa004e030 con 0x7fafa81557d0 2024-04-20T10:20:40.844 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.844+0000 7fafad400640 1 -- 172.21.15.123:0/2039227366 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2ee24a08-1624-4961-9642-d7b510282484", "id": 5} v 0) -- 0x7fafa80b6710 con 0x7fafa81557d0 2024-04-20T10:20:40.852 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.852+0000 7fafa4c00640 1 -- 172.21.15.123:0/2039227366 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2ee24a08-1624-4961-9642-d7b510282484", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fafa806f290 con 0x7fafa81557d0 2024-04-20T10:20:40.852 INFO:teuthology.orchestra.run.smithi123.stdout:5 2024-04-20T10:20:40.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.853+0000 7fafad400640 1 -- 172.21.15.123:0/2039227366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafa81557d0 msgr2=0x7fafa806ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:40.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.853+0000 7fafad400640 1 --2- 172.21.15.123:0/2039227366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafa81557d0 0x7fafa806ed50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fafa0032000 tx=0x7fafa0004a40 comp rx=0 tx=0).stop 2024-04-20T10:20:40.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.853+0000 7fafad400640 1 -- 172.21.15.123:0/2039227366 shutdown_connections 2024-04-20T10:20:40.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.853+0000 7fafad400640 1 --2- 172.21.15.123:0/2039227366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafa81557d0 0x7fafa806ed50 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:40.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.853+0000 7fafad400640 1 --2- 172.21.15.123:0/2039227366 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fafa81553f0 0x7fafa806e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:40.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.853+0000 7fafad400640 1 --2- 172.21.15.123:0/2039227366 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fafa80b6130 0x7fafa8154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:40.853 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.853+0000 7fafad400640 1 -- 172.21.15.123:0/2039227366 >> 172.21.15.123:0/2039227366 conn(0x7fafa8070090 msgr2=0x7fafa80b3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:20:40.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.853+0000 7fafad400640 1 -- 172.21.15.123:0/2039227366 shutdown_connections 2024-04-20T10:20:40.854 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:40.853+0000 7fafad400640 1 -- 172.21.15.123:0/2039227366 wait complete. 2024-04-20T10:20:40.869 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new 186118cc-6128-49de-acca-55260945c1b5 6 2024-04-20T10:20:40.904 INFO:tasks.ceph.mgr.y.smithi121.stderr:2024-04-20T10:20:40.903+0000 7f985f644040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T10:20:41.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.090+0000 7ff70e000640 1 Processor -- start 2024-04-20T10:20:41.091 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.091+0000 7ff70e000640 1 -- start start 2024-04-20T10:20:41.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.092+0000 7ff70e000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff7080d4270 0x7ff7080d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:41.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.092+0000 7ff70e000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff7080d4650 0x7ff7080b1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:41.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.092+0000 7ff70e000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7080d3820 con 0x7ff7080b17d0 2024-04-20T10:20:41.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.092+0000 7ff70e000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7ff7080d40c0 con 0x7ff7080d4270 2024-04-20T10:20:41.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.092+0000 7ff70e000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ff7080d3b00 con 0x7ff7080d4650 2024-04-20T10:20:41.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.092+0000 7ff706a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff7080d4650 0x7ff7080b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:41.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.092+0000 7ff706a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff7080d4650 0x7ff7080b1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51010/0 (socket says 172.21.15.123:51010) 2024-04-20T10:20:41.093 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.092+0000 7ff706a00640 1 -- 172.21.15.123:0/4147701610 learned_addr learned my addr 172.21.15.123:0/4147701610 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-20T10:20:41.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.093+0000 7ff707400640 1 --2- 172.21.15.123:0/4147701610 >> v2:172.21.15.121:3300/0 conn(0x7ff7080d4270 0x7ff7080d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:41.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.093+0000 7ff706a00640 1 -- 172.21.15.123:0/4147701610 >> v2:172.21.15.121:3300/0 conn(0x7ff7080d4270 msgr2=0x7ff7080d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:20:41.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.093+0000 7ff706a00640 1 --2- 172.21.15.123:0/4147701610 >> v2:172.21.15.121:3300/0 conn(0x7ff7080d4270 0x7ff7080d1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:41.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.093+0000 7ff706a00640 1 -- 172.21.15.123:0/4147701610 >> v1:172.21.15.123:6789/0 conn(0x7ff7080b17d0 legacy=0x7ff7080d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:41.094 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.093+0000 7ff706a00640 1 -- 172.21.15.123:0/4147701610 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7080d3de0 con 0x7ff7080d4650 2024-04-20T10:20:41.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.094+0000 7ff706a00640 1 --2- 172.21.15.123:0/4147701610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7080d4650 0x7ff7080b1290 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff7080b8060 tx=0x7ff6f402b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=764bb3f2eefa672 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:41.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.094+0000 7ff706000640 1 -- 172.21.15.123:0/4147701610 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4034070 con 0x7ff7080d4650 2024-04-20T10:20:41.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.094+0000 7ff706000640 1 -- 172.21.15.123:0/4147701610 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6f4030860 con 0x7ff7080d4650 2024-04-20T10:20:41.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.094+0000 7ff706000640 1 -- 172.21.15.123:0/4147701610 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4030c30 con 0x7ff7080d4650 2024-04-20T10:20:41.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.095+0000 7ff70e000640 1 -- 172.21.15.123:0/4147701610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7080d4650 msgr2=0x7ff7080b1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:41.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.095+0000 7ff70e000640 1 --2- 172.21.15.123:0/4147701610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7080d4650 0x7ff7080b1290 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff7080b8060 tx=0x7ff6f402b9a0 comp rx=0 tx=0).stop 2024-04-20T10:20:41.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.095+0000 7ff70e000640 1 -- 172.21.15.123:0/4147701610 shutdown_connections 2024-04-20T10:20:41.095 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.095+0000 7ff70e000640 1 --2- 172.21.15.123:0/4147701610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7080d4650 0x7ff7080b1290 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:41.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.095+0000 7ff70e000640 1 --2- 172.21.15.123:0/4147701610 >> v2:172.21.15.121:3300/0 conn(0x7ff7080d4270 0x7ff7080d1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:41.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.095+0000 7ff70e000640 1 -- 172.21.15.123:0/4147701610 >> 172.21.15.123:0/4147701610 conn(0x7ff7080b99c0 msgr2=0x7ff7080b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:20:41.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.095+0000 7ff70e000640 1 -- 172.21.15.123:0/4147701610 shutdown_connections 2024-04-20T10:20:41.096 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.095+0000 7ff70e000640 1 -- 172.21.15.123:0/4147701610 wait complete. 2024-04-20T10:20:41.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.097+0000 7ff70e000640 1 Processor -- start 2024-04-20T10:20:41.097 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.097+0000 7ff70e000640 1 -- start start 2024-04-20T10:20:41.099 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.098+0000 7ff70e000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7080b17d0 0x7ff70806c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:41.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.098+0000 7ff70e000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7080d4270 0x7ff70806b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:41.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.098+0000 7ff70e000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7080d4650 0x7ff7080c8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:41.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.098+0000 7ff70e000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff7080d4ae0 con 0x7ff7080b17d0 2024-04-20T10:20:41.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.098+0000 7ff70e000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff7080d53b0 con 0x7ff7080d4270 2024-04-20T10:20:41.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.098+0000 7ff70e000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ff7080d4c60 con 0x7ff7080d4650 2024-04-20T10:20:41.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.098+0000 7ff706a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7080d4270 0x7ff70806b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:41.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.098+0000 7ff707e00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7080d4650 0x7ff7080c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:41.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.098+0000 7ff706a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7080d4270 0x7ff70806b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51014/0 (socket says 172.21.15.123:51014) 2024-04-20T10:20:41.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.098+0000 7ff707e00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7080d4650 0x7ff7080c8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.123:48912/0 (socket says 172.21.15.123:48912) 2024-04-20T10:20:41.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.098+0000 7ff706a00640 1 -- 172.21.15.123:0/795614269 learned_addr learned my addr 172.21.15.123:0/795614269 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-20T10:20:41.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.098+0000 7ff707400640 1 --2- 172.21.15.123:0/795614269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7080b17d0 0x7ff70806c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:41.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.099+0000 7ff707e00640 1 -- 172.21.15.123:0/795614269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7080d4270 msgr2=0x7ff70806b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:41.100 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.099+0000 7ff707e00640 1 --2- 172.21.15.123:0/795614269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7080d4270 0x7ff70806b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:41.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.099+0000 7ff707e00640 1 -- 172.21.15.123:0/795614269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7080b17d0 msgr2=0x7ff70806c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:41.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.099+0000 7ff707e00640 1 --2- 172.21.15.123:0/795614269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7080b17d0 0x7ff70806c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:41.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.099+0000 7ff707e00640 1 -- 172.21.15.123:0/795614269 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff708155fd0 con 0x7ff7080d4650 2024-04-20T10:20:41.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.099+0000 7ff707400640 1 --2- 172.21.15.123:0/795614269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7080b17d0 0x7ff70806c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:20:41.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.099+0000 7ff706a00640 1 --2- 172.21.15.123:0/795614269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7080d4270 0x7ff70806b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:20:41.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.099+0000 7ff707e00640 1 --2- 172.21.15.123:0/795614269 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7080d4650 0x7ff7080c8730 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff70000c690 tx=0x7ff70000cb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:41.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.099+0000 7ff6ff400640 1 -- 172.21.15.123:0/795614269 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff700003b70 con 0x7ff7080d4650 2024-04-20T10:20:41.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.099+0000 7ff6ff400640 1 -- 172.21.15.123:0/795614269 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff700003d10 con 0x7ff7080d4650 2024-04-20T10:20:41.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.099+0000 7ff70e000640 1 -- 172.21.15.123:0/795614269 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff70806ca80 con 0x7ff7080d4650 2024-04-20T10:20:41.101 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.099+0000 7ff6ff400640 1 -- 172.21.15.123:0/795614269 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff700005310 con 0x7ff7080d4650 2024-04-20T10:20:41.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.099+0000 7ff70e000640 1 -- 172.21.15.123:0/795614269 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7080c8210 con 0x7ff7080d4650 2024-04-20T10:20:41.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.100+0000 7ff6ff400640 1 -- 172.21.15.123:0/795614269 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff7000054b0 con 0x7ff7080d4650 2024-04-20T10:20:41.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.100+0000 7ff6ff400640 1 -- 172.21.15.123:0/795614269 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7ff7000057a0 con 0x7ff7080d4650 2024-04-20T10:20:41.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.100+0000 7ff70e000640 1 -- 172.21.15.123:0/795614269 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff708068980 con 0x7ff7080d4650 2024-04-20T10:20:41.102 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.103+0000 7ff6ff400640 1 -- 172.21.15.123:0/795614269 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7ff70000a640 con 0x7ff7080d4650 2024-04-20T10:20:41.104 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-04-20T10:20:41.104+0000 7f0707812040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T10:20:41.184 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.184+0000 7ff70e000640 1 -- 172.21.15.123:0/795614269 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd new", "uuid": "186118cc-6128-49de-acca-55260945c1b5", "id": 6} v 0) -- 0x7ff7080b3400 con 0x7ff7080d4650 2024-04-20T10:20:41.194 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.194+0000 7ff6ff400640 1 -- 172.21.15.123:0/795614269 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "186118cc-6128-49de-acca-55260945c1b5", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7ff70000a820 con 0x7ff7080d4650 2024-04-20T10:20:41.195 INFO:teuthology.orchestra.run.smithi123.stdout:6 2024-04-20T10:20:41.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.196+0000 7ff70e000640 1 -- 172.21.15.123:0/795614269 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7080d4650 msgr2=0x7ff7080c8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:41.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.196+0000 7ff70e000640 1 --2- 172.21.15.123:0/795614269 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7080d4650 0x7ff7080c8730 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff70000c690 tx=0x7ff70000cb50 comp rx=0 tx=0).stop 2024-04-20T10:20:41.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.196+0000 7ff70e000640 1 -- 172.21.15.123:0/795614269 shutdown_connections 2024-04-20T10:20:41.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.196+0000 7ff70e000640 1 --2- 172.21.15.123:0/795614269 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff7080d4650 0x7ff7080c8730 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:41.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.196+0000 7ff70e000640 1 --2- 172.21.15.123:0/795614269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7080d4270 0x7ff70806b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:41.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.196+0000 7ff70e000640 1 --2- 172.21.15.123:0/795614269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff7080b17d0 0x7ff70806c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:41.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.196+0000 7ff70e000640 1 -- 172.21.15.123:0/795614269 >> 172.21.15.123:0/795614269 conn(0x7ff7080b99c0 msgr2=0x7ff7080d01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:20:41.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.197+0000 7ff70e000640 1 -- 172.21.15.123:0/795614269 shutdown_connections 2024-04-20T10:20:41.197 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.197+0000 7ff70e000640 1 -- 172.21.15.123:0/795614269 wait complete. 2024-04-20T10:20:41.199 INFO:tasks.ceph.mgr.y.smithi121.stderr:2024-04-20T10:20:41.198+0000 7f985f644040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T10:20:41.218 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new a00aac6e-4bd9-415a-a93d-ae85210865ab 7 2024-04-20T10:20:41.259 INFO:tasks.ceph.mgr.y.smithi121.stderr:2024-04-20T10:20:41.259+0000 7f985f644040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T10:20:41.390 INFO:tasks.ceph.mgr.y.smithi121.stderr:2024-04-20T10:20:41.389+0000 7f985f644040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T10:20:41.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.402+0000 7fc97dc00640 1 Processor -- start 2024-04-20T10:20:41.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.402+0000 7fc97dc00640 1 -- start start 2024-04-20T10:20:41.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.403+0000 7fc97dc00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc97815e8c0 0x7fc97806ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:41.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.403+0000 7fc97dc00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc9780d1010 0x7fc9780b6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:41.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.403+0000 7fc97dc00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc9780d46f0 0x7fc9780d4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:41.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.403+0000 7fc97dc00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fc9780b6170 con 0x7fc97815e8c0 2024-04-20T10:20:41.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.403+0000 7fc97dc00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fc9780d15b0 con 0x7fc9780d1010 2024-04-20T10:20:41.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.403+0000 7fc97dc00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fc9780d1890 con 0x7fc9780d46f0 2024-04-20T10:20:41.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.404+0000 7fc97d200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc9780d46f0 0x7fc9780d4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:41.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.404+0000 7fc977400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc9780d1010 0x7fc9780b6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:41.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.404+0000 7fc97d200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc9780d46f0 0x7fc9780d4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.123:48920/0 (socket says 172.21.15.123:48920) 2024-04-20T10:20:41.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.404+0000 7fc977400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc9780d1010 0x7fc9780b6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51024/0 (socket says 172.21.15.123:51024) 2024-04-20T10:20:41.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.404+0000 7fc97d200640 1 -- 172.21.15.123:0/1647813303 learned_addr learned my addr 172.21.15.123:0/1647813303 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-20T10:20:41.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.404+0000 7fc977e00640 1 --2- 172.21.15.123:0/1647813303 >> v2:172.21.15.121:3300/0 conn(0x7fc97815e8c0 0x7fc97806ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:41.405 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.404+0000 7fc977400640 1 -- 172.21.15.123:0/1647813303 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc9780d46f0 msgr2=0x7fc9780d4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:20:41.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.404+0000 7fc977400640 1 --2- 172.21.15.123:0/1647813303 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc9780d46f0 0x7fc9780d4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:41.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.404+0000 7fc977400640 1 -- 172.21.15.123:0/1647813303 >> v2:172.21.15.121:3300/0 conn(0x7fc97815e8c0 msgr2=0x7fc97806ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:20:41.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.404+0000 7fc977400640 1 --2- 172.21.15.123:0/1647813303 >> v2:172.21.15.121:3300/0 conn(0x7fc97815e8c0 0x7fc97806ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:41.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.404+0000 7fc977400640 1 -- 172.21.15.123:0/1647813303 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9780d1b70 con 0x7fc9780d1010 2024-04-20T10:20:41.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.404+0000 7fc97d200640 1 --2- 172.21.15.123:0/1647813303 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc9780d46f0 0x7fc9780d4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:20:41.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.405+0000 7fc977400640 1 --2- 172.21.15.123:0/1647813303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9780d1010 0x7fc9780b6600 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc978162230 tx=0x7fc96402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fe21bb7eb8d9abb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:41.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.405+0000 7fc976a00640 1 -- 172.21.15.123:0/1647813303 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9640315a0 con 0x7fc9780d1010 2024-04-20T10:20:41.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.406+0000 7fc976a00640 1 -- 172.21.15.123:0/1647813303 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc964037070 con 0x7fc9780d1010 2024-04-20T10:20:41.406 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.406+0000 7fc976a00640 1 -- 172.21.15.123:0/1647813303 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc964031960 con 0x7fc9780d1010 2024-04-20T10:20:41.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.406+0000 7fc97dc00640 1 -- 172.21.15.123:0/1647813303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9780d1010 msgr2=0x7fc9780b6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:41.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.406+0000 7fc97dc00640 1 --2- 172.21.15.123:0/1647813303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9780d1010 0x7fc9780b6600 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc978162230 tx=0x7fc96402b840 comp rx=0 tx=0).stop 2024-04-20T10:20:41.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.406+0000 7fc97dc00640 1 -- 172.21.15.123:0/1647813303 shutdown_connections 2024-04-20T10:20:41.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.406+0000 7fc97dc00640 1 --2- 172.21.15.123:0/1647813303 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc9780d46f0 0x7fc9780d4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:41.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.406+0000 7fc97dc00640 1 --2- 172.21.15.123:0/1647813303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9780d1010 0x7fc9780b6600 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:41.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.406+0000 7fc97dc00640 1 --2- 172.21.15.123:0/1647813303 >> v2:172.21.15.121:3300/0 conn(0x7fc97815e8c0 0x7fc97806ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:41.407 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.406+0000 7fc97dc00640 1 -- 172.21.15.123:0/1647813303 >> 172.21.15.123:0/1647813303 conn(0x7fc97809d6f0 msgr2=0x7fc9780b9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:20:41.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.406+0000 7fc97dc00640 1 -- 172.21.15.123:0/1647813303 shutdown_connections 2024-04-20T10:20:41.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.406+0000 7fc97dc00640 1 -- 172.21.15.123:0/1647813303 wait complete. 2024-04-20T10:20:41.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.408+0000 7fc97dc00640 1 Processor -- start 2024-04-20T10:20:41.408 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.408+0000 7fc97dc00640 1 -- start start 2024-04-20T10:20:41.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.409+0000 7fc97dc00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9780d1010 0x7fc97815c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:41.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.409+0000 7fc97dc00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc9780d46f0 0x7fc9781586c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:41.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.409+0000 7fc97dc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc97815e8c0 0x7fc97806b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:41.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.409+0000 7fc97dc00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc9780b6b40 con 0x7fc9780d1010 2024-04-20T10:20:41.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.409+0000 7fc97dc00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc97806f180 con 0x7fc97815e8c0 2024-04-20T10:20:41.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.409+0000 7fc97dc00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fc9780b6cc0 con 0x7fc9780d46f0 2024-04-20T10:20:41.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.409+0000 7fc977400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc9780d46f0 0x7fc9781586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:41.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.409+0000 7fc97d200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc97815e8c0 0x7fc97806b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:41.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.409+0000 7fc97d200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc97815e8c0 0x7fc97806b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:51038/0 (socket says 172.21.15.123:51038) 2024-04-20T10:20:41.411 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.409+0000 7fc977400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc9780d46f0 0x7fc9781586c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.123:48932/0 (socket says 172.21.15.123:48932) 2024-04-20T10:20:41.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.409+0000 7fc97d200640 1 -- 172.21.15.123:0/947769698 learned_addr learned my addr 172.21.15.123:0/947769698 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-04-20T10:20:41.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.410+0000 7fc977e00640 1 --2- 172.21.15.123:0/947769698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9780d1010 0x7fc97815c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:41.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.410+0000 7fc97d200640 1 -- 172.21.15.123:0/947769698 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc9780d46f0 msgr2=0x7fc9781586c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:41.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.410+0000 7fc97d200640 1 --2- 172.21.15.123:0/947769698 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc9780d46f0 0x7fc9781586c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:41.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.410+0000 7fc97d200640 1 -- 172.21.15.123:0/947769698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9780d1010 msgr2=0x7fc97815c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:41.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.410+0000 7fc97d200640 1 --2- 172.21.15.123:0/947769698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9780d1010 0x7fc97815c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:41.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.410+0000 7fc97d200640 1 -- 172.21.15.123:0/947769698 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc97806c940 con 0x7fc97815e8c0 2024-04-20T10:20:41.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.410+0000 7fc97d200640 1 --2- 172.21.15.123:0/947769698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc97815e8c0 0x7fc97806b900 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc970006cc0 tx=0x7fc97000ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:41.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.410+0000 7fc96fe00640 1 -- 172.21.15.123:0/947769698 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc970017070 con 0x7fc97815e8c0 2024-04-20T10:20:41.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.410+0000 7fc96fe00640 1 -- 172.21.15.123:0/947769698 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc97000c940 con 0x7fc97815e8c0 2024-04-20T10:20:41.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.411+0000 7fc97dc00640 1 -- 172.21.15.123:0/947769698 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc97806c0a0 con 0x7fc97815e8c0 2024-04-20T10:20:41.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.411+0000 7fc96fe00640 1 -- 172.21.15.123:0/947769698 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97000ccf0 con 0x7fc97815e8c0 2024-04-20T10:20:41.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.411+0000 7fc97dc00640 1 -- 172.21.15.123:0/947769698 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc97806c660 con 0x7fc97815e8c0 2024-04-20T10:20:41.412 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.411+0000 7fc96fe00640 1 -- 172.21.15.123:0/947769698 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc97001a070 con 0x7fc97815e8c0 2024-04-20T10:20:41.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.411+0000 7fc96fe00640 1 -- 172.21.15.123:0/947769698 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fc970006020 con 0x7fc97815e8c0 2024-04-20T10:20:41.413 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.412+0000 7fc97dc00640 1 -- 172.21.15.123:0/947769698 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc978075b30 con 0x7fc97815e8c0 2024-04-20T10:20:41.414 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.414+0000 7fc96fe00640 1 -- 172.21.15.123:0/947769698 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7fc970006610 con 0x7fc97815e8c0 2024-04-20T10:20:41.424 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-04-20T10:20:41.424+0000 7f0707812040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T10:20:41.500 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-04-20T10:20:41.500+0000 7f0707812040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T10:20:41.507 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.507+0000 7fc97dc00640 1 -- 172.21.15.123:0/947769698 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a00aac6e-4bd9-415a-a93d-ae85210865ab", "id": 7} v 0) -- 0x7fc9780d3450 con 0x7fc97815e8c0 2024-04-20T10:20:41.517 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.517+0000 7fc96fe00640 1 -- 172.21.15.123:0/947769698 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a00aac6e-4bd9-415a-a93d-ae85210865ab", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fc9700067f0 con 0x7fc97815e8c0 2024-04-20T10:20:41.518 INFO:teuthology.orchestra.run.smithi123.stdout:7 2024-04-20T10:20:41.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.519+0000 7fc97dc00640 1 -- 172.21.15.123:0/947769698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc97815e8c0 msgr2=0x7fc97806b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:41.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.519+0000 7fc97dc00640 1 --2- 172.21.15.123:0/947769698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc97815e8c0 0x7fc97806b900 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc970006cc0 tx=0x7fc97000ed60 comp rx=0 tx=0).stop 2024-04-20T10:20:41.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.519+0000 7fc97dc00640 1 -- 172.21.15.123:0/947769698 shutdown_connections 2024-04-20T10:20:41.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.519+0000 7fc97dc00640 1 --2- 172.21.15.123:0/947769698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc97815e8c0 0x7fc97806b900 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:41.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.519+0000 7fc97dc00640 1 --2- 172.21.15.123:0/947769698 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc9780d46f0 0x7fc9781586c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:41.519 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.519+0000 7fc97dc00640 1 --2- 172.21.15.123:0/947769698 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc9780d1010 0x7fc97815c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:41.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.519+0000 7fc97dc00640 1 -- 172.21.15.123:0/947769698 >> 172.21.15.123:0/947769698 conn(0x7fc97809d6f0 msgr2=0x7fc9780b4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:20:41.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.519+0000 7fc97dc00640 1 -- 172.21.15.123:0/947769698 shutdown_connections 2024-04-20T10:20:41.520 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:20:41.519+0000 7fc97dc00640 1 -- 172.21.15.123:0/947769698 wait complete. 2024-04-20T10:20:41.536 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-20T10:20:41.536 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-20T10:20:41.543 INFO:tasks.ceph.osd.0:Started 2024-04-20T10:20:41.544 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-20T10:20:41.544 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-20T10:20:41.551 INFO:tasks.ceph.osd.1:Started 2024-04-20T10:20:41.551 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-20T10:20:41.551 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-20T10:20:41.558 INFO:tasks.ceph.osd.2:Started 2024-04-20T10:20:41.558 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-20T10:20:41.558 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-20T10:20:41.565 INFO:tasks.ceph.osd.3:Started 2024-04-20T10:20:41.565 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-20T10:20:41.565 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-20T10:20:41.570 INFO:tasks.ceph.osd.4:Started 2024-04-20T10:20:41.570 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-20T10:20:41.571 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-20T10:20:41.576 INFO:tasks.ceph.osd.5:Started 2024-04-20T10:20:41.576 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-20T10:20:41.576 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-20T10:20:41.581 INFO:tasks.ceph.osd.6:Started 2024-04-20T10:20:41.581 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-20T10:20:41.582 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-20T10:20:41.588 INFO:tasks.ceph.osd.7:Started 2024-04-20T10:20:41.589 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:20:41.660 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-04-20T10:20:41.660+0000 7f0707812040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T10:20:41.774 INFO:tasks.ceph.mgr.y.smithi121.stderr:2024-04-20T10:20:41.773+0000 7f985f644040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T10:20:41.808 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.807+0000 7f7f64600640 1 Processor -- start 2024-04-20T10:20:41.808 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.808+0000 7f7f64600640 1 -- start start 2024-04-20T10:20:41.809 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.809+0000 7f7f64600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7f5c0cfa00 0x7f7f5c15ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:41.809 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.809+0000 7f7f64600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7f5c0cfde0 0x7f7f5c0cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:41.809 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.809+0000 7f7f64600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f5c0ce200 con 0x7f7f5c158780 2024-04-20T10:20:41.809 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.809+0000 7f7f64600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f7f5c0d0680 con 0x7f7f5c0cfde0 2024-04-20T10:20:41.810 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.809+0000 7f7f64600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f7f5c0ce4a0 con 0x7f7f5c0cfa00 2024-04-20T10:20:41.810 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.809+0000 7f7f61200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7f5c0cfde0 0x7f7f5c0cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:41.810 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.809+0000 7f7f62600640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f7f5c158780 0x7f7f5c158bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:41942/0 (socket says 172.21.15.121:41942) 2024-04-20T10:20:41.810 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.809+0000 7f7f62600640 1 -- 172.21.15.121:0/643996343 learned_addr learned my addr 172.21.15.121:0/643996343 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:20:41.811 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.809+0000 7f7f61c00640 1 --2- 172.21.15.121:0/643996343 >> v2:172.21.15.123:3300/0 conn(0x7f7f5c0cfa00 0x7f7f5c15ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:41.811 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.810+0000 7f7f61200640 1 -- 172.21.15.121:0/643996343 >> v2:172.21.15.123:3300/0 conn(0x7f7f5c0cfa00 msgr2=0x7f7f5c15ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:20:41.811 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.810+0000 7f7f61200640 1 --2- 172.21.15.121:0/643996343 >> v2:172.21.15.123:3300/0 conn(0x7f7f5c0cfa00 0x7f7f5c15ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:41.811 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.810+0000 7f7f61200640 1 -- 172.21.15.121:0/643996343 >> v1:172.21.15.121:6789/0 conn(0x7f7f5c158780 legacy=0x7f7f5c158bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:41.811 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.810+0000 7f7f61200640 1 -- 172.21.15.121:0/643996343 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f5c0ce780 con 0x7f7f5c0cfde0 2024-04-20T10:20:41.811 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.811+0000 7f7f61200640 1 --2- 172.21.15.121:0/643996343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f5c0cfde0 0x7f7f5c0cdcc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c0b6120 tx=0x7f7f4802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e58c15b709b80cb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:41.812 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.811+0000 7f7f5be00640 1 -- 172.21.15.121:0/643996343 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f480315a0 con 0x7f7f5c0cfde0 2024-04-20T10:20:41.812 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.812+0000 7f7f5be00640 1 -- 172.21.15.121:0/643996343 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f48037070 con 0x7f7f5c0cfde0 2024-04-20T10:20:41.812 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.812+0000 7f7f5be00640 1 -- 172.21.15.121:0/643996343 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f48031960 con 0x7f7f5c0cfde0 2024-04-20T10:20:41.813 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.812+0000 7f7f64600640 1 -- 172.21.15.121:0/643996343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f5c0cfde0 msgr2=0x7f7f5c0cdcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:41.813 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.812+0000 7f7f64600640 1 --2- 172.21.15.121:0/643996343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f5c0cfde0 0x7f7f5c0cdcc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c0b6120 tx=0x7f7f4802b840 comp rx=0 tx=0).stop 2024-04-20T10:20:41.813 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.812+0000 7f7f64600640 1 -- 172.21.15.121:0/643996343 shutdown_connections 2024-04-20T10:20:41.813 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.812+0000 7f7f64600640 1 --2- 172.21.15.121:0/643996343 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f5c0cfde0 0x7f7f5c0cdcc0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:41.813 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.812+0000 7f7f64600640 1 --2- 172.21.15.121:0/643996343 >> v2:172.21.15.123:3300/0 conn(0x7f7f5c0cfa00 0x7f7f5c15ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:41.813 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.812+0000 7f7f64600640 1 -- 172.21.15.121:0/643996343 >> 172.21.15.121:0/643996343 conn(0x7f7f5c070450 msgr2=0x7f7f5c070860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:20:41.813 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.812+0000 7f7f64600640 1 -- 172.21.15.121:0/643996343 shutdown_connections 2024-04-20T10:20:41.813 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.812+0000 7f7f64600640 1 -- 172.21.15.121:0/643996343 wait complete. 2024-04-20T10:20:41.814 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.814+0000 7f7f64600640 1 Processor -- start 2024-04-20T10:20:41.814 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.814+0000 7f7f64600640 1 -- start start 2024-04-20T10:20:41.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.815+0000 7f7f64600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f5c0cfa00 0x7f7f5c147da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:41.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.815+0000 7f7f64600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f5c0cfde0 0x7f7f5c06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:41.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.815+0000 7f7f64600640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7f5c158780 0x7f7f5c155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:41.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.815+0000 7f7f64600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c0b4350 con 0x7f7f5c0cfde0 2024-04-20T10:20:41.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.815+0000 7f7f64600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7f5c0ced40 con 0x7f7f5c0cfa00 2024-04-20T10:20:41.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.815+0000 7f7f64600640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f7f5c0b0220 con 0x7f7f5c158780 2024-04-20T10:20:41.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.815+0000 7f7f61200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f5c0cfde0 0x7f7f5c06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:41.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.815+0000 7f7f61200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f5c0cfde0 0x7f7f5c06e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36610/0 (socket says 172.21.15.121:36610) 2024-04-20T10:20:41.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.815+0000 7f7f61c00640 1 --2- 172.21.15.121:0/3198317867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f5c0cfa00 0x7f7f5c147da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:41.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.815+0000 7f7f62600640 1 --2- 172.21.15.121:0/3198317867 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7f5c158780 0x7f7f5c155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:41.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.815+0000 7f7f61200640 1 -- 172.21.15.121:0/3198317867 learned_addr learned my addr 172.21.15.121:0/3198317867 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:20:41.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.816+0000 7f7f61200640 1 -- 172.21.15.121:0/3198317867 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7f5c158780 msgr2=0x7f7f5c155370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:41.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.816+0000 7f7f61200640 1 --2- 172.21.15.121:0/3198317867 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7f5c158780 0x7f7f5c155370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:41.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.816+0000 7f7f61200640 1 -- 172.21.15.121:0/3198317867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f5c0cfa00 msgr2=0x7f7f5c147da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:41.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.816+0000 7f7f61200640 1 --2- 172.21.15.121:0/3198317867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f5c0cfa00 0x7f7f5c147da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:41.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.816+0000 7f7f61200640 1 -- 172.21.15.121:0/3198317867 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f5c06ee30 con 0x7f7f5c0cfde0 2024-04-20T10:20:41.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.816+0000 7f7f61200640 1 --2- 172.21.15.121:0/3198317867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f5c0cfde0 0x7f7f5c06e8f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7f48000c00 tx=0x7f7f480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:41.819 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.816+0000 7f7f59600640 1 -- 172.21.15.121:0/3198317867 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f48032630 con 0x7f7f5c0cfde0 2024-04-20T10:20:41.819 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.817+0000 7f7f59600640 1 -- 172.21.15.121:0/3198317867 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f480327f0 con 0x7f7f5c0cfde0 2024-04-20T10:20:41.819 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.817+0000 7f7f59600640 1 -- 172.21.15.121:0/3198317867 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f48032be0 con 0x7f7f5c0cfde0 2024-04-20T10:20:41.819 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.817+0000 7f7f64600640 1 -- 172.21.15.121:0/3198317867 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f5c1545a0 con 0x7f7f5c0cfde0 2024-04-20T10:20:41.819 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.817+0000 7f7f64600640 1 -- 172.21.15.121:0/3198317867 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f5c06c230 con 0x7f7f5c0cfde0 2024-04-20T10:20:41.819 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.817+0000 7f7f59600640 1 -- 172.21.15.121:0/3198317867 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7f48045780 con 0x7f7f5c0cfde0 2024-04-20T10:20:41.819 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.817+0000 7f7f59600640 1 -- 172.21.15.121:0/3198317867 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f7f4802f030 con 0x7f7f5c0cfde0 2024-04-20T10:20:41.819 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.818+0000 7f7f64600640 1 -- 172.21.15.121:0/3198317867 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f5c068980 con 0x7f7f5c0cfde0 2024-04-20T10:20:41.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.820+0000 7f7f59600640 1 -- 172.21.15.121:0/3198317867 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f7f48040070 con 0x7f7f5c0cfde0 2024-04-20T10:20:41.835 INFO:tasks.ceph.mgr.y.smithi121.stderr:2024-04-20T10:20:41.835+0000 7f985f644040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T10:20:41.885 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.884+0000 7f7f64600640 1 -- 172.21.15.121:0/3198317867 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7f5c06f060 con 0x7f7f5c0cfde0 2024-04-20T10:20:41.885 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.885+0000 7f7f59600640 1 -- 172.21.15.121:0/3198317867 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f7f48032d80 con 0x7f7f5c0cfde0 2024-04-20T10:20:41.886 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:20:41.886 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":10,"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","created":"2024-04-20T10:20:38.076076+0000","modified":"2024-04-20T10:20:41.510382+0000","last_up_change":"0.000000","last_in_change":"2024-04-20T10:20:41.510382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"629a7b8c-a40a-4a2a-a3cc-d9f357774310","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f2570399-aa50-4ff3-bf5d-3d09207290db","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"50893807-dc30-49f4-b41f-739366626658","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"dadb01bc-5491-4ffc-8824-d48a057240b5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b3f3b4a1-b375-4526-b69e-344ab24c2da1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2ee24a08-1624-4961-9642-d7b510282484","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"186118cc-6128-49de-acca-55260945c1b5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a00aac6e-4bd9-415a-a93d-ae85210865ab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:20:41.887 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.887+0000 7f7f64600640 1 -- 172.21.15.121:0/3198317867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f5c0cfde0 msgr2=0x7f7f5c06e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:41.888 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.887+0000 7f7f64600640 1 --2- 172.21.15.121:0/3198317867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f5c0cfde0 0x7f7f5c06e8f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7f48000c00 tx=0x7f7f480046c0 comp rx=0 tx=0).stop 2024-04-20T10:20:41.888 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.887+0000 7f7f64600640 1 -- 172.21.15.121:0/3198317867 shutdown_connections 2024-04-20T10:20:41.888 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.887+0000 7f7f64600640 1 --2- 172.21.15.121:0/3198317867 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7f5c158780 0x7f7f5c155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:41.888 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.887+0000 7f7f64600640 1 --2- 172.21.15.121:0/3198317867 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7f5c0cfde0 0x7f7f5c06e8f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:41.888 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.887+0000 7f7f64600640 1 --2- 172.21.15.121:0/3198317867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f5c0cfa00 0x7f7f5c147da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:41.888 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.887+0000 7f7f64600640 1 -- 172.21.15.121:0/3198317867 >> 172.21.15.121:0/3198317867 conn(0x7f7f5c070450 msgr2=0x7f7f5c0af9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:20:41.888 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.887+0000 7f7f64600640 1 -- 172.21.15.121:0/3198317867 shutdown_connections 2024-04-20T10:20:41.888 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:41.887+0000 7f7f64600640 1 -- 172.21.15.121:0/3198317867 wait complete. 2024-04-20T10:20:41.903 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-20T10:20:41.903 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-20T10:20:42.022 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-04-20T10:20:42.022+0000 7f0707812040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T10:20:42.024 INFO:tasks.ceph.mgr.y.smithi121.stderr:2024-04-20T10:20:42.023+0000 7f985f644040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T10:20:42.080 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-04-20T10:20:42.080+0000 7f0707812040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T10:20:42.194 INFO:tasks.ceph.mgr.y.smithi121.stderr:2024-04-20T10:20:42.193+0000 7f985f644040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T10:20:42.229 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T10:20:42.228+0000 7f9c15723640 -1 Falling back to public interface 2024-04-20T10:20:42.241 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-04-20T10:20:42.241+0000 7f4dd2c0c640 -1 Falling back to public interface 2024-04-20T10:20:42.245 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-04-20T10:20:42.245+0000 7fb3d2199640 -1 Falling back to public interface 2024-04-20T10:20:42.254 INFO:tasks.ceph.mgr.y.smithi121.stderr:2024-04-20T10:20:42.254+0000 7f985f644040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T10:20:42.263 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-04-20T10:20:42.263+0000 7f0707812040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T10:20:42.266 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-04-20T10:20:42.266+0000 7f5d7be23640 -1 Falling back to public interface 2024-04-20T10:20:42.276 INFO:tasks.ceph.osd.2.smithi121.stderr:2024-04-20T10:20:42.276+0000 7f11e37dc640 -1 Falling back to public interface 2024-04-20T10:20:42.279 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T10:20:42.279+0000 7fad7162e640 -1 Falling back to public interface 2024-04-20T10:20:42.279 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-04-20T10:20:42.279+0000 7f11475f6640 -1 Falling back to public interface 2024-04-20T10:20:42.281 INFO:tasks.ceph.osd.3.smithi121.stderr:2024-04-20T10:20:42.281+0000 7fca44801640 -1 Falling back to public interface 2024-04-20T10:20:42.340 INFO:tasks.ceph.mgr.y.smithi121.stderr:2024-04-20T10:20:42.339+0000 7f985f644040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T10:20:42.392 INFO:tasks.ceph.mgr.y.smithi121.stderr:2024-04-20T10:20:42.391+0000 7f985f644040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T10:20:42.435 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-04-20T10:20:42.435+0000 7f0707812040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T10:20:42.492 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-04-20T10:20:42.492+0000 7f0707812040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T10:20:42.576 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-04-20T10:20:42.576+0000 7f0707812040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T10:20:42.628 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-04-20T10:20:42.628+0000 7f0707812040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T10:20:42.707 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-20T10:20:42.866 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.865+0000 7f8301000640 1 Processor -- start 2024-04-20T10:20:42.866 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.866+0000 7f8301000640 1 -- start start 2024-04-20T10:20:42.868 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.867+0000 7f8301000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f82fc0d4a60 0x7f82fc0d1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:42.868 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.867+0000 7f8301000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f82fc15e930 0x7f82fc0d09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:42.868 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.867+0000 7f8301000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82fc0d1b30 con 0x7f82fc0d4e40 2024-04-20T10:20:42.868 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.867+0000 7f8301000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f82fc0d27b0 con 0x7f82fc0d4a60 2024-04-20T10:20:42.868 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.867+0000 7f8301000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f82fc0d1e10 con 0x7f82fc15e930 2024-04-20T10:20:42.868 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.867+0000 7f82f9e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f82fc0d4a60 0x7f82fc0d1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:42.868 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.867+0000 7f82fa800640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f82fc0d4e40 0x7f82fc0b6010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:41954/0 (socket says 172.21.15.121:41954) 2024-04-20T10:20:42.868 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.867+0000 7f82fa800640 1 -- 172.21.15.121:0/3687684208 learned_addr learned my addr 172.21.15.121:0/3687684208 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:20:42.868 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.867+0000 7f82f3e00640 1 --2- 172.21.15.121:0/3687684208 >> v2:172.21.15.123:3300/0 conn(0x7f82fc15e930 0x7f82fc0d09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:42.869 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.868+0000 7f82f9e00640 1 -- 172.21.15.121:0/3687684208 >> v2:172.21.15.123:3300/0 conn(0x7f82fc15e930 msgr2=0x7f82fc0d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:20:42.869 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.868+0000 7f82f9e00640 1 --2- 172.21.15.121:0/3687684208 >> v2:172.21.15.123:3300/0 conn(0x7f82fc15e930 0x7f82fc0d09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:42.869 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.868+0000 7f82f9e00640 1 -- 172.21.15.121:0/3687684208 >> v1:172.21.15.121:6789/0 conn(0x7f82fc0d4e40 legacy=0x7f82fc0b6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:42.869 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.868+0000 7f82f9e00640 1 -- 172.21.15.121:0/3687684208 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82fc0d2160 con 0x7f82fc0d4a60 2024-04-20T10:20:42.870 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.869+0000 7f82f9e00640 1 --2- 172.21.15.121:0/3687684208 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82fc0d4a60 0x7f82fc0d1450 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f82fc08e5b0 tx=0x7f82ec02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bfcde6d3cec1b94 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:42.870 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.869+0000 7f82f9400640 1 -- 172.21.15.121:0/3687684208 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec035020 con 0x7f82fc0d4a60 2024-04-20T10:20:42.870 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.869+0000 7f82f9400640 1 -- 172.21.15.121:0/3687684208 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82ec031830 con 0x7f82fc0d4a60 2024-04-20T10:20:42.870 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.869+0000 7f82f9400640 1 -- 172.21.15.121:0/3687684208 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ec031c00 con 0x7f82fc0d4a60 2024-04-20T10:20:42.870 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.869+0000 7f8301000640 1 -- 172.21.15.121:0/3687684208 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82fc0d4a60 msgr2=0x7f82fc0d1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:42.871 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.869+0000 7f8301000640 1 --2- 172.21.15.121:0/3687684208 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82fc0d4a60 0x7f82fc0d1450 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f82fc08e5b0 tx=0x7f82ec02b8d0 comp rx=0 tx=0).stop 2024-04-20T10:20:42.871 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.870+0000 7f8301000640 1 -- 172.21.15.121:0/3687684208 shutdown_connections 2024-04-20T10:20:42.871 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.870+0000 7f8301000640 1 --2- 172.21.15.121:0/3687684208 >> v2:172.21.15.123:3300/0 conn(0x7f82fc15e930 0x7f82fc0d09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:42.871 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.870+0000 7f8301000640 1 --2- 172.21.15.121:0/3687684208 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82fc0d4a60 0x7f82fc0d1450 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:42.871 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.870+0000 7f8301000640 1 -- 172.21.15.121:0/3687684208 >> 172.21.15.121:0/3687684208 conn(0x7f82fc0b9730 msgr2=0x7f82fc0b9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:20:42.871 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.870+0000 7f8301000640 1 -- 172.21.15.121:0/3687684208 shutdown_connections 2024-04-20T10:20:42.871 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.870+0000 7f8301000640 1 -- 172.21.15.121:0/3687684208 wait complete. 2024-04-20T10:20:42.871 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.871+0000 7f8301000640 1 Processor -- start 2024-04-20T10:20:42.872 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.871+0000 7f8301000640 1 -- start start 2024-04-20T10:20:42.874 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.872+0000 7f8301000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82fc0d4a60 0x7f82fc1f8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:42.874 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.872+0000 7f8301000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82fc0d4e40 0x7f82fc0b3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:42.874 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.872+0000 7f8301000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82fc15e930 0x7f82fc06cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:42.874 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.872+0000 7f8301000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f82fc0d3c50 con 0x7f82fc0d4e40 2024-04-20T10:20:42.874 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.872+0000 7f8301000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f82fc0d52b0 con 0x7f82fc0d4a60 2024-04-20T10:20:42.874 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.872+0000 7f8301000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f82fc0d3dd0 con 0x7f82fc15e930 2024-04-20T10:20:42.874 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.873+0000 7f82f3e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82fc0d4e40 0x7f82fc0b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:42.874 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.873+0000 7f82f3e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82fc0d4e40 0x7f82fc0b3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:36632/0 (socket says 172.21.15.121:36632) 2024-04-20T10:20:42.874 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.873+0000 7f82f3e00640 1 -- 172.21.15.121:0/1729360209 learned_addr learned my addr 172.21.15.121:0/1729360209 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:20:42.874 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.873+0000 7f82f9e00640 1 --2- 172.21.15.121:0/1729360209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82fc0d4a60 0x7f82fc1f8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:42.874 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.873+0000 7f82fa800640 1 --2- 172.21.15.121:0/1729360209 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82fc15e930 0x7f82fc06cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:42.875 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.873+0000 7f82f3e00640 1 -- 172.21.15.121:0/1729360209 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82fc15e930 msgr2=0x7f82fc06cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:42.875 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.873+0000 7f82f3e00640 1 --2- 172.21.15.121:0/1729360209 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82fc15e930 0x7f82fc06cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:42.875 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.873+0000 7f82f3e00640 1 -- 172.21.15.121:0/1729360209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82fc0d4a60 msgr2=0x7f82fc1f8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:42.875 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.873+0000 7f82f3e00640 1 --2- 172.21.15.121:0/1729360209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82fc0d4a60 0x7f82fc1f8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:42.875 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.873+0000 7f82f3e00640 1 -- 172.21.15.121:0/1729360209 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82fc06be00 con 0x7f82fc0d4e40 2024-04-20T10:20:42.875 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.874+0000 7f82f3e00640 1 --2- 172.21.15.121:0/1729360209 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82fc0d4e40 0x7f82fc0b3790 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f82e400e840 tx=0x7f82e400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:42.876 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.874+0000 7f82f2000640 1 -- 172.21.15.121:0/1729360209 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e4017070 con 0x7f82fc0d4e40 2024-04-20T10:20:42.876 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.874+0000 7f82f2000640 1 -- 172.21.15.121:0/1729360209 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82e400c940 con 0x7f82fc0d4e40 2024-04-20T10:20:42.876 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.874+0000 7f82f2000640 1 -- 172.21.15.121:0/1729360209 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82e400cd10 con 0x7f82fc0d4e40 2024-04-20T10:20:42.876 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.874+0000 7f8301000640 1 -- 172.21.15.121:0/1729360209 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82fc06c0e0 con 0x7f82fc0d4e40 2024-04-20T10:20:42.876 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.874+0000 7f8301000640 1 -- 172.21.15.121:0/1729360209 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82fc06c6a0 con 0x7f82fc0d4e40 2024-04-20T10:20:42.876 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.874+0000 7f82f2000640 1 -- 172.21.15.121:0/1729360209 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f82e401a070 con 0x7f82fc0d4e40 2024-04-20T10:20:42.876 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.874+0000 7f82f2000640 1 -- 172.21.15.121:0/1729360209 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f82e401c450 con 0x7f82fc0d4e40 2024-04-20T10:20:42.877 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.875+0000 7f8301000640 1 -- 172.21.15.121:0/1729360209 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82fc069430 con 0x7f82fc0d4e40 2024-04-20T10:20:42.878 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.878+0000 7f82f2000640 1 -- 172.21.15.121:0/1729360209 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f82e4013470 con 0x7f82fc0d4e40 2024-04-20T10:20:42.946 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.946+0000 7f8301000640 1 -- 172.21.15.121:0/1729360209 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f82fc0b19d0 con 0x7f82fc0d4e40 2024-04-20T10:20:42.947 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.946+0000 7f82f2000640 1 -- 172.21.15.121:0/1729360209 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f82e4013650 con 0x7f82fc0d4e40 2024-04-20T10:20:42.947 INFO:teuthology.misc.health.smithi121.stdout: 2024-04-20T10:20:42.947 INFO:teuthology.misc.health.smithi121.stdout:{"epoch":10,"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","created":"2024-04-20T10:20:38.076076+0000","modified":"2024-04-20T10:20:41.510382+0000","last_up_change":"0.000000","last_in_change":"2024-04-20T10:20:41.510382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"629a7b8c-a40a-4a2a-a3cc-d9f357774310","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f2570399-aa50-4ff3-bf5d-3d09207290db","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"50893807-dc30-49f4-b41f-739366626658","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"dadb01bc-5491-4ffc-8824-d48a057240b5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b3f3b4a1-b375-4526-b69e-344ab24c2da1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2ee24a08-1624-4961-9642-d7b510282484","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"186118cc-6128-49de-acca-55260945c1b5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a00aac6e-4bd9-415a-a93d-ae85210865ab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:20:42.948 INFO:tasks.ceph.mgr.y.smithi121.stderr:2024-04-20T10:20:42.948+0000 7f985f644040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T10:20:42.949 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.948+0000 7f8301000640 1 -- 172.21.15.121:0/1729360209 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82fc0d4e40 msgr2=0x7f82fc0b3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:42.949 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.948+0000 7f8301000640 1 --2- 172.21.15.121:0/1729360209 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82fc0d4e40 0x7f82fc0b3790 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f82e400e840 tx=0x7f82e400ed00 comp rx=0 tx=0).stop 2024-04-20T10:20:42.949 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.948+0000 7f8301000640 1 -- 172.21.15.121:0/1729360209 shutdown_connections 2024-04-20T10:20:42.949 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.948+0000 7f8301000640 1 --2- 172.21.15.121:0/1729360209 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82fc15e930 0x7f82fc06cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:42.949 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.948+0000 7f8301000640 1 --2- 172.21.15.121:0/1729360209 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82fc0d4e40 0x7f82fc0b3790 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:42.949 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.948+0000 7f8301000640 1 --2- 172.21.15.121:0/1729360209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82fc0d4a60 0x7f82fc1f8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:42.949 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.948+0000 7f8301000640 1 -- 172.21.15.121:0/1729360209 >> 172.21.15.121:0/1729360209 conn(0x7f82fc0b9730 msgr2=0x7f82fc15edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:20:42.949 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.948+0000 7f8301000640 1 -- 172.21.15.121:0/1729360209 shutdown_connections 2024-04-20T10:20:42.949 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:42.948+0000 7f8301000640 1 -- 172.21.15.121:0/1729360209 wait complete. 2024-04-20T10:20:42.961 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-20T10:20:43.082 INFO:tasks.ceph.mgr.y.smithi121.stderr:2024-04-20T10:20:43.081+0000 7f985f644040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T10:20:43.178 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-04-20T10:20:43.178+0000 7f0707812040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T10:20:43.307 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-04-20T10:20:43.307+0000 7f0707812040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T10:20:44.158 INFO:tasks.ceph.mgr.y.smithi121.stderr:2024-04-20T10:20:44.157+0000 7f97ffe00640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-20T10:20:44.159 INFO:tasks.ceph.mgr.y.smithi121.stderr:2024-04-20T10:20:44.158+0000 7f97ffe00640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-20T10:20:45.658 INFO:tasks.ceph.osd.2.smithi121.stderr:2024-04-20T10:20:45.658+0000 7f11e37dc640 -1 osd.2 0 log_to_monitors true 2024-04-20T10:20:45.871 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-04-20T10:20:45.871+0000 7f5d7be23640 -1 osd.7 0 log_to_monitors true 2024-04-20T10:20:45.875 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-04-20T10:20:45.875+0000 7fb3d2199640 -1 osd.0 0 log_to_monitors true 2024-04-20T10:20:45.916 INFO:tasks.ceph.osd.3.smithi121.stderr:2024-04-20T10:20:45.915+0000 7fca44801640 -1 osd.3 0 log_to_monitors true 2024-04-20T10:20:45.921 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T10:20:45.922+0000 7fad7162e640 -1 osd.6 0 log_to_monitors true 2024-04-20T10:20:45.925 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T10:20:45.925+0000 7f9c15723640 -1 osd.1 0 log_to_monitors true 2024-04-20T10:20:46.121 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-04-20T10:20:46.121+0000 7f4dd2c0c640 -1 osd.4 0 log_to_monitors true 2024-04-20T10:20:46.172 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-04-20T10:20:46.172+0000 7f11475f6640 -1 osd.5 0 log_to_monitors true 2024-04-20T10:20:49.157 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-04-20T10:20:49.156+0000 7f5d77000640 -1 osd.7 0 waiting for initial osdmap 2024-04-20T10:20:49.163 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-04-20T10:20:49.163+0000 7f5d6c600640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T10:20:49.262 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T10:20:49.263+0000 7fad6c400640 -1 osd.6 0 waiting for initial osdmap 2024-04-20T10:20:49.269 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-04-20T10:20:49.269+0000 7f4dcd400640 -1 osd.4 0 waiting for initial osdmap 2024-04-20T10:20:49.270 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T10:20:49.270+0000 7fad61a00640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T10:20:49.277 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-04-20T10:20:49.277+0000 7f4dc3400640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T10:20:49.472 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-04-20T10:20:49.469+0000 7fb3cd000640 -1 osd.0 0 waiting for initial osdmap 2024-04-20T10:20:49.473 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T10:20:49.469+0000 7f9c10600640 -1 osd.1 0 waiting for initial osdmap 2024-04-20T10:20:49.475 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-04-20T10:20:49.475+0000 7fb3c3000640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T10:20:49.481 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T10:20:49.481+0000 7f9c06600640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T10:20:49.535 INFO:tasks.ceph.osd.3.smithi121.stderr:2024-04-20T10:20:49.535+0000 7fca3f600640 -1 osd.3 0 waiting for initial osdmap 2024-04-20T10:20:49.542 INFO:tasks.ceph.osd.3.smithi121.stderr:2024-04-20T10:20:49.542+0000 7fca35600640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T10:20:49.581 INFO:tasks.ceph.osd.2.smithi121.stderr:2024-04-20T10:20:49.580+0000 7f11de600640 -1 osd.2 0 waiting for initial osdmap 2024-04-20T10:20:49.589 INFO:tasks.ceph.osd.2.smithi121.stderr:2024-04-20T10:20:49.588+0000 7f11d4600640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T10:20:49.615 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-04-20T10:20:49.615+0000 7f1142e00640 -1 osd.5 0 waiting for initial osdmap 2024-04-20T10:20:49.621 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-04-20T10:20:49.621+0000 7f1138400640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T10:20:49.768 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-20T10:20:49.935 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.934+0000 7fd8f5600640 1 Processor -- start 2024-04-20T10:20:49.935 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.934+0000 7fd8f5600640 1 -- start start 2024-04-20T10:20:49.936 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.936+0000 7fd8f5600640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd8f006d0a0 0x7fd8f00ad980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:49.936 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.936+0000 7fd8f5600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd8f00adec0 0x7fd8f015e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:49.936 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.936+0000 7fd8f5600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8f00ae670 con 0x7fd8f015a0a0 2024-04-20T10:20:49.936 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.936+0000 7fd8f5600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fd8f00ae950 con 0x7fd8f00adec0 2024-04-20T10:20:49.936 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.936+0000 7fd8f5600640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fd8f00aec30 con 0x7fd8f006d0a0 2024-04-20T10:20:49.938 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.936+0000 7fd8ee000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd8f006d0a0 0x7fd8f00ad980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:49.938 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.936+0000 7fd8ef400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd8f00adec0 0x7fd8f015e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:49.938 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.937+0000 7fd8eea00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fd8f015a0a0 0x7fd8f006c990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.121:40454/0 (socket says 172.21.15.121:40454) 2024-04-20T10:20:49.938 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.937+0000 7fd8ee000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd8f006d0a0 0x7fd8f00ad980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.121:43668/0 (socket says 172.21.15.121:43668) 2024-04-20T10:20:49.938 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.937+0000 7fd8eea00640 1 -- 172.21.15.121:0/921694343 learned_addr learned my addr 172.21.15.121:0/921694343 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:20:49.939 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.937+0000 7fd8ef400640 1 -- 172.21.15.121:0/921694343 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd8f006d0a0 msgr2=0x7fd8f00ad980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:20:49.939 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.937+0000 7fd8ef400640 1 --2- 172.21.15.121:0/921694343 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd8f006d0a0 0x7fd8f00ad980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:49.939 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.937+0000 7fd8ef400640 1 -- 172.21.15.121:0/921694343 >> v1:172.21.15.123:6789/0 conn(0x7fd8f015a0a0 legacy=0x7fd8f006c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:49.939 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.937+0000 7fd8ef400640 1 -- 172.21.15.121:0/921694343 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f015de10 con 0x7fd8f00adec0 2024-04-20T10:20:49.939 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.937+0000 7fd8ee000640 1 --2- 172.21.15.121:0/921694343 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd8f006d0a0 0x7fd8f00ad980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:20:49.939 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.939+0000 7fd8ef400640 1 --2- 172.21.15.121:0/921694343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8f00adec0 0x7fd8f015e330 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8009d90 tx=0x7fd8e802bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c30c4551a90dfc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:49.940 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.939+0000 7fd8ed600640 1 -- 172.21.15.121:0/921694343 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8039070 con 0x7fd8f00adec0 2024-04-20T10:20:49.940 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.940+0000 7fd8ed600640 1 -- 172.21.15.121:0/921694343 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8e802cc50 con 0x7fd8f00adec0 2024-04-20T10:20:49.940 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.940+0000 7fd8ed600640 1 -- 172.21.15.121:0/921694343 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8034440 con 0x7fd8f00adec0 2024-04-20T10:20:49.941 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.940+0000 7fd8f5600640 1 -- 172.21.15.121:0/921694343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8f00adec0 msgr2=0x7fd8f015e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:49.941 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.940+0000 7fd8f5600640 1 --2- 172.21.15.121:0/921694343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8f00adec0 0x7fd8f015e330 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8009d90 tx=0x7fd8e802bea0 comp rx=0 tx=0).stop 2024-04-20T10:20:49.941 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.940+0000 7fd8f5600640 1 -- 172.21.15.121:0/921694343 shutdown_connections 2024-04-20T10:20:49.941 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.940+0000 7fd8f5600640 1 --2- 172.21.15.121:0/921694343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8f00adec0 0x7fd8f015e330 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:49.942 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.940+0000 7fd8f5600640 1 --2- 172.21.15.121:0/921694343 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd8f006d0a0 0x7fd8f00ad980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:49.942 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.940+0000 7fd8f5600640 1 -- 172.21.15.121:0/921694343 >> 172.21.15.121:0/921694343 conn(0x7fd8f00b20d0 msgr2=0x7fd8f00b24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:20:49.942 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.940+0000 7fd8f5600640 1 -- 172.21.15.121:0/921694343 shutdown_connections 2024-04-20T10:20:49.942 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.940+0000 7fd8f5600640 1 -- 172.21.15.121:0/921694343 wait complete. 2024-04-20T10:20:49.942 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.942+0000 7fd8f5600640 1 Processor -- start 2024-04-20T10:20:49.943 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.942+0000 7fd8f5600640 1 -- start start 2024-04-20T10:20:49.945 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.943+0000 7fd8f5600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8f006d0a0 0x7fd8f0152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:49.945 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.943+0000 7fd8f5600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8f00adec0 0x7fd8f01e7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:49.945 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.943+0000 7fd8f5600640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd8f015a0a0 0x7fd8f01e8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:49.946 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.943+0000 7fd8f5600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f015e870 con 0x7fd8f00adec0 2024-04-20T10:20:49.946 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.943+0000 7fd8f5600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f00ae310 con 0x7fd8f006d0a0 2024-04-20T10:20:49.946 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.943+0000 7fd8f5600640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fd8f015e9f0 con 0x7fd8f015a0a0 2024-04-20T10:20:49.946 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.943+0000 7fd8ee000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8f00adec0 0x7fd8f01e7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:49.946 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.943+0000 7fd8ee000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8f00adec0 0x7fd8f01e7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:51642/0 (socket says 172.21.15.121:51642) 2024-04-20T10:20:49.946 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.943+0000 7fd8ee000640 1 -- 172.21.15.121:0/3594403661 learned_addr learned my addr 172.21.15.121:0/3594403661 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:20:49.946 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.944+0000 7fd8eea00640 1 --2- 172.21.15.121:0/3594403661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8f006d0a0 0x7fd8f0152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:49.946 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.944+0000 7fd8ef400640 1 --2- 172.21.15.121:0/3594403661 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd8f015a0a0 0x7fd8f01e8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:49.946 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.944+0000 7fd8ee000640 1 -- 172.21.15.121:0/3594403661 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd8f015a0a0 msgr2=0x7fd8f01e8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:49.946 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.944+0000 7fd8ee000640 1 --2- 172.21.15.121:0/3594403661 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd8f015a0a0 0x7fd8f01e8470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:49.947 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.944+0000 7fd8ee000640 1 -- 172.21.15.121:0/3594403661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8f006d0a0 msgr2=0x7fd8f0152ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:49.947 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.944+0000 7fd8ee000640 1 --2- 172.21.15.121:0/3594403661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8f006d0a0 0x7fd8f0152ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:49.947 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.944+0000 7fd8ee000640 1 -- 172.21.15.121:0/3594403661 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f00738c0 con 0x7fd8f00adec0 2024-04-20T10:20:49.947 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.944+0000 7fd8eea00640 1 --2- 172.21.15.121:0/3594403661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8f006d0a0 0x7fd8f0152ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:20:49.947 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.944+0000 7fd8ee000640 1 --2- 172.21.15.121:0/3594403661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8f00adec0 0x7fd8f01e7f30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc00ebe0 tx=0x7fd8dc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:49.947 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.944+0000 7fd8e6a00640 1 -- 172.21.15.121:0/3594403661 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc017070 con 0x7fd8f00adec0 2024-04-20T10:20:49.947 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.944+0000 7fd8e6a00640 1 -- 172.21.15.121:0/3594403661 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8dc00cc30 con 0x7fd8f00adec0 2024-04-20T10:20:49.947 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.944+0000 7fd8f5600640 1 -- 172.21.15.121:0/3594403661 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8f0154430 con 0x7fd8f00adec0 2024-04-20T10:20:49.947 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.944+0000 7fd8e6a00640 1 -- 172.21.15.121:0/3594403661 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc013470 con 0x7fd8f00adec0 2024-04-20T10:20:49.947 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.944+0000 7fd8f5600640 1 -- 172.21.15.121:0/3594403661 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8f0072c00 con 0x7fd8f00adec0 2024-04-20T10:20:49.949 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.946+0000 7fd8f5600640 1 -- 172.21.15.121:0/3594403661 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8f0069430 con 0x7fd8f00adec0 2024-04-20T10:20:49.949 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.946+0000 7fd8e6a00640 1 -- 172.21.15.121:0/3594403661 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7fd8dc01a070 con 0x7fd8f00adec0 2024-04-20T10:20:49.949 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.947+0000 7fd8e6a00640 1 --2- 172.21.15.121:0/3594403661 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fd8d006c860 0x7fd8d006ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:49.949 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.947+0000 7fd8e6a00640 1 -- 172.21.15.121:0/3594403661 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2231+0+0 (secure 0 0 0) 0x7fd8dc08afe0 con 0x7fd8f00adec0 2024-04-20T10:20:49.949 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.947+0000 7fd8eea00640 1 --2- 172.21.15.121:0/3594403661 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fd8d006c860 0x7fd8d006ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:49.949 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.948+0000 7fd8eea00640 1 --2- 172.21.15.121:0/3594403661 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fd8d006c860 0x7fd8d006ed20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0000f60 tx=0x7fd8e0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:49.953 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:49.952+0000 7fd8e6a00640 1 -- 172.21.15.121:0/3594403661 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd8dc05a290 con 0x7fd8f00adec0 2024-04-20T10:20:50.137 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:50.137+0000 7fd8f5600640 1 -- 172.21.15.121:0/3594403661 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd8f006f780 con 0x7fd8f00adec0 2024-04-20T10:20:50.138 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:50.138+0000 7fd8e6a00640 1 -- 172.21.15.121:0/3594403661 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+7058 (secure 0 0 0) 0x7fd8dc059b90 con 0x7fd8f00adec0 2024-04-20T10:20:50.138 INFO:teuthology.misc.health.smithi121.stdout: 2024-04-20T10:20:50.139 INFO:teuthology.misc.health.smithi121.stdout:{"epoch":13,"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","created":"2024-04-20T10:20:38.076076+0000","modified":"2024-04-20T10:20:48.172459+0000","last_up_change":"0.000000","last_in_change":"2024-04-20T10:20:41.510382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"629a7b8c-a40a-4a2a-a3cc-d9f357774310","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f2570399-aa50-4ff3-bf5d-3d09207290db","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"50893807-dc30-49f4-b41f-739366626658","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"dadb01bc-5491-4ffc-8824-d48a057240b5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b3f3b4a1-b375-4526-b69e-344ab24c2da1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2ee24a08-1624-4961-9642-d7b510282484","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"186118cc-6128-49de-acca-55260945c1b5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"a00aac6e-4bd9-415a-a93d-ae85210865ab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:20:50.141 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:50.140+0000 7fd8f5600640 1 -- 172.21.15.121:0/3594403661 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fd8d006c860 msgr2=0x7fd8d006ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:50.141 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:50.140+0000 7fd8f5600640 1 --2- 172.21.15.121:0/3594403661 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fd8d006c860 0x7fd8d006ed20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0000f60 tx=0x7fd8e0009040 comp rx=0 tx=0).stop 2024-04-20T10:20:50.141 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:50.140+0000 7fd8f5600640 1 -- 172.21.15.121:0/3594403661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8f00adec0 msgr2=0x7fd8f01e7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:50.141 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:50.140+0000 7fd8f5600640 1 --2- 172.21.15.121:0/3594403661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8f00adec0 0x7fd8f01e7f30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc00ebe0 tx=0x7fd8dc00c520 comp rx=0 tx=0).stop 2024-04-20T10:20:50.141 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:50.140+0000 7fd8f5600640 1 -- 172.21.15.121:0/3594403661 shutdown_connections 2024-04-20T10:20:50.141 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:50.140+0000 7fd8f5600640 1 --2- 172.21.15.121:0/3594403661 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd8f015a0a0 0x7fd8f01e8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:50.141 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:50.140+0000 7fd8f5600640 1 --2- 172.21.15.121:0/3594403661 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd8f00adec0 0x7fd8f01e7f30 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:50.141 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:50.140+0000 7fd8f5600640 1 --2- 172.21.15.121:0/3594403661 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fd8d006c860 0x7fd8d006ed20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:50.141 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:50.140+0000 7fd8f5600640 1 --2- 172.21.15.121:0/3594403661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8f006d0a0 0x7fd8f0152ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:50.141 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:50.141+0000 7fd8f5600640 1 -- 172.21.15.121:0/3594403661 >> 172.21.15.121:0/3594403661 conn(0x7fd8f00b20d0 msgr2=0x7fd8f015bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:20:50.142 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:50.141+0000 7fd8f5600640 1 -- 172.21.15.121:0/3594403661 shutdown_connections 2024-04-20T10:20:50.142 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:50.141+0000 7fd8f5600640 1 -- 172.21.15.121:0/3594403661 wait complete. 2024-04-20T10:20:50.154 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-20T10:20:55.122 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:20:55.131 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:20:56.961 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-20T10:20:57.141 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.140+0000 7f288a800640 1 Processor -- start 2024-04-20T10:20:57.141 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.140+0000 7f288a800640 1 -- start start 2024-04-20T10:20:57.144 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.141+0000 7f288a800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2884158c70 0x7f28840ae030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:57.144 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.142+0000 7f288a800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2884162b50 0x7f288406ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:57.144 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.142+0000 7f288a800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2884163170 con 0x7f288406c990 2024-04-20T10:20:57.145 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.142+0000 7f288a800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f2884072680 con 0x7f2884162b50 2024-04-20T10:20:57.145 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.142+0000 7f288a800640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f2884071d90 con 0x7f2884158c70 2024-04-20T10:20:57.145 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.142+0000 7f2882a00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f288406c990 0x7f2884162440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:58796/0 (socket says 172.21.15.121:58796) 2024-04-20T10:20:57.145 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.142+0000 7f2882a00640 1 -- 172.21.15.121:0/4211844665 learned_addr learned my addr 172.21.15.121:0/4211844665 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:20:57.145 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.142+0000 7f2882000640 1 -- 172.21.15.121:0/4211844665 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2951610353 0 0) 0x7f2884163170 con 0x7f288406c990 2024-04-20T10:20:57.145 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.142+0000 7f2882000640 1 -- 172.21.15.121:0/4211844665 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2868003520 con 0x7f288406c990 2024-04-20T10:20:57.145 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.142+0000 7f2883400640 1 --2- 172.21.15.121:0/4211844665 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2884158c70 0x7f28840ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:57.145 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.142+0000 7f2883e00640 1 --2- 172.21.15.121:0/4211844665 >> v2:172.21.15.123:3300/0 conn(0x7f2884162b50 0x7f288406ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:57.145 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.143+0000 7f2882000640 1 -- 172.21.15.121:0/4211844665 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 473139121 0 0) 0x7f2868003520 con 0x7f288406c990 2024-04-20T10:20:57.145 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.143+0000 7f2882000640 1 -- 172.21.15.121:0/4211844665 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2884158c70 msgr2=0x7f28840ae030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:20:57.145 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.143+0000 7f2882000640 1 --2- 172.21.15.121:0/4211844665 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2884158c70 0x7f28840ae030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:57.145 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.143+0000 7f2882000640 1 -- 172.21.15.121:0/4211844665 >> v2:172.21.15.123:3300/0 conn(0x7f2884162b50 msgr2=0x7f288406ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:20:57.145 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.143+0000 7f2882000640 1 --2- 172.21.15.121:0/4211844665 >> v2:172.21.15.123:3300/0 conn(0x7f2884162b50 0x7f288406ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:57.146 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.143+0000 7f2882000640 1 -- 172.21.15.121:0/4211844665 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2884072030 con 0x7f288406c990 2024-04-20T10:20:57.146 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.143+0000 7f2882000640 1 -- 172.21.15.121:0/4211844665 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f2870001670 con 0x7f288406c990 2024-04-20T10:20:57.146 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.143+0000 7f2882000640 1 -- 172.21.15.121:0/4211844665 >> v1:172.21.15.121:6789/0 conn(0x7f288406c990 legacy=0x7f2884162440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:57.146 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.143+0000 7f2882000640 1 --2- 172.21.15.121:0/4211844665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28680039b0 0x7f2868003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:57.146 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.143+0000 7f2882000640 1 --2- 172.21.15.121:0/4211844665 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2884158c70 0x7f2868004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:57.146 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.143+0000 7f2882000640 1 --2- 172.21.15.121:0/4211844665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2868004d40 0x7f28680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:57.146 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.143+0000 7f2882000640 1 -- 172.21.15.121:0/4211844665 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2884072680 con 0x7f2868004d40 2024-04-20T10:20:57.146 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.143+0000 7f2882000640 1 -- 172.21.15.121:0/4211844665 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2884071d90 con 0x7f28680039b0 2024-04-20T10:20:57.146 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.143+0000 7f2882000640 1 -- 172.21.15.121:0/4211844665 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f286800ba70 con 0x7f2884158c70 2024-04-20T10:20:57.146 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.143+0000 7f2883e00640 1 --2- 172.21.15.121:0/4211844665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2868004d40 0x7f28680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:57.146 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.143+0000 7f2882a00640 1 --2- 172.21.15.121:0/4211844665 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2884158c70 0x7f2868004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:57.146 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.143+0000 7f2883400640 1 --2- 172.21.15.121:0/4211844665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28680039b0 0x7f2868003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:57.146 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.144+0000 7f2883e00640 1 -- 172.21.15.121:0/4211844665 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2884158c70 msgr2=0x7f2868004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:20:57.146 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.144+0000 7f2883e00640 1 --2- 172.21.15.121:0/4211844665 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2884158c70 0x7f2868004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:57.147 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.144+0000 7f2883e00640 1 -- 172.21.15.121:0/4211844665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28680039b0 msgr2=0x7f2868003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:20:57.147 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.144+0000 7f2883e00640 1 --2- 172.21.15.121:0/4211844665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28680039b0 0x7f2868003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:57.147 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.144+0000 7f2883e00640 1 -- 172.21.15.121:0/4211844665 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2868003520 con 0x7f2868004d40 2024-04-20T10:20:57.148 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.145+0000 7f2883e00640 1 --2- 172.21.15.121:0/4211844665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2868004d40 0x7f28680051c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f287c007360 tx=0x7f2868005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d64bf308cf5b6f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:57.148 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.145+0000 7f2882000640 1 -- 172.21.15.121:0/4211844665 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287c039070 con 0x7f2868004d40 2024-04-20T10:20:57.148 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.145+0000 7f2882000640 1 -- 172.21.15.121:0/4211844665 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f287c02ccf0 con 0x7f2868004d40 2024-04-20T10:20:57.148 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.145+0000 7f2882000640 1 -- 172.21.15.121:0/4211844665 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287c009be0 con 0x7f2868004d40 2024-04-20T10:20:57.148 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.145+0000 7f288a800640 1 -- 172.21.15.121:0/4211844665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2868004d40 msgr2=0x7f28680051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:57.148 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.145+0000 7f288a800640 1 --2- 172.21.15.121:0/4211844665 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2868004d40 0x7f28680051c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f287c007360 tx=0x7f2868005830 comp rx=0 tx=0).stop 2024-04-20T10:20:57.148 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.146+0000 7f2883e00640 1 -- 172.21.15.121:0/4211844665 reap_dead start 2024-04-20T10:20:57.148 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.146+0000 7f288a800640 1 -- 172.21.15.121:0/4211844665 shutdown_connections 2024-04-20T10:20:57.149 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.146+0000 7f288a800640 1 -- 172.21.15.121:0/4211844665 >> 172.21.15.121:0/4211844665 conn(0x7f28840b20d0 msgr2=0x7f28840b24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:20:57.149 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.146+0000 7f288a800640 1 -- 172.21.15.121:0/4211844665 shutdown_connections 2024-04-20T10:20:57.149 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.146+0000 7f288a800640 1 -- 172.21.15.121:0/4211844665 wait complete. 2024-04-20T10:20:57.149 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.147+0000 7f288a800640 1 Processor -- start 2024-04-20T10:20:57.149 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.148+0000 7f288a800640 1 -- start start 2024-04-20T10:20:57.151 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.148+0000 7f288a800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2868004d40 0x7f28840afc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:57.151 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.148+0000 7f288a800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2884160540 0x7f28841609c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:57.151 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.148+0000 7f288a800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2884145a40 0x7f2884145ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:57.151 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.149+0000 7f288a800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f28840ae030 con 0x7f2884145a40 2024-04-20T10:20:57.151 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.149+0000 7f288a800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2884159050 con 0x7f2884160540 2024-04-20T10:20:57.151 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.149+0000 7f288a800640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f28840ae1b0 con 0x7f2868004d40 2024-04-20T10:20:57.151 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.149+0000 7f2883e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2884145a40 0x7f2884145ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:57.151 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.149+0000 7f2883e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2884145a40 0x7f2884145ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44058/0 (socket says 172.21.15.121:44058) 2024-04-20T10:20:57.151 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.149+0000 7f2883e00640 1 -- 172.21.15.121:0/140644979 learned_addr learned my addr 172.21.15.121:0/140644979 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:20:57.152 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.149+0000 7f2883400640 1 --2- 172.21.15.121:0/140644979 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2868004d40 0x7f28840afc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:57.152 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.149+0000 7f2883e00640 1 -- 172.21.15.121:0/140644979 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2868004d40 msgr2=0x7f28840afc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:57.152 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.149+0000 7f2883e00640 1 --2- 172.21.15.121:0/140644979 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2868004d40 0x7f28840afc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:57.152 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.149+0000 7f2883e00640 1 -- 172.21.15.121:0/140644979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2884160540 msgr2=0x7f28841609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:57.152 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.149+0000 7f2883e00640 1 --2- 172.21.15.121:0/140644979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2884160540 0x7f28841609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:57.152 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.149+0000 7f2883e00640 1 -- 172.21.15.121:0/140644979 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28841467b0 con 0x7f2884145a40 2024-04-20T10:20:57.152 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.149+0000 7f2883e00640 1 --2- 172.21.15.121:0/140644979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2884145a40 0x7f2884145ec0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f287c02c760 tx=0x7f287c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:57.152 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.150+0000 7f287b400640 1 -- 172.21.15.121:0/140644979 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287c039030 con 0x7f2884145a40 2024-04-20T10:20:57.152 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.150+0000 7f287b400640 1 -- 172.21.15.121:0/140644979 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f287c043070 con 0x7f2884145a40 2024-04-20T10:20:57.152 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.150+0000 7f287b400640 1 -- 172.21.15.121:0/140644979 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287c009be0 con 0x7f2884145a40 2024-04-20T10:20:57.152 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.150+0000 7f288a800640 1 -- 172.21.15.121:0/140644979 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2884159270 con 0x7f2884145a40 2024-04-20T10:20:57.152 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.150+0000 7f288a800640 1 -- 172.21.15.121:0/140644979 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2884159830 con 0x7f2884145a40 2024-04-20T10:20:57.153 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.151+0000 7f288a800640 1 -- 172.21.15.121:0/140644979 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2884069430 con 0x7f2884145a40 2024-04-20T10:20:57.153 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.152+0000 7f287b400640 1 -- 172.21.15.121:0/140644979 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f287c03d7d0 con 0x7f2884145a40 2024-04-20T10:20:57.153 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.153+0000 7f287b400640 1 --2- 172.21.15.121:0/140644979 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f286406c920 0x7f286406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:57.153 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.153+0000 7f287b400640 1 -- 172.21.15.121:0/140644979 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f287c0adc20 con 0x7f2884145a40 2024-04-20T10:20:57.154 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.153+0000 7f2883400640 1 --2- 172.21.15.121:0/140644979 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f286406c920 0x7f286406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:57.154 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.154+0000 7f2883400640 1 --2- 172.21.15.121:0/140644979 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f286406c920 0x7f286406ede0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2874009990 tx=0x7f2874008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:57.158 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.158+0000 7f287b400640 1 -- 172.21.15.121:0/140644979 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f287c07c270 con 0x7f2884145a40 2024-04-20T10:20:57.360 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.359+0000 7f288a800640 1 -- 172.21.15.121:0/140644979 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2884074df0 con 0x7f2884145a40 2024-04-20T10:20:57.361 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.360+0000 7f287b400640 1 -- 172.21.15.121:0/140644979 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12920 (secure 0 0 0) 0x7f287c07bb70 con 0x7f2884145a40 2024-04-20T10:20:57.361 INFO:teuthology.misc.health.smithi121.stdout: 2024-04-20T10:20:57.361 INFO:teuthology.misc.health.smithi121.stdout:{"epoch":17,"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","created":"2024-04-20T10:20:38.076076+0000","modified":"2024-04-20T10:20:53.276076+0000","last_up_change":"2024-04-20T10:20:50.162700+0000","last_in_change":"2024-04-20T10:20:41.510382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:20:51.141365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"629a7b8c-a40a-4a2a-a3cc-d9f357774310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6809","nonce":1044588835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6811","nonce":1044588835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6815","nonce":1044588835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6813","nonce":1044588835}]},"public_addr":"172.21.15.121:6809/1044588835","cluster_addr":"172.21.15.121:6811/1044588835","heartbeat_back_addr":"172.21.15.121:6815/1044588835","heartbeat_front_addr":"172.21.15.121:6813/1044588835","state":["exists","up"]},{"osd":1,"uuid":"f2570399-aa50-4ff3-bf5d-3d09207290db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6801","nonce":3505199624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6803","nonce":3505199624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6807","nonce":3505199624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6805","nonce":3505199624}]},"public_addr":"172.21.15.121:6801/3505199624","cluster_addr":"172.21.15.121:6803/3505199624","heartbeat_back_addr":"172.21.15.121:6807/3505199624","heartbeat_front_addr":"172.21.15.121:6805/3505199624","state":["exists","up"]},{"osd":2,"uuid":"50893807-dc30-49f4-b41f-739366626658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6817","nonce":3656724207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6819","nonce":3656724207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6827","nonce":3656724207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6822","nonce":3656724207}]},"public_addr":"172.21.15.121:6817/3656724207","cluster_addr":"172.21.15.121:6819/3656724207","heartbeat_back_addr":"172.21.15.121:6827/3656724207","heartbeat_front_addr":"172.21.15.121:6822/3656724207","state":["exists","up"]},{"osd":3,"uuid":"dadb01bc-5491-4ffc-8824-d48a057240b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6823","nonce":2232342832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6826","nonce":2232342832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6831","nonce":2232342832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6829","nonce":2232342832}]},"public_addr":"172.21.15.121:6823/2232342832","cluster_addr":"172.21.15.121:6826/2232342832","heartbeat_back_addr":"172.21.15.121:6831/2232342832","heartbeat_front_addr":"172.21.15.121:6829/2232342832","state":["exists","up"]},{"osd":4,"uuid":"b3f3b4a1-b375-4526-b69e-344ab24c2da1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6801","nonce":99353111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6803","nonce":99353111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6807","nonce":99353111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6805","nonce":99353111}]},"public_addr":"172.21.15.123:6801/99353111","cluster_addr":"172.21.15.123:6803/99353111","heartbeat_back_addr":"172.21.15.123:6807/99353111","heartbeat_front_addr":"172.21.15.123:6805/99353111","state":["exists","up"]},{"osd":5,"uuid":"2ee24a08-1624-4961-9642-d7b510282484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6818","nonce":2504374688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6822","nonce":2504374688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6830","nonce":2504374688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6826","nonce":2504374688}]},"public_addr":"172.21.15.123:6818/2504374688","cluster_addr":"172.21.15.123:6822/2504374688","heartbeat_back_addr":"172.21.15.123:6830/2504374688","heartbeat_front_addr":"172.21.15.123:6826/2504374688","state":["exists","up"]},{"osd":6,"uuid":"186118cc-6128-49de-acca-55260945c1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6819","nonce":4139299599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6823","nonce":4139299599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6829","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6831","nonce":4139299599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6827","nonce":4139299599}]},"public_addr":"172.21.15.123:6819/4139299599","cluster_addr":"172.21.15.123:6823/4139299599","heartbeat_back_addr":"172.21.15.123:6831/4139299599","heartbeat_front_addr":"172.21.15.123:6827/4139299599","state":["exists","up"]},{"osd":7,"uuid":"a00aac6e-4bd9-415a-a93d-ae85210865ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6809","nonce":4043981069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6811","nonce":4043981069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6815","nonce":4043981069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6813","nonce":4043981069}]},"public_addr":"172.21.15.123:6809/4043981069","cluster_addr":"172.21.15.123:6811/4043981069","heartbeat_back_addr":"172.21.15.123:6815/4043981069","heartbeat_front_addr":"172.21.15.123:6813/4043981069","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.904540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.958585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.616367+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.885244+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.103127+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.167248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.884348+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.862356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:20:57.364 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.363+0000 7f288a800640 1 -- 172.21.15.121:0/140644979 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f286406c920 msgr2=0x7f286406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:57.364 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.363+0000 7f288a800640 1 --2- 172.21.15.121:0/140644979 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f286406c920 0x7f286406ede0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2874009990 tx=0x7f2874008040 comp rx=0 tx=0).stop 2024-04-20T10:20:57.364 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.363+0000 7f288a800640 1 -- 172.21.15.121:0/140644979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2884145a40 msgr2=0x7f2884145ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:57.364 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.363+0000 7f288a800640 1 --2- 172.21.15.121:0/140644979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2884145a40 0x7f2884145ec0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f287c02c760 tx=0x7f287c004ab0 comp rx=0 tx=0).stop 2024-04-20T10:20:57.365 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.363+0000 7f288a800640 1 -- 172.21.15.121:0/140644979 shutdown_connections 2024-04-20T10:20:57.365 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.363+0000 7f288a800640 1 --2- 172.21.15.121:0/140644979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2884145a40 0x7f2884145ec0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:57.365 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.363+0000 7f288a800640 1 --2- 172.21.15.121:0/140644979 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f286406c920 0x7f286406ede0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:57.365 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.363+0000 7f288a800640 1 --2- 172.21.15.121:0/140644979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2884160540 0x7f28841609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:57.365 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.363+0000 7f288a800640 1 --2- 172.21.15.121:0/140644979 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2868004d40 0x7f28840afc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:57.365 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.363+0000 7f288a800640 1 -- 172.21.15.121:0/140644979 >> 172.21.15.121:0/140644979 conn(0x7f28840b20d0 msgr2=0x7f28840729d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:20:57.365 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.364+0000 7f288a800640 1 -- 172.21.15.121:0/140644979 shutdown_connections 2024-04-20T10:20:57.365 INFO:teuthology.misc.health.smithi121.stderr:2024-04-20T10:20:57.364+0000 7f288a800640 1 -- 172.21.15.121:0/140644979 wait complete. 2024-04-20T10:20:57.379 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-20T10:20:57.379 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-20T10:20:57.379 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-20T10:20:57.380 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-20T10:20:57.384 INFO:tasks.ceph.mds.a:Started 2024-04-20T10:20:57.384 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-20T10:20:57.384 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-20T10:20:57.389 INFO:tasks.ceph.mds.c:Started 2024-04-20T10:20:57.389 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-20T10:20:57.389 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-20T10:20:57.397 INFO:tasks.ceph.mds.b:Started 2024-04-20T10:20:57.397 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-20T10:20:57.397 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-20T10:20:57.404 INFO:tasks.ceph.mds.d:Started 2024-04-20T10:20:57.404 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-20T10:20:57.405 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:20:57.528 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2024-04-20T10:20:57.536 INFO:tasks.ceph.mds.c.smithi121.stdout:starting mds.c at 2024-04-20T10:20:57.552 INFO:tasks.ceph.mds.b.smithi123.stdout:starting mds.b at 2024-04-20T10:20:57.555 INFO:tasks.ceph.mds.d.smithi123.stdout:starting mds.d at 2024-04-20T10:20:57.580 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.580+0000 7feb0ae00640 1 Processor -- start 2024-04-20T10:20:57.580 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.580+0000 7feb0ae00640 1 -- start start 2024-04-20T10:20:57.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.581+0000 7feb0ae00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7feb04161d50 0x7feb040d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:57.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.581+0000 7feb0ae00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7feb040b3330 0x7feb040b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:57.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.581+0000 7feb0ae00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb0406c190 con 0x7feb0406c6d0 2024-04-20T10:20:57.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.581+0000 7feb0ae00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7feb04162130 con 0x7feb040b3330 2024-04-20T10:20:57.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.581+0000 7feb0ae00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7feb0406c470 con 0x7feb04161d50 2024-04-20T10:20:57.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.582+0000 7feb09a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7feb04161d50 0x7feb040d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:57.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.582+0000 7feb09a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7feb04161d50 0x7feb040d38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.121:52630/0 (socket says 172.21.15.121:52630) 2024-04-20T10:20:57.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.582+0000 7feb09a00640 1 -- 172.21.15.121:0/255335077 learned_addr learned my addr 172.21.15.121:0/255335077 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:20:57.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.582+0000 7feb09000640 1 --2- 172.21.15.121:0/255335077 >> v2:172.21.15.121:3300/0 conn(0x7feb040b3330 0x7feb040b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:57.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.582+0000 7feb0a400640 1 --1- 172.21.15.121:0/255335077 >> v1:172.21.15.123:6789/0 conn(0x7feb0406c6d0 0x7feb040cfa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.121:45138/0 (socket says 172.21.15.121:45138) 2024-04-20T10:20:57.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.582+0000 7feb09000640 1 -- 172.21.15.121:0/255335077 >> v2:172.21.15.123:3300/0 conn(0x7feb04161d50 msgr2=0x7feb040d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:20:57.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.582+0000 7feb09000640 1 --2- 172.21.15.121:0/255335077 >> v2:172.21.15.123:3300/0 conn(0x7feb04161d50 0x7feb040d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:57.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.582+0000 7feb09000640 1 -- 172.21.15.121:0/255335077 >> v1:172.21.15.123:6789/0 conn(0x7feb0406c6d0 legacy=0x7feb040cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:57.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.582+0000 7feb09000640 1 -- 172.21.15.121:0/255335077 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb040d2e00 con 0x7feb040b3330 2024-04-20T10:20:57.584 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.583+0000 7feb09000640 1 --2- 172.21.15.121:0/255335077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb040b3330 0x7feb040b3880 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7feb040b6f50 tx=0x7feaf002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9cb7a2f54738d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:57.584 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.584+0000 7feb03e00640 1 -- 172.21.15.121:0/255335077 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf0035020 con 0x7feb040b3330 2024-04-20T10:20:57.584 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.584+0000 7feb03e00640 1 -- 172.21.15.121:0/255335077 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaf0031640 con 0x7feb040b3330 2024-04-20T10:20:57.584 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.584+0000 7feb03e00640 1 -- 172.21.15.121:0/255335077 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf002d040 con 0x7feb040b3330 2024-04-20T10:20:57.585 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.584+0000 7feb0ae00640 1 -- 172.21.15.121:0/255335077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb040b3330 msgr2=0x7feb040b3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:57.585 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.584+0000 7feb0ae00640 1 --2- 172.21.15.121:0/255335077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb040b3330 0x7feb040b3880 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7feb040b6f50 tx=0x7feaf002b8f0 comp rx=0 tx=0).stop 2024-04-20T10:20:57.585 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.584+0000 7feb0ae00640 1 -- 172.21.15.121:0/255335077 shutdown_connections 2024-04-20T10:20:57.585 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.584+0000 7feb0ae00640 1 --2- 172.21.15.121:0/255335077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb040b3330 0x7feb040b3880 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:57.585 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.584+0000 7feb0ae00640 1 --2- 172.21.15.121:0/255335077 >> v2:172.21.15.123:3300/0 conn(0x7feb04161d50 0x7feb040d38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:57.585 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.584+0000 7feb0ae00640 1 -- 172.21.15.121:0/255335077 >> 172.21.15.121:0/255335077 conn(0x7feb040d4450 msgr2=0x7feb040d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:20:57.585 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.584+0000 7feb0ae00640 1 -- 172.21.15.121:0/255335077 shutdown_connections 2024-04-20T10:20:57.586 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.584+0000 7feb0ae00640 1 -- 172.21.15.121:0/255335077 wait complete. 2024-04-20T10:20:57.586 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.586+0000 7feb0ae00640 1 Processor -- start 2024-04-20T10:20:57.586 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.586+0000 7feb0ae00640 1 -- start start 2024-04-20T10:20:57.588 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.587+0000 7feb0ae00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb0406c6d0 0x7feb040cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:57.588 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.587+0000 7feb0ae00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb040b3330 0x7feb040ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:57.588 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.587+0000 7feb0ae00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb04161d50 0x7feb040cd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:57.588 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.587+0000 7feb0ae00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7feb0406cb20 con 0x7feb0406c6d0 2024-04-20T10:20:57.589 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.587+0000 7feb0ae00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feb040d33c0 con 0x7feb040b3330 2024-04-20T10:20:57.589 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.587+0000 7feb0ae00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7feb0406cca0 con 0x7feb04161d50 2024-04-20T10:20:57.589 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.587+0000 7feb09a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb0406c6d0 0x7feb040cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:57.589 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.587+0000 7feb09a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb0406c6d0 0x7feb040cb520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44106/0 (socket says 172.21.15.121:44106) 2024-04-20T10:20:57.589 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.587+0000 7feb09a00640 1 -- 172.21.15.121:0/3464037107 learned_addr learned my addr 172.21.15.121:0/3464037107 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:20:57.589 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.587+0000 7feb09000640 1 --2- 172.21.15.121:0/3464037107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb040b3330 0x7feb040ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:57.589 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.588+0000 7feb0a400640 1 --2- 172.21.15.121:0/3464037107 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb04161d50 0x7feb040cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:57.589 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.588+0000 7feb09a00640 1 -- 172.21.15.121:0/3464037107 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb04161d50 msgr2=0x7feb040cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:57.589 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.588+0000 7feb09a00640 1 --2- 172.21.15.121:0/3464037107 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb04161d50 0x7feb040cd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:57.589 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.588+0000 7feb09a00640 1 -- 172.21.15.121:0/3464037107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb040b3330 msgr2=0x7feb040ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:57.589 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.588+0000 7feb09a00640 1 --2- 172.21.15.121:0/3464037107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb040b3330 0x7feb040ccef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:57.589 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.588+0000 7feb09a00640 1 -- 172.21.15.121:0/3464037107 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb040cba60 con 0x7feb0406c6d0 2024-04-20T10:20:57.589 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.588+0000 7feb09a00640 1 --2- 172.21.15.121:0/3464037107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb0406c6d0 0x7feb040cb520 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7feaf800e840 tx=0x7feaf800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:57.590 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.588+0000 7feb01600640 1 -- 172.21.15.121:0/3464037107 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf8017070 con 0x7feb0406c6d0 2024-04-20T10:20:57.590 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.588+0000 7feb01600640 1 -- 172.21.15.121:0/3464037107 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaf800c8e0 con 0x7feb0406c6d0 2024-04-20T10:20:57.590 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.588+0000 7feb0ae00640 1 -- 172.21.15.121:0/3464037107 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb040ce2b0 con 0x7feb0406c6d0 2024-04-20T10:20:57.590 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.589+0000 7feb01600640 1 -- 172.21.15.121:0/3464037107 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf800ccd0 con 0x7feb0406c6d0 2024-04-20T10:20:57.591 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.589+0000 7feb0ae00640 1 -- 172.21.15.121:0/3464037107 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb040b8af0 con 0x7feb0406c6d0 2024-04-20T10:20:57.591 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.590+0000 7feb0ae00640 1 -- 172.21.15.121:0/3464037107 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb04075b60 con 0x7feb0406c6d0 2024-04-20T10:20:57.591 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.591+0000 7feb01600640 1 -- 172.21.15.121:0/3464037107 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feaf801a070 con 0x7feb0406c6d0 2024-04-20T10:20:57.591 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.591+0000 7feb01600640 1 --2- 172.21.15.121:0/3464037107 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7feae406c8f0 0x7feae406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:57.592 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.592+0000 7feb01600640 1 -- 172.21.15.121:0/3464037107 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7feaf805ff20 con 0x7feb0406c6d0 2024-04-20T10:20:57.592 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.592+0000 7feb09000640 1 --2- 172.21.15.121:0/3464037107 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7feae406c8f0 0x7feae406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:57.593 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.593+0000 7feb09000640 1 --2- 172.21.15.121:0/3464037107 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7feae406c8f0 0x7feae406edb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7feaf0036000 tx=0x7feaf003c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:57.598 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.597+0000 7feb01600640 1 -- 172.21.15.121:0/3464037107 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7feaf8059730 con 0x7feb0406c6d0 2024-04-20T10:20:57.787 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.787+0000 7feb0ae00640 1 -- 172.21.15.121:0/3464037107 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feb040d4c00 con 0x7feb0406c6d0 2024-04-20T10:20:57.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.789+0000 7feb01600640 1 -- 172.21.15.121:0/3464037107 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12920 (secure 0 0 0) 0x7feaf805da30 con 0x7feb0406c6d0 2024-04-20T10:20:57.790 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:20:57.790 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":17,"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","created":"2024-04-20T10:20:38.076076+0000","modified":"2024-04-20T10:20:53.276076+0000","last_up_change":"2024-04-20T10:20:50.162700+0000","last_in_change":"2024-04-20T10:20:41.510382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:20:51.141365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"629a7b8c-a40a-4a2a-a3cc-d9f357774310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6809","nonce":1044588835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6811","nonce":1044588835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6815","nonce":1044588835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6813","nonce":1044588835}]},"public_addr":"172.21.15.121:6809/1044588835","cluster_addr":"172.21.15.121:6811/1044588835","heartbeat_back_addr":"172.21.15.121:6815/1044588835","heartbeat_front_addr":"172.21.15.121:6813/1044588835","state":["exists","up"]},{"osd":1,"uuid":"f2570399-aa50-4ff3-bf5d-3d09207290db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6801","nonce":3505199624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6803","nonce":3505199624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6807","nonce":3505199624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6805","nonce":3505199624}]},"public_addr":"172.21.15.121:6801/3505199624","cluster_addr":"172.21.15.121:6803/3505199624","heartbeat_back_addr":"172.21.15.121:6807/3505199624","heartbeat_front_addr":"172.21.15.121:6805/3505199624","state":["exists","up"]},{"osd":2,"uuid":"50893807-dc30-49f4-b41f-739366626658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6817","nonce":3656724207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6819","nonce":3656724207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6827","nonce":3656724207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6822","nonce":3656724207}]},"public_addr":"172.21.15.121:6817/3656724207","cluster_addr":"172.21.15.121:6819/3656724207","heartbeat_back_addr":"172.21.15.121:6827/3656724207","heartbeat_front_addr":"172.21.15.121:6822/3656724207","state":["exists","up"]},{"osd":3,"uuid":"dadb01bc-5491-4ffc-8824-d48a057240b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6823","nonce":2232342832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6826","nonce":2232342832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6831","nonce":2232342832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6829","nonce":2232342832}]},"public_addr":"172.21.15.121:6823/2232342832","cluster_addr":"172.21.15.121:6826/2232342832","heartbeat_back_addr":"172.21.15.121:6831/2232342832","heartbeat_front_addr":"172.21.15.121:6829/2232342832","state":["exists","up"]},{"osd":4,"uuid":"b3f3b4a1-b375-4526-b69e-344ab24c2da1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6801","nonce":99353111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6803","nonce":99353111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6807","nonce":99353111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6805","nonce":99353111}]},"public_addr":"172.21.15.123:6801/99353111","cluster_addr":"172.21.15.123:6803/99353111","heartbeat_back_addr":"172.21.15.123:6807/99353111","heartbeat_front_addr":"172.21.15.123:6805/99353111","state":["exists","up"]},{"osd":5,"uuid":"2ee24a08-1624-4961-9642-d7b510282484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6818","nonce":2504374688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6822","nonce":2504374688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6830","nonce":2504374688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6826","nonce":2504374688}]},"public_addr":"172.21.15.123:6818/2504374688","cluster_addr":"172.21.15.123:6822/2504374688","heartbeat_back_addr":"172.21.15.123:6830/2504374688","heartbeat_front_addr":"172.21.15.123:6826/2504374688","state":["exists","up"]},{"osd":6,"uuid":"186118cc-6128-49de-acca-55260945c1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6819","nonce":4139299599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6823","nonce":4139299599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6829","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6831","nonce":4139299599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6827","nonce":4139299599}]},"public_addr":"172.21.15.123:6819/4139299599","cluster_addr":"172.21.15.123:6823/4139299599","heartbeat_back_addr":"172.21.15.123:6831/4139299599","heartbeat_front_addr":"172.21.15.123:6827/4139299599","state":["exists","up"]},{"osd":7,"uuid":"a00aac6e-4bd9-415a-a93d-ae85210865ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6809","nonce":4043981069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6811","nonce":4043981069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6815","nonce":4043981069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6813","nonce":4043981069}]},"public_addr":"172.21.15.123:6809/4043981069","cluster_addr":"172.21.15.123:6811/4043981069","heartbeat_back_addr":"172.21.15.123:6815/4043981069","heartbeat_front_addr":"172.21.15.123:6813/4043981069","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.904540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.958585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.616367+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.885244+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.103127+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.167248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.884348+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.862356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:20:57.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.792+0000 7feb0ae00640 1 -- 172.21.15.121:0/3464037107 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7feae406c8f0 msgr2=0x7feae406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:57.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.792+0000 7feb0ae00640 1 --2- 172.21.15.121:0/3464037107 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7feae406c8f0 0x7feae406edb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7feaf0036000 tx=0x7feaf003c040 comp rx=0 tx=0).stop 2024-04-20T10:20:57.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.792+0000 7feb0ae00640 1 -- 172.21.15.121:0/3464037107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb0406c6d0 msgr2=0x7feb040cb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:57.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.792+0000 7feb0ae00640 1 --2- 172.21.15.121:0/3464037107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb0406c6d0 0x7feb040cb520 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7feaf800e840 tx=0x7feaf800ed00 comp rx=0 tx=0).stop 2024-04-20T10:20:57.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.792+0000 7feb0ae00640 1 -- 172.21.15.121:0/3464037107 shutdown_connections 2024-04-20T10:20:57.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.792+0000 7feb0ae00640 1 --2- 172.21.15.121:0/3464037107 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb04161d50 0x7feb040cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:57.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.792+0000 7feb0ae00640 1 --2- 172.21.15.121:0/3464037107 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7feae406c8f0 0x7feae406edb0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:57.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.792+0000 7feb0ae00640 1 --2- 172.21.15.121:0/3464037107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb040b3330 0x7feb040ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:57.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.792+0000 7feb0ae00640 1 --2- 172.21.15.121:0/3464037107 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb0406c6d0 0x7feb040cb520 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:57.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.792+0000 7feb0ae00640 1 -- 172.21.15.121:0/3464037107 >> 172.21.15.121:0/3464037107 conn(0x7feb040d4450 msgr2=0x7feb040b3730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:20:57.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.792+0000 7feb0ae00640 1 -- 172.21.15.121:0/3464037107 shutdown_connections 2024-04-20T10:20:57.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:57.792+0000 7feb0ae00640 1 -- 172.21.15.121:0/3464037107 wait complete. 2024-04-20T10:20:57.809 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:20:51.141365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T10:20:57.809 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:20:58.005 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.004+0000 7f1b56c00640 1 Processor -- start 2024-04-20T10:20:58.005 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.004+0000 7f1b56c00640 1 -- start start 2024-04-20T10:20:58.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.006+0000 7f1b56c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1b5006eb40 0x7f1b50162850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:58.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.006+0000 7f1b56c00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1b5006efd0 0x7f1b50162d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:58.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.006+0000 7f1b56c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b500d09b0 con 0x7f1b5015e8c0 2024-04-20T10:20:58.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.006+0000 7f1b56c00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f1b500cfe30 con 0x7f1b5006eb40 2024-04-20T10:20:58.007 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.006+0000 7f1b56c00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f1b500d0110 con 0x7f1b5006efd0 2024-04-20T10:20:58.007 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.006+0000 7f1b47e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1b5006eb40 0x7f1b50162850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:58.007 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.006+0000 7f1b47e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1b5006eb40 0x7f1b50162850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44118/0 (socket says 172.21.15.121:44118) 2024-04-20T10:20:58.007 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.006+0000 7f1b47e00640 1 -- 172.21.15.121:0/4118089187 learned_addr learned my addr 172.21.15.121:0/4118089187 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:20:58.008 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.006+0000 7f1b54c00640 1 --2- 172.21.15.121:0/4118089187 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1b5006efd0 0x7f1b50162d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:58.008 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.006+0000 7f1b47e00640 1 -- 172.21.15.121:0/4118089187 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1b5006efd0 msgr2=0x7f1b50162d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:20:58.008 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.006+0000 7f1b47e00640 1 --2- 172.21.15.121:0/4118089187 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1b5006efd0 0x7f1b50162d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:58.008 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.006+0000 7f1b47e00640 1 -- 172.21.15.121:0/4118089187 >> v1:172.21.15.123:6789/0 conn(0x7f1b5015e8c0 legacy=0x7f1b500ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:20:58.008 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.006+0000 7f1b47e00640 1 -- 172.21.15.121:0/4118089187 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b500d03f0 con 0x7f1b5006eb40 2024-04-20T10:20:58.008 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.007+0000 7f1b47e00640 1 --2- 172.21.15.121:0/4118089187 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b5006eb40 0x7f1b50162850 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1b38009870 tx=0x7f1b3802b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df52e59379744cb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:58.009 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.008+0000 7f1b4f400640 1 -- 172.21.15.121:0/4118089187 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b38038070 con 0x7f1b5006eb40 2024-04-20T10:20:58.009 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.008+0000 7f1b4f400640 1 -- 172.21.15.121:0/4118089187 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b38031740 con 0x7f1b5006eb40 2024-04-20T10:20:58.009 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.008+0000 7f1b4f400640 1 -- 172.21.15.121:0/4118089187 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3802f050 con 0x7f1b5006eb40 2024-04-20T10:20:58.009 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.008+0000 7f1b56c00640 1 -- 172.21.15.121:0/4118089187 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b5006eb40 msgr2=0x7f1b50162850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:58.010 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.008+0000 7f1b56c00640 1 --2- 172.21.15.121:0/4118089187 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b5006eb40 0x7f1b50162850 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1b38009870 tx=0x7f1b3802b7e0 comp rx=0 tx=0).stop 2024-04-20T10:20:58.010 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.008+0000 7f1b56c00640 1 -- 172.21.15.121:0/4118089187 shutdown_connections 2024-04-20T10:20:58.010 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.008+0000 7f1b56c00640 1 --2- 172.21.15.121:0/4118089187 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1b5006efd0 0x7f1b50162d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:58.010 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.008+0000 7f1b56c00640 1 --2- 172.21.15.121:0/4118089187 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b5006eb40 0x7f1b50162850 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:58.010 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.008+0000 7f1b56c00640 1 -- 172.21.15.121:0/4118089187 >> 172.21.15.121:0/4118089187 conn(0x7f1b5009d910 msgr2=0x7f1b500b94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:20:58.010 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.009+0000 7f1b56c00640 1 -- 172.21.15.121:0/4118089187 shutdown_connections 2024-04-20T10:20:58.010 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.009+0000 7f1b56c00640 1 -- 172.21.15.121:0/4118089187 wait complete. 2024-04-20T10:20:58.011 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.010+0000 7f1b56c00640 1 Processor -- start 2024-04-20T10:20:58.011 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.010+0000 7f1b56c00640 1 -- start start 2024-04-20T10:20:58.013 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.011+0000 7f1b56c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b5006eb40 0x7f1b50154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:58.013 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.011+0000 7f1b56c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b5006efd0 0x7f1b500c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:58.013 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.011+0000 7f1b56c00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1b5015e8c0 0x7f1b500b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:58.013 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.011+0000 7f1b56c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5015eca0 con 0x7f1b5006eb40 2024-04-20T10:20:58.013 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.011+0000 7f1b56c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1b501632d0 con 0x7f1b5006efd0 2024-04-20T10:20:58.014 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.011+0000 7f1b56c00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f1b500b6230 con 0x7f1b5015e8c0 2024-04-20T10:20:58.014 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.011+0000 7f1b4fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b5006eb40 0x7f1b50154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:58.014 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.011+0000 7f1b4fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b5006eb40 0x7f1b50154890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44126/0 (socket says 172.21.15.121:44126) 2024-04-20T10:20:58.014 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.012+0000 7f1b4fe00640 1 -- 172.21.15.121:0/1664243559 learned_addr learned my addr 172.21.15.121:0/1664243559 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:20:58.014 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.012+0000 7f1b47e00640 1 --2- 172.21.15.121:0/1664243559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b5006efd0 0x7f1b500c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:58.014 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.012+0000 7f1b54c00640 1 --2- 172.21.15.121:0/1664243559 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1b5015e8c0 0x7f1b500b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:58.014 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.012+0000 7f1b4fe00640 1 -- 172.21.15.121:0/1664243559 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1b5015e8c0 msgr2=0x7f1b500b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:58.015 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.012+0000 7f1b4fe00640 1 --2- 172.21.15.121:0/1664243559 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1b5015e8c0 0x7f1b500b47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:58.015 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.012+0000 7f1b4fe00640 1 -- 172.21.15.121:0/1664243559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b5006efd0 msgr2=0x7f1b500c9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:58.015 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.012+0000 7f1b4fe00640 1 --2- 172.21.15.121:0/1664243559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b5006efd0 0x7f1b500c9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:58.015 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.012+0000 7f1b4fe00640 1 -- 172.21.15.121:0/1664243559 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b500ca4c0 con 0x7f1b5006eb40 2024-04-20T10:20:58.015 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.012+0000 7f1b47e00640 1 --2- 172.21.15.121:0/1664243559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b5006efd0 0x7f1b500c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:20:58.015 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.012+0000 7f1b4fe00640 1 --2- 172.21.15.121:0/1664243559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b5006eb40 0x7f1b50154890 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1b4000b3f0 tx=0x7f1b4000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:58.015 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.012+0000 7f1b4cc00640 1 -- 172.21.15.121:0/1664243559 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b40015020 con 0x7f1b5006eb40 2024-04-20T10:20:58.015 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.012+0000 7f1b4cc00640 1 -- 172.21.15.121:0/1664243559 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b40011830 con 0x7f1b5006eb40 2024-04-20T10:20:58.015 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.012+0000 7f1b4cc00640 1 -- 172.21.15.121:0/1664243559 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b40011c20 con 0x7f1b5006eb40 2024-04-20T10:20:58.015 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.012+0000 7f1b56c00640 1 -- 172.21.15.121:0/1664243559 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b500ca7a0 con 0x7f1b5006eb40 2024-04-20T10:20:58.016 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.013+0000 7f1b56c00640 1 -- 172.21.15.121:0/1664243559 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b500cad60 con 0x7f1b5006eb40 2024-04-20T10:20:58.016 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.014+0000 7f1b56c00640 1 -- 172.21.15.121:0/1664243559 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b50068980 con 0x7f1b5006eb40 2024-04-20T10:20:58.017 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.015+0000 7f1b4cc00640 1 -- 172.21.15.121:0/1664243559 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b40018020 con 0x7f1b5006eb40 2024-04-20T10:20:58.017 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.015+0000 7f1b4cc00640 1 --2- 172.21.15.121:0/1664243559 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f1b2c06c8f0 0x7f1b2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:58.017 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.015+0000 7f1b4cc00640 1 -- 172.21.15.121:0/1664243559 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f1b4008b410 con 0x7f1b5006eb40 2024-04-20T10:20:58.017 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.016+0000 7f1b47e00640 1 --2- 172.21.15.121:0/1664243559 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f1b2c06c8f0 0x7f1b2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:58.017 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.016+0000 7f1b47e00640 1 --2- 172.21.15.121:0/1664243559 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f1b2c06c8f0 0x7f1b2c06edb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1b38035000 tx=0x7f1b38008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:58.021 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.021+0000 7f1b4cc00640 1 -- 172.21.15.121:0/1664243559 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1b4001f030 con 0x7f1b5006eb40 2024-04-20T10:20:58.206 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.206+0000 7f1b56c00640 1 -- 172.21.15.121:0/1664243559 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1b500d5120 con 0x7f1b5006eb40 2024-04-20T10:20:58.207 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.207+0000 7f1b4cc00640 1 -- 172.21.15.121:0/1664243559 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) ==== 93+0+10 (secure 0 0 0) 0x7f1b400172c0 con 0x7f1b5006eb40 2024-04-20T10:20:58.207 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2024-04-20T10:20:58.210 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.209+0000 7f1b56c00640 1 -- 172.21.15.121:0/1664243559 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f1b2c06c8f0 msgr2=0x7f1b2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:58.210 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.209+0000 7f1b56c00640 1 --2- 172.21.15.121:0/1664243559 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f1b2c06c8f0 0x7f1b2c06edb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1b38035000 tx=0x7f1b38008030 comp rx=0 tx=0).stop 2024-04-20T10:20:58.210 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.209+0000 7f1b56c00640 1 -- 172.21.15.121:0/1664243559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b5006eb40 msgr2=0x7f1b50154890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:58.210 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.209+0000 7f1b56c00640 1 --2- 172.21.15.121:0/1664243559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b5006eb40 0x7f1b50154890 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1b4000b3f0 tx=0x7f1b4000b8b0 comp rx=0 tx=0).stop 2024-04-20T10:20:58.210 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.209+0000 7f1b56c00640 1 -- 172.21.15.121:0/1664243559 shutdown_connections 2024-04-20T10:20:58.210 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.209+0000 7f1b56c00640 1 --2- 172.21.15.121:0/1664243559 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1b5015e8c0 0x7f1b500b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:58.210 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.209+0000 7f1b56c00640 1 --2- 172.21.15.121:0/1664243559 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f1b2c06c8f0 0x7f1b2c06edb0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:58.210 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.209+0000 7f1b56c00640 1 --2- 172.21.15.121:0/1664243559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b5006efd0 0x7f1b500c9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:58.210 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.209+0000 7f1b56c00640 1 --2- 172.21.15.121:0/1664243559 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1b5006eb40 0x7f1b50154890 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:58.210 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.209+0000 7f1b56c00640 1 -- 172.21.15.121:0/1664243559 >> 172.21.15.121:0/1664243559 conn(0x7f1b5009d910 msgr2=0x7f1b500b94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:20:58.211 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.209+0000 7f1b56c00640 1 -- 172.21.15.121:0/1664243559 shutdown_connections 2024-04-20T10:20:58.211 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.209+0000 7f1b56c00640 1 -- 172.21.15.121:0/1664243559 wait complete. 2024-04-20T10:20:58.228 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:20:58.412 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.412+0000 7f6dd1e00640 1 Processor -- start 2024-04-20T10:20:58.413 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.412+0000 7f6dd1e00640 1 -- start start 2024-04-20T10:20:58.414 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.413+0000 7f6dd1e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6dcc06cde0 0x7f6dcc0d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:58.414 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.413+0000 7f6dd1e00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dcc0b3330 0x7f6dcc0b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:58.414 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.413+0000 7f6dd1e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6dcc06c190 con 0x7f6dcc161d50 2024-04-20T10:20:58.414 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.413+0000 7f6dd1e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f6dcc162130 con 0x7f6dcc06cde0 2024-04-20T10:20:58.414 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.413+0000 7f6dd1e00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f6dcc06c470 con 0x7f6dcc0b3330 2024-04-20T10:20:58.415 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.414+0000 7f6dd1400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dcc0b3330 0x7f6dcc0b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:58.415 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.414+0000 7f6dd1400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dcc0b3330 0x7f6dcc0b3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.121:47532/0 (socket says 172.21.15.121:47532) 2024-04-20T10:20:58.415 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.414+0000 7f6dd1400640 1 -- 172.21.15.121:0/312354932 learned_addr learned my addr 172.21.15.121:0/312354932 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:20:58.415 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.414+0000 7f6dcbe00640 1 --2- 172.21.15.121:0/312354932 >> v2:172.21.15.123:3300/0 conn(0x7f6dcc06cde0 0x7f6dcc0d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:58.415 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.415+0000 7f6dd1400640 1 -- 172.21.15.121:0/312354932 >> v2:172.21.15.123:3300/0 conn(0x7f6dcc06cde0 msgr2=0x7f6dcc0d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:20:58.415 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.415+0000 7f6dd1400640 1 --2- 172.21.15.121:0/312354932 >> v2:172.21.15.123:3300/0 conn(0x7f6dcc06cde0 0x7f6dcc0d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:58.416 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.415+0000 7f6dd1400640 1 -- 172.21.15.121:0/312354932 >> v1:172.21.15.123:6789/0 conn(0x7f6dcc161d50 legacy=0x7f6dcc06c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:58.416 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.415+0000 7f6dd1400640 1 -- 172.21.15.121:0/312354932 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dcc0d2e00 con 0x7f6dcc0b3330 2024-04-20T10:20:58.416 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.416+0000 7f6dd1400640 1 --2- 172.21.15.121:0/312354932 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dcc0b3330 0x7f6dcc0b3880 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4009e50 tx=0x7f6dc402bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d97a4a1a715e36e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:58.417 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.417+0000 7f6dcb400640 1 -- 172.21.15.121:0/312354932 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc402cb00 con 0x7f6dcc0b3330 2024-04-20T10:20:58.417 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.417+0000 7f6dcb400640 1 -- 172.21.15.121:0/312354932 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dc402cca0 con 0x7f6dcc0b3330 2024-04-20T10:20:58.417 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.417+0000 7f6dcb400640 1 -- 172.21.15.121:0/312354932 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4034610 con 0x7f6dcc0b3330 2024-04-20T10:20:58.418 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.418+0000 7f6dd1e00640 1 -- 172.21.15.121:0/312354932 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dcc0b3330 msgr2=0x7f6dcc0b3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:58.418 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.418+0000 7f6dd1e00640 1 --2- 172.21.15.121:0/312354932 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dcc0b3330 0x7f6dcc0b3880 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4009e50 tx=0x7f6dc402bf60 comp rx=0 tx=0).stop 2024-04-20T10:20:58.418 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.418+0000 7f6dd1e00640 1 -- 172.21.15.121:0/312354932 shutdown_connections 2024-04-20T10:20:58.418 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.418+0000 7f6dd1e00640 1 --2- 172.21.15.121:0/312354932 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dcc0b3330 0x7f6dcc0b3880 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:58.418 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.418+0000 7f6dd1e00640 1 --2- 172.21.15.121:0/312354932 >> v2:172.21.15.123:3300/0 conn(0x7f6dcc06cde0 0x7f6dcc0d38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:58.418 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.418+0000 7f6dd1e00640 1 -- 172.21.15.121:0/312354932 >> 172.21.15.121:0/312354932 conn(0x7f6dcc0d4450 msgr2=0x7f6dcc0d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:20:58.419 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.418+0000 7f6dd1e00640 1 -- 172.21.15.121:0/312354932 shutdown_connections 2024-04-20T10:20:58.419 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.418+0000 7f6dd1e00640 1 -- 172.21.15.121:0/312354932 wait complete. 2024-04-20T10:20:58.420 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.420+0000 7f6dd1e00640 1 Processor -- start 2024-04-20T10:20:58.420 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.420+0000 7f6dd1e00640 1 -- start start 2024-04-20T10:20:58.423 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.421+0000 7f6dd1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dcc06cde0 0x7f6dcc0c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:58.423 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.421+0000 7f6dd1e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6dcc0b3330 0x7f6dcc1f7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:58.423 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.421+0000 7f6dd1e00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dcc161d50 0x7f6dcc0ca570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:58.423 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.421+0000 7f6dd1e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6dcc0b7660 con 0x7f6dcc0b3330 2024-04-20T10:20:58.423 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.421+0000 7f6dd1e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6dcc0d33c0 con 0x7f6dcc06cde0 2024-04-20T10:20:58.423 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.421+0000 7f6dd1e00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f6dcc0b77e0 con 0x7f6dcc161d50 2024-04-20T10:20:58.423 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.421+0000 7f6dcbe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6dcc0b3330 0x7f6dcc1f7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:58.423 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.421+0000 7f6dcbe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6dcc0b3330 0x7f6dcc1f7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44134/0 (socket says 172.21.15.121:44134) 2024-04-20T10:20:58.423 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.421+0000 7f6dcbe00640 1 -- 172.21.15.121:0/1120679486 learned_addr learned my addr 172.21.15.121:0/1120679486 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:20:58.423 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.422+0000 7f6dd1400640 1 --2- 172.21.15.121:0/1120679486 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dcc161d50 0x7f6dcc0ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:58.423 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.422+0000 7f6dcbe00640 1 -- 172.21.15.121:0/1120679486 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dcc161d50 msgr2=0x7f6dcc0ca570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:58.424 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.422+0000 7f6dcbe00640 1 --2- 172.21.15.121:0/1120679486 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dcc161d50 0x7f6dcc0ca570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:58.424 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.422+0000 7f6dd0a00640 1 --2- 172.21.15.121:0/1120679486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dcc06cde0 0x7f6dcc0c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:58.424 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.422+0000 7f6dcbe00640 1 -- 172.21.15.121:0/1120679486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dcc06cde0 msgr2=0x7f6dcc0c8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:58.424 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.422+0000 7f6dcbe00640 1 --2- 172.21.15.121:0/1120679486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dcc06cde0 0x7f6dcc0c8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:58.424 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.422+0000 7f6dcbe00640 1 -- 172.21.15.121:0/1120679486 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dcc1f7f30 con 0x7f6dcc0b3330 2024-04-20T10:20:58.424 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.422+0000 7f6dcbe00640 1 --2- 172.21.15.121:0/1120679486 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6dcc0b3330 0x7f6dcc1f7630 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6db800d800 tx=0x7f6db800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:58.424 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.422+0000 7f6dc8c00640 1 -- 172.21.15.121:0/1120679486 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db800b6c0 con 0x7f6dcc0b3330 2024-04-20T10:20:58.424 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.422+0000 7f6dc8c00640 1 -- 172.21.15.121:0/1120679486 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6db8019070 con 0x7f6dcc0b3330 2024-04-20T10:20:58.424 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.422+0000 7f6dd1e00640 1 -- 172.21.15.121:0/1120679486 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dcc1f8210 con 0x7f6dcc0b3330 2024-04-20T10:20:58.424 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.422+0000 7f6dc8c00640 1 -- 172.21.15.121:0/1120679486 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db800ba80 con 0x7f6dcc0b3330 2024-04-20T10:20:58.424 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.422+0000 7f6dd1e00640 1 -- 172.21.15.121:0/1120679486 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dcc1f87d0 con 0x7f6dcc0b3330 2024-04-20T10:20:58.425 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.423+0000 7f6dd1e00640 1 -- 172.21.15.121:0/1120679486 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dcc075b60 con 0x7f6dcc0b3330 2024-04-20T10:20:58.425 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.424+0000 7f6dc8c00640 1 -- 172.21.15.121:0/1120679486 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6db800bc20 con 0x7f6dcc0b3330 2024-04-20T10:20:58.426 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.426+0000 7f6dc8c00640 1 --2- 172.21.15.121:0/1120679486 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f6dac06c8f0 0x7f6dac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:58.426 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.426+0000 7f6dc8c00640 1 -- 172.21.15.121:0/1120679486 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f6db808bd90 con 0x7f6dcc0b3330 2024-04-20T10:20:58.426 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.426+0000 7f6dd0a00640 1 --2- 172.21.15.121:0/1120679486 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f6dac06c8f0 0x7f6dac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:58.427 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.426+0000 7f6dd0a00640 1 --2- 172.21.15.121:0/1120679486 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f6dac06c8f0 0x7f6dac06edb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc0cafa0 tx=0x7f6dc0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:58.430 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.430+0000 7f6dc8c00640 1 -- 172.21.15.121:0/1120679486 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6db801c360 con 0x7f6dcc0b3330 2024-04-20T10:20:58.653 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.652+0000 7f6dd1e00640 1 -- 172.21.15.121:0/1120679486 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6dcc0ceaa0 con 0x7f6dcc0b3330 2024-04-20T10:20:58.654 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.654+0000 7f6dc8c00640 1 -- 172.21.15.121:0/1120679486 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3873 (secure 0 0 0) 0x7f6db805a470 con 0x7f6dcc0b3330 2024-04-20T10:20:58.654 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:20:58.655 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1376743826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1376743826},{"type":"v1","addr":"172.21.15.123:6833","nonce":1376743826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/999167058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":999167058},{"type":"v1","addr":"172.21.15.121:6835","nonce":999167058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2160736935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2160736935},{"type":"v1","addr":"172.21.15.121:6837","nonce":2160736935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4281,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3336775250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3336775250},{"type":"v1","addr":"172.21.15.123:6835","nonce":3336775250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-20T10:20:58.655 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 3 2024-04-20T10:20:58.657 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.656+0000 7f6dd1e00640 1 -- 172.21.15.121:0/1120679486 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f6dac06c8f0 msgr2=0x7f6dac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:58.657 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.656+0000 7f6dd1e00640 1 --2- 172.21.15.121:0/1120679486 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f6dac06c8f0 0x7f6dac06edb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc0cafa0 tx=0x7f6dc0009040 comp rx=0 tx=0).stop 2024-04-20T10:20:58.657 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.656+0000 7f6dd1e00640 1 -- 172.21.15.121:0/1120679486 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6dcc0b3330 msgr2=0x7f6dcc1f7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:58.657 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.656+0000 7f6dd1e00640 1 --2- 172.21.15.121:0/1120679486 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6dcc0b3330 0x7f6dcc1f7630 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6db800d800 tx=0x7f6db800dcc0 comp rx=0 tx=0).stop 2024-04-20T10:20:58.657 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.656+0000 7f6dd1e00640 1 -- 172.21.15.121:0/1120679486 shutdown_connections 2024-04-20T10:20:58.657 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.656+0000 7f6dd1e00640 1 --2- 172.21.15.121:0/1120679486 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dcc161d50 0x7f6dcc0ca570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:58.657 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.656+0000 7f6dd1e00640 1 --2- 172.21.15.121:0/1120679486 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6dcc0b3330 0x7f6dcc1f7630 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:58.657 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.656+0000 7f6dd1e00640 1 --2- 172.21.15.121:0/1120679486 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f6dac06c8f0 0x7f6dac06edb0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:58.657 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.656+0000 7f6dd1e00640 1 --2- 172.21.15.121:0/1120679486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dcc06cde0 0x7f6dcc0c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:58.658 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.656+0000 7f6dd1e00640 1 -- 172.21.15.121:0/1120679486 >> 172.21.15.121:0/1120679486 conn(0x7f6dcc0d4450 msgr2=0x7f6dcc160be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:20:58.658 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.656+0000 7f6dd1e00640 1 -- 172.21.15.121:0/1120679486 shutdown_connections 2024-04-20T10:20:58.658 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.656+0000 7f6dd1e00640 1 -- 172.21.15.121:0/1120679486 wait complete. 2024-04-20T10:20:58.675 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:20:58.867 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.867+0000 7fa39b200640 1 Processor -- start 2024-04-20T10:20:58.868 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.867+0000 7fa39b200640 1 -- start start 2024-04-20T10:20:58.869 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.869+0000 7fa39b200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa39c15e550 0x7fa39c0d06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:58.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.869+0000 7fa39b200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa39c0cf950 con 0x7fa39c1628e0 2024-04-20T10:20:58.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.869+0000 7fa39b200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa39c162e90 con 0x7fa39c0d4380 2024-04-20T10:20:58.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.869+0000 7fa39b200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fa39c0cfbf0 con 0x7fa39c15e550 2024-04-20T10:20:58.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.869+0000 7fa39a800640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fa39c1628e0 0x7fa39c0d1fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:58816/0 (socket says 172.21.15.121:58816) 2024-04-20T10:20:58.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.869+0000 7fa39a800640 1 -- 172.21.15.121:0/2750066993 learned_addr learned my addr 172.21.15.121:0/2750066993 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:20:58.872 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.870+0000 7fa399e00640 1 --2- 172.21.15.121:0/2750066993 >> v2:172.21.15.123:3300/0 conn(0x7fa39c15e550 0x7fa39c0d06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:58.872 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.870+0000 7fa398a00640 1 -- 172.21.15.121:0/2750066993 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2900177946 0 0) 0x7fa39c0cf950 con 0x7fa39c1628e0 2024-04-20T10:20:58.872 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.870+0000 7fa398a00640 1 -- 172.21.15.121:0/2750066993 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa380003520 con 0x7fa39c1628e0 2024-04-20T10:20:58.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.870+0000 7fa398a00640 1 -- 172.21.15.121:0/2750066993 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 251181349 0 0) 0x7fa380003520 con 0x7fa39c1628e0 2024-04-20T10:20:58.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.870+0000 7fa398a00640 1 -- 172.21.15.121:0/2750066993 >> v2:172.21.15.123:3300/0 conn(0x7fa39c15e550 msgr2=0x7fa39c0d06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:20:58.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.870+0000 7fa398a00640 1 --2- 172.21.15.121:0/2750066993 >> v2:172.21.15.123:3300/0 conn(0x7fa39c15e550 0x7fa39c0d06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:58.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.870+0000 7fa398a00640 1 -- 172.21.15.121:0/2750066993 >> v1:172.21.15.123:6789/0 conn(0x7fa39c0d4380 legacy=0x7fa39c1621d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:58.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.870+0000 7fa398a00640 1 -- 172.21.15.121:0/2750066993 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa39c0cfed0 con 0x7fa39c1628e0 2024-04-20T10:20:58.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.870+0000 7fa398a00640 1 -- 172.21.15.121:0/2750066993 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7fa394001690 con 0x7fa39c1628e0 2024-04-20T10:20:58.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.870+0000 7fa398a00640 1 -- 172.21.15.121:0/2750066993 >> v1:172.21.15.121:6789/0 conn(0x7fa39c1628e0 legacy=0x7fa39c0d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:58.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.870+0000 7fa398a00640 1 --2- 172.21.15.121:0/2750066993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3800039b0 0x7fa380003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:58.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.870+0000 7fa398a00640 1 --2- 172.21.15.121:0/2750066993 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa380004800 0x7fa380004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:58.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.870+0000 7fa398a00640 1 --2- 172.21.15.121:0/2750066993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3800051c0 0x7fa38000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:58.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.870+0000 7fa398a00640 1 -- 172.21.15.121:0/2750066993 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa39c0cfbf0 con 0x7fa3800039b0 2024-04-20T10:20:58.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.870+0000 7fa398a00640 1 -- 172.21.15.121:0/2750066993 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa38000c160 con 0x7fa3800051c0 2024-04-20T10:20:58.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.870+0000 7fa398a00640 1 -- 172.21.15.121:0/2750066993 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fa38000c360 con 0x7fa380004800 2024-04-20T10:20:58.874 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.871+0000 7fa399e00640 1 --2- 172.21.15.121:0/2750066993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3800039b0 0x7fa380003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:58.874 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.871+0000 7fa399400640 1 --2- 172.21.15.121:0/2750066993 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa380004800 0x7fa380004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:58.874 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.871+0000 7fa399e00640 1 -- 172.21.15.121:0/2750066993 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa380004800 msgr2=0x7fa380004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:20:58.874 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.871+0000 7fa399e00640 1 --2- 172.21.15.121:0/2750066993 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa380004800 0x7fa380004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:58.874 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.871+0000 7fa399e00640 1 -- 172.21.15.121:0/2750066993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3800051c0 msgr2=0x7fa38000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:20:58.874 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.871+0000 7fa39a800640 1 --2- 172.21.15.121:0/2750066993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3800051c0 0x7fa38000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:58.874 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.871+0000 7fa399e00640 1 --2- 172.21.15.121:0/2750066993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3800051c0 0x7fa38000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:58.874 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.871+0000 7fa399e00640 1 -- 172.21.15.121:0/2750066993 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa380003520 con 0x7fa3800039b0 2024-04-20T10:20:58.874 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.871+0000 7fa39a800640 1 --2- 172.21.15.121:0/2750066993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3800051c0 0x7fa38000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:20:58.874 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.871+0000 7fa39a800640 1 -- 172.21.15.121:0/2750066993 reap_dead start 2024-04-20T10:20:58.874 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.872+0000 7fa399e00640 1 --2- 172.21.15.121:0/2750066993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3800039b0 0x7fa380003dd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa38c009fc0 tx=0x7fa380005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a5a04f1aea80a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:58.875 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.872+0000 7fa398a00640 1 -- 172.21.15.121:0/2750066993 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38c039070 con 0x7fa3800039b0 2024-04-20T10:20:58.876 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.872+0000 7fa398a00640 1 -- 172.21.15.121:0/2750066993 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa38c02cc20 con 0x7fa3800039b0 2024-04-20T10:20:58.876 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.872+0000 7fa398a00640 1 -- 172.21.15.121:0/2750066993 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38c009c10 con 0x7fa3800039b0 2024-04-20T10:20:58.876 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.873+0000 7fa39b200640 1 -- 172.21.15.121:0/2750066993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3800039b0 msgr2=0x7fa380003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:58.876 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.873+0000 7fa39b200640 1 --2- 172.21.15.121:0/2750066993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3800039b0 0x7fa380003dd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa38c009fc0 tx=0x7fa380005830 comp rx=0 tx=0).stop 2024-04-20T10:20:58.876 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.873+0000 7fa39b200640 1 -- 172.21.15.121:0/2750066993 shutdown_connections 2024-04-20T10:20:58.876 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.873+0000 7fa39b200640 1 --2- 172.21.15.121:0/2750066993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa3800039b0 0x7fa380003dd0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:58.876 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.873+0000 7fa39b200640 1 -- 172.21.15.121:0/2750066993 >> 172.21.15.121:0/2750066993 conn(0x7fa39c08bf80 msgr2=0x7fa39c09d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:20:58.876 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.873+0000 7fa39b200640 1 -- 172.21.15.121:0/2750066993 shutdown_connections 2024-04-20T10:20:58.876 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.873+0000 7fa39b200640 1 -- 172.21.15.121:0/2750066993 wait complete. 2024-04-20T10:20:58.876 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.875+0000 7fa39b200640 1 Processor -- start 2024-04-20T10:20:58.876 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.875+0000 7fa39b200640 1 -- start start 2024-04-20T10:20:58.878 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.876+0000 7fa39b200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa3800039b0 0x7fa39c157850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:58.878 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.876+0000 7fa39b200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa39c0b3e50 0x7fa39c0b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:58.878 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.876+0000 7fa39b200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa39c160d90 0x7fa39c0b61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:58.878 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.876+0000 7fa39b200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa39c0d4810 con 0x7fa39c160d90 2024-04-20T10:20:58.878 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.876+0000 7fa39b200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa39c0d4990 con 0x7fa39c0b3e50 2024-04-20T10:20:58.878 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.876+0000 7fa39b200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fa39c0d4b10 con 0x7fa3800039b0 2024-04-20T10:20:58.879 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.877+0000 7fa39a800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa39c160d90 0x7fa39c0b61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:58.879 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.877+0000 7fa39a800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa39c160d90 0x7fa39c0b61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44148/0 (socket says 172.21.15.121:44148) 2024-04-20T10:20:58.879 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.877+0000 7fa39a800640 1 -- 172.21.15.121:0/2703356809 learned_addr learned my addr 172.21.15.121:0/2703356809 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:20:58.879 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.877+0000 7fa399e00640 1 --2- 172.21.15.121:0/2703356809 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa3800039b0 0x7fa39c157850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:58.879 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.877+0000 7fa399400640 1 --2- 172.21.15.121:0/2703356809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa39c0b3e50 0x7fa39c0b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:58.879 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.877+0000 7fa39a800640 1 -- 172.21.15.121:0/2703356809 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa3800039b0 msgr2=0x7fa39c157850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:58.879 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.877+0000 7fa39a800640 1 --2- 172.21.15.121:0/2703356809 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa3800039b0 0x7fa39c157850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:58.879 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.877+0000 7fa39a800640 1 -- 172.21.15.121:0/2703356809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa39c0b3e50 msgr2=0x7fa39c0b53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:58.879 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.877+0000 7fa39a800640 1 --2- 172.21.15.121:0/2703356809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa39c0b3e50 0x7fa39c0b53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:58.879 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.877+0000 7fa39a800640 1 -- 172.21.15.121:0/2703356809 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa39c0b4350 con 0x7fa39c160d90 2024-04-20T10:20:58.879 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.877+0000 7fa39a800640 1 --2- 172.21.15.121:0/2703356809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa39c160d90 0x7fa39c0b61f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa39c0cfb20 tx=0x7fa39400d870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:58.879 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.878+0000 7fa392000640 1 -- 172.21.15.121:0/2703356809 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa394007440 con 0x7fa39c160d90 2024-04-20T10:20:58.879 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.878+0000 7fa392000640 1 -- 172.21.15.121:0/2703356809 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3940075e0 con 0x7fa39c160d90 2024-04-20T10:20:58.879 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.878+0000 7fa392000640 1 -- 172.21.15.121:0/2703356809 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3940049e0 con 0x7fa39c160d90 2024-04-20T10:20:58.880 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.878+0000 7fa39b200640 1 -- 172.21.15.121:0/2703356809 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa39c0b4630 con 0x7fa39c160d90 2024-04-20T10:20:58.880 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.878+0000 7fa39b200640 1 -- 172.21.15.121:0/2703356809 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa39c0b4bf0 con 0x7fa39c160d90 2024-04-20T10:20:58.880 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.879+0000 7fa39b200640 1 -- 172.21.15.121:0/2703356809 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa39c075b30 con 0x7fa39c160d90 2024-04-20T10:20:58.880 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.880+0000 7fa392000640 1 -- 172.21.15.121:0/2703356809 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa394004b80 con 0x7fa39c160d90 2024-04-20T10:20:58.881 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.881+0000 7fa392000640 1 --2- 172.21.15.121:0/2703356809 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fa37806c8f0 0x7fa37806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:58.881 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.881+0000 7fa392000640 1 -- 172.21.15.121:0/2703356809 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fa39408aea0 con 0x7fa39c160d90 2024-04-20T10:20:58.881 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.881+0000 7fa399e00640 1 --2- 172.21.15.121:0/2703356809 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fa37806c8f0 0x7fa37806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:58.882 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.882+0000 7fa399e00640 1 --2- 172.21.15.121:0/2703356809 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fa37806c8f0 0x7fa37806edb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa38c036000 tx=0x7fa38c02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:58.886 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:58.886+0000 7fa392000640 1 -- 172.21.15.121:0/2703356809 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa394059580 con 0x7fa39c160d90 2024-04-20T10:20:59.087 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.087+0000 7fa39b200640 1 -- 172.21.15.121:0/2703356809 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa39c0b9730 con 0x7fa39c160d90 2024-04-20T10:20:59.088 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.088+0000 7fa392000640 1 -- 172.21.15.121:0/2703356809 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12920 (secure 0 0 0) 0x7fa394058e80 con 0x7fa39c160d90 2024-04-20T10:20:59.088 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:20:59.089 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":17,"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","created":"2024-04-20T10:20:38.076076+0000","modified":"2024-04-20T10:20:53.276076+0000","last_up_change":"2024-04-20T10:20:50.162700+0000","last_in_change":"2024-04-20T10:20:41.510382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:20:51.141365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"629a7b8c-a40a-4a2a-a3cc-d9f357774310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6809","nonce":1044588835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6811","nonce":1044588835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6815","nonce":1044588835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6813","nonce":1044588835}]},"public_addr":"172.21.15.121:6809/1044588835","cluster_addr":"172.21.15.121:6811/1044588835","heartbeat_back_addr":"172.21.15.121:6815/1044588835","heartbeat_front_addr":"172.21.15.121:6813/1044588835","state":["exists","up"]},{"osd":1,"uuid":"f2570399-aa50-4ff3-bf5d-3d09207290db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6801","nonce":3505199624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6803","nonce":3505199624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6807","nonce":3505199624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6805","nonce":3505199624}]},"public_addr":"172.21.15.121:6801/3505199624","cluster_addr":"172.21.15.121:6803/3505199624","heartbeat_back_addr":"172.21.15.121:6807/3505199624","heartbeat_front_addr":"172.21.15.121:6805/3505199624","state":["exists","up"]},{"osd":2,"uuid":"50893807-dc30-49f4-b41f-739366626658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6817","nonce":3656724207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6819","nonce":3656724207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6827","nonce":3656724207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6822","nonce":3656724207}]},"public_addr":"172.21.15.121:6817/3656724207","cluster_addr":"172.21.15.121:6819/3656724207","heartbeat_back_addr":"172.21.15.121:6827/3656724207","heartbeat_front_addr":"172.21.15.121:6822/3656724207","state":["exists","up"]},{"osd":3,"uuid":"dadb01bc-5491-4ffc-8824-d48a057240b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6823","nonce":2232342832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6826","nonce":2232342832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6831","nonce":2232342832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6829","nonce":2232342832}]},"public_addr":"172.21.15.121:6823/2232342832","cluster_addr":"172.21.15.121:6826/2232342832","heartbeat_back_addr":"172.21.15.121:6831/2232342832","heartbeat_front_addr":"172.21.15.121:6829/2232342832","state":["exists","up"]},{"osd":4,"uuid":"b3f3b4a1-b375-4526-b69e-344ab24c2da1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6801","nonce":99353111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6803","nonce":99353111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6807","nonce":99353111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6805","nonce":99353111}]},"public_addr":"172.21.15.123:6801/99353111","cluster_addr":"172.21.15.123:6803/99353111","heartbeat_back_addr":"172.21.15.123:6807/99353111","heartbeat_front_addr":"172.21.15.123:6805/99353111","state":["exists","up"]},{"osd":5,"uuid":"2ee24a08-1624-4961-9642-d7b510282484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6818","nonce":2504374688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6822","nonce":2504374688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6830","nonce":2504374688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6826","nonce":2504374688}]},"public_addr":"172.21.15.123:6818/2504374688","cluster_addr":"172.21.15.123:6822/2504374688","heartbeat_back_addr":"172.21.15.123:6830/2504374688","heartbeat_front_addr":"172.21.15.123:6826/2504374688","state":["exists","up"]},{"osd":6,"uuid":"186118cc-6128-49de-acca-55260945c1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6819","nonce":4139299599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6823","nonce":4139299599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6829","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6831","nonce":4139299599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6827","nonce":4139299599}]},"public_addr":"172.21.15.123:6819/4139299599","cluster_addr":"172.21.15.123:6823/4139299599","heartbeat_back_addr":"172.21.15.123:6831/4139299599","heartbeat_front_addr":"172.21.15.123:6827/4139299599","state":["exists","up"]},{"osd":7,"uuid":"a00aac6e-4bd9-415a-a93d-ae85210865ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6809","nonce":4043981069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6811","nonce":4043981069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6815","nonce":4043981069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6813","nonce":4043981069}]},"public_addr":"172.21.15.123:6809/4043981069","cluster_addr":"172.21.15.123:6811/4043981069","heartbeat_back_addr":"172.21.15.123:6815/4043981069","heartbeat_front_addr":"172.21.15.123:6813/4043981069","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.904540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.958585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.616367+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.885244+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.103127+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.167248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.884348+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.862356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:20:59.091 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.090+0000 7fa39b200640 1 -- 172.21.15.121:0/2703356809 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fa37806c8f0 msgr2=0x7fa37806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:59.091 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.090+0000 7fa39b200640 1 --2- 172.21.15.121:0/2703356809 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fa37806c8f0 0x7fa37806edb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa38c036000 tx=0x7fa38c02e000 comp rx=0 tx=0).stop 2024-04-20T10:20:59.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.090+0000 7fa39b200640 1 -- 172.21.15.121:0/2703356809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa39c160d90 msgr2=0x7fa39c0b61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:59.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.090+0000 7fa39b200640 1 --2- 172.21.15.121:0/2703356809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa39c160d90 0x7fa39c0b61f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa39c0cfb20 tx=0x7fa39400d870 comp rx=0 tx=0).stop 2024-04-20T10:20:59.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.091+0000 7fa39b200640 1 -- 172.21.15.121:0/2703356809 shutdown_connections 2024-04-20T10:20:59.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.091+0000 7fa39b200640 1 --2- 172.21.15.121:0/2703356809 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa39c160d90 0x7fa39c0b61f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:59.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.091+0000 7fa39b200640 1 --2- 172.21.15.121:0/2703356809 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fa37806c8f0 0x7fa37806edb0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:59.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.091+0000 7fa39b200640 1 --2- 172.21.15.121:0/2703356809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa39c0b3e50 0x7fa39c0b53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:59.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.091+0000 7fa39b200640 1 --2- 172.21.15.121:0/2703356809 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa3800039b0 0x7fa39c157850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:59.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.091+0000 7fa39b200640 1 -- 172.21.15.121:0/2703356809 >> 172.21.15.121:0/2703356809 conn(0x7fa39c08bf80 msgr2=0x7fa39c06db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:20:59.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.091+0000 7fa39b200640 1 -- 172.21.15.121:0/2703356809 shutdown_connections 2024-04-20T10:20:59.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.091+0000 7fa39b200640 1 -- 172.21.15.121:0/2703356809 wait complete. 2024-04-20T10:20:59.111 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:20:51.141365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T10:20:59.111 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:20:59.318 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.317+0000 7f7188400640 1 Processor -- start 2024-04-20T10:20:59.318 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.318+0000 7f7188400640 1 -- start start 2024-04-20T10:20:59.321 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.319+0000 7f7188400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f718015ca40 0x7f71800ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:59.321 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.319+0000 7f7188400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7180141c60 con 0x7f7180156c20 2024-04-20T10:20:59.322 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.319+0000 7f7188400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f718015cf90 con 0x7f71800ad8e0 2024-04-20T10:20:59.322 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.319+0000 7f7188400640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f718006c010 con 0x7f718015ca40 2024-04-20T10:20:59.322 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.319+0000 7f7185a00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f7180156c20 0x7f718006b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:58826/0 (socket says 172.21.15.121:58826) 2024-04-20T10:20:59.322 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.319+0000 7f7185a00640 1 -- 172.21.15.121:0/2170875857 learned_addr learned my addr 172.21.15.121:0/2170875857 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:20:59.322 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.320+0000 7f7185000640 1 --2- 172.21.15.121:0/2170875857 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f718015ca40 0x7f71800ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:59.322 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.320+0000 7f717fe00640 1 -- 172.21.15.121:0/2170875857 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 419187539 0 0) 0x7f7180141c60 con 0x7f7180156c20 2024-04-20T10:20:59.322 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.320+0000 7f717fe00640 1 -- 172.21.15.121:0/2170875857 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7168003520 con 0x7f7180156c20 2024-04-20T10:20:59.322 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.320+0000 7f717fe00640 1 -- 172.21.15.121:0/2170875857 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1375844263 0 0) 0x7f7168003520 con 0x7f7180156c20 2024-04-20T10:20:59.322 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.320+0000 7f717fe00640 1 -- 172.21.15.121:0/2170875857 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f718015ca40 msgr2=0x7f71800ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:20:59.322 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.320+0000 7f717fe00640 1 --2- 172.21.15.121:0/2170875857 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f718015ca40 0x7f71800ad3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:59.322 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.320+0000 7f717fe00640 1 -- 172.21.15.121:0/2170875857 >> v1:172.21.15.123:6789/0 conn(0x7f71800ad8e0 legacy=0x7f71800d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:59.322 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.320+0000 7f717fe00640 1 -- 172.21.15.121:0/2170875857 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f718015d270 con 0x7f7180156c20 2024-04-20T10:20:59.322 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.320+0000 7f717fe00640 1 -- 172.21.15.121:0/2170875857 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f7174001670 con 0x7f7180156c20 2024-04-20T10:20:59.323 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.320+0000 7f717fe00640 1 -- 172.21.15.121:0/2170875857 >> v1:172.21.15.121:6789/0 conn(0x7f7180156c20 legacy=0x7f718006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:59.323 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.320+0000 7f717fe00640 1 --2- 172.21.15.121:0/2170875857 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f718015ca40 0x7f71680039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:59.323 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.320+0000 7f717fe00640 1 --2- 172.21.15.121:0/2170875857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7168004010 0x7f7168004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:59.323 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.320+0000 7f717fe00640 1 --2- 172.21.15.121:0/2170875857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7168004d40 0x7f71680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:59.323 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.320+0000 7f717fe00640 1 -- 172.21.15.121:0/2170875857 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f718006c010 con 0x7f7168004d40 2024-04-20T10:20:59.323 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.320+0000 7f717fe00640 1 -- 172.21.15.121:0/2170875857 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f716800bb80 con 0x7f7168004010 2024-04-20T10:20:59.323 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.320+0000 7f717fe00640 1 -- 172.21.15.121:0/2170875857 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f716800bd80 con 0x7f718015ca40 2024-04-20T10:20:59.323 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.321+0000 7f7186400640 1 --2- 172.21.15.121:0/2170875857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7168004d40 0x7f71680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:59.323 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.321+0000 7f7185a00640 1 --2- 172.21.15.121:0/2170875857 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f718015ca40 0x7f71680039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:59.323 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.321+0000 7f7186400640 1 -- 172.21.15.121:0/2170875857 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f718015ca40 msgr2=0x7f71680039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:20:59.323 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.321+0000 7f7186400640 1 --2- 172.21.15.121:0/2170875857 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f718015ca40 0x7f71680039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:59.323 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.321+0000 7f7185000640 1 --2- 172.21.15.121:0/2170875857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7168004010 0x7f7168004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:59.323 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.321+0000 7f7186400640 1 -- 172.21.15.121:0/2170875857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7168004010 msgr2=0x7f7168004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:20:59.324 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.321+0000 7f7186400640 1 --2- 172.21.15.121:0/2170875857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7168004010 0x7f7168004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:59.324 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.321+0000 7f7186400640 1 -- 172.21.15.121:0/2170875857 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7168003520 con 0x7f7168004d40 2024-04-20T10:20:59.325 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.322+0000 7f7186400640 1 --2- 172.21.15.121:0/2170875857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7168004d40 0x7f71680051c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7178001e20 tx=0x7f7168005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85cc7010ca732b0e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:59.325 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.322+0000 7f717fe00640 1 -- 172.21.15.121:0/2170875857 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7178001a70 con 0x7f7168004d40 2024-04-20T10:20:59.325 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.322+0000 7f717fe00640 1 -- 172.21.15.121:0/2170875857 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7178003b70 con 0x7f7168004d40 2024-04-20T10:20:59.325 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.322+0000 7f717fe00640 1 -- 172.21.15.121:0/2170875857 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71780050d0 con 0x7f7168004d40 2024-04-20T10:20:59.325 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.323+0000 7f7188400640 1 -- 172.21.15.121:0/2170875857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7168004d40 msgr2=0x7f71680051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:59.325 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.323+0000 7f7188400640 1 --2- 172.21.15.121:0/2170875857 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7168004d40 0x7f71680051c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7178001e20 tx=0x7f7168005830 comp rx=0 tx=0).stop 2024-04-20T10:20:59.325 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.323+0000 7f7186400640 1 -- 172.21.15.121:0/2170875857 reap_dead start 2024-04-20T10:20:59.326 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.323+0000 7f7188400640 1 -- 172.21.15.121:0/2170875857 shutdown_connections 2024-04-20T10:20:59.326 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.323+0000 7f7188400640 1 -- 172.21.15.121:0/2170875857 >> 172.21.15.121:0/2170875857 conn(0x7f71800b3350 msgr2=0x7f71800b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:20:59.326 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.323+0000 7f7188400640 1 -- 172.21.15.121:0/2170875857 shutdown_connections 2024-04-20T10:20:59.326 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.323+0000 7f7188400640 1 -- 172.21.15.121:0/2170875857 wait complete. 2024-04-20T10:20:59.326 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.325+0000 7f7188400640 1 Processor -- start 2024-04-20T10:20:59.326 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.325+0000 7f7188400640 1 -- start start 2024-04-20T10:20:59.328 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.326+0000 7f7188400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7168004d40 0x7f71800ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:59.328 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.326+0000 7f7188400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7180151e00 0x7f7180152280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:59.328 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.326+0000 7f7188400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71801f6050 0x7f71801f6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:59.328 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.326+0000 7f7188400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f71800d0a70 con 0x7f7168004d40 2024-04-20T10:20:59.328 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.326+0000 7f7188400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f718015d830 con 0x7f7180151e00 2024-04-20T10:20:59.328 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.326+0000 7f7188400640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f71800d0bf0 con 0x7f71801f6050 2024-04-20T10:20:59.328 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.326+0000 7f7185a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7168004d40 0x7f71800ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:59.328 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.326+0000 7f7185a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7168004d40 0x7f71800ce4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44162/0 (socket says 172.21.15.121:44162) 2024-04-20T10:20:59.328 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.326+0000 7f7185a00640 1 -- 172.21.15.121:0/3951953915 learned_addr learned my addr 172.21.15.121:0/3951953915 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:20:59.328 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.326+0000 7f7186400640 1 --2- 172.21.15.121:0/3951953915 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71801f6050 0x7f71801f6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:59.329 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.326+0000 7f7185a00640 1 -- 172.21.15.121:0/3951953915 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71801f6050 msgr2=0x7f71801f6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:59.329 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.326+0000 7f7185a00640 1 --2- 172.21.15.121:0/3951953915 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71801f6050 0x7f71801f6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:59.329 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.326+0000 7f7185a00640 1 -- 172.21.15.121:0/3951953915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7180151e00 msgr2=0x7f7180152280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:59.329 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.326+0000 7f7185a00640 1 --2- 172.21.15.121:0/3951953915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7180151e00 0x7f7180152280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:59.329 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.326+0000 7f7185a00640 1 -- 172.21.15.121:0/3951953915 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71800b1950 con 0x7f7168004d40 2024-04-20T10:20:59.329 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.327+0000 7f7185a00640 1 --2- 172.21.15.121:0/3951953915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7168004d40 0x7f71800ce4c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f717400e800 tx=0x7f717400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:59.329 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.327+0000 7f717d600640 1 -- 172.21.15.121:0/3951953915 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717400c8d0 con 0x7f7168004d40 2024-04-20T10:20:59.329 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.327+0000 7f717d600640 1 -- 172.21.15.121:0/3951953915 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7174018070 con 0x7f7168004d40 2024-04-20T10:20:59.329 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.327+0000 7f7188400640 1 -- 172.21.15.121:0/3951953915 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71800b1c30 con 0x7f7168004d40 2024-04-20T10:20:59.329 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.327+0000 7f717d600640 1 -- 172.21.15.121:0/3951953915 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717400ccb0 con 0x7f7168004d40 2024-04-20T10:20:59.329 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.327+0000 7f7188400640 1 -- 172.21.15.121:0/3951953915 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71800b21f0 con 0x7f7168004d40 2024-04-20T10:20:59.330 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.328+0000 7f7188400640 1 -- 172.21.15.121:0/3951953915 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7180068980 con 0x7f7168004d40 2024-04-20T10:20:59.330 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.329+0000 7f717d600640 1 -- 172.21.15.121:0/3951953915 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f717401d460 con 0x7f7168004d40 2024-04-20T10:20:59.330 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.330+0000 7f717d600640 1 --2- 172.21.15.121:0/3951953915 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f716006c920 0x7f716006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:59.330 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.330+0000 7f717d600640 1 -- 172.21.15.121:0/3951953915 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f717408b870 con 0x7f7168004d40 2024-04-20T10:20:59.330 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.330+0000 7f7185000640 1 --2- 172.21.15.121:0/3951953915 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f716006c920 0x7f716006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:59.331 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.331+0000 7f7185000640 1 --2- 172.21.15.121:0/3951953915 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f716006c920 0x7f716006ede0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f716c006a90 tx=0x7f716c006a20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:59.334 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.334+0000 7f717d600640 1 -- 172.21.15.121:0/3951953915 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7174059f50 con 0x7f7168004d40 2024-04-20T10:20:59.535 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.534+0000 7f7188400640 1 -- 172.21.15.121:0/3951953915 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f71800cfd90 con 0x7f7168004d40 2024-04-20T10:20:59.536 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.535+0000 7f717d600640 1 -- 172.21.15.121:0/3951953915 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) ==== 93+0+10 (secure 0 0 0) 0x7f7174059850 con 0x7f7168004d40 2024-04-20T10:20:59.536 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2024-04-20T10:20:59.538 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.538+0000 7f7188400640 1 -- 172.21.15.121:0/3951953915 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f716006c920 msgr2=0x7f716006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:59.539 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.538+0000 7f7188400640 1 --2- 172.21.15.121:0/3951953915 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f716006c920 0x7f716006ede0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f716c006a90 tx=0x7f716c006a20 comp rx=0 tx=0).stop 2024-04-20T10:20:59.539 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.538+0000 7f7188400640 1 -- 172.21.15.121:0/3951953915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7168004d40 msgr2=0x7f71800ce4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:59.539 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.538+0000 7f7188400640 1 --2- 172.21.15.121:0/3951953915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7168004d40 0x7f71800ce4c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f717400e800 tx=0x7f717400ecc0 comp rx=0 tx=0).stop 2024-04-20T10:20:59.539 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.538+0000 7f7188400640 1 -- 172.21.15.121:0/3951953915 shutdown_connections 2024-04-20T10:20:59.539 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.538+0000 7f7188400640 1 --2- 172.21.15.121:0/3951953915 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f71801f6050 0x7f71801f6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:59.539 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.538+0000 7f7188400640 1 --2- 172.21.15.121:0/3951953915 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f716006c920 0x7f716006ede0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:59.539 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.538+0000 7f7188400640 1 --2- 172.21.15.121:0/3951953915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7180151e00 0x7f7180152280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:59.539 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.538+0000 7f7188400640 1 --2- 172.21.15.121:0/3951953915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7168004d40 0x7f71800ce4c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:59.539 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.538+0000 7f7188400640 1 -- 172.21.15.121:0/3951953915 >> 172.21.15.121:0/3951953915 conn(0x7f71800b3350 msgr2=0x7f71800ad050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:20:59.539 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.538+0000 7f7188400640 1 -- 172.21.15.121:0/3951953915 shutdown_connections 2024-04-20T10:20:59.539 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.538+0000 7f7188400640 1 -- 172.21.15.121:0/3951953915 wait complete. 2024-04-20T10:20:59.559 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-20T10:20:59.759 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.758+0000 7efc5fa00640 1 Processor -- start 2024-04-20T10:20:59.759 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.759+0000 7efc5fa00640 1 -- start start 2024-04-20T10:20:59.760 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.760+0000 7efc5fa00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc580d4270 0x7efc580d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:59.760 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.760+0000 7efc5fa00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc580d3820 con 0x7efc580d0380 2024-04-20T10:20:59.760 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.760+0000 7efc5fa00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc580d3b00 con 0x7efc580d4650 2024-04-20T10:20:59.761 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.760+0000 7efc5fa00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7efc580d40c0 con 0x7efc580d4270 2024-04-20T10:20:59.761 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.760+0000 7efc5da00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7efc580d0380 0x7efc580d0810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:58832/0 (socket says 172.21.15.121:58832) 2024-04-20T10:20:59.761 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.760+0000 7efc5da00640 1 -- 172.21.15.121:0/1613069160 learned_addr learned my addr 172.21.15.121:0/1613069160 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:20:59.762 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.761+0000 7efc57e00640 1 -- 172.21.15.121:0/1613069160 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2919000989 0 0) 0x7efc580d3820 con 0x7efc580d0380 2024-04-20T10:20:59.762 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.761+0000 7efc5d000640 1 --2- 172.21.15.121:0/1613069160 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc580d4270 0x7efc580d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:59.762 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.761+0000 7efc57e00640 1 -- 172.21.15.121:0/1613069160 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc3c003520 con 0x7efc580d0380 2024-04-20T10:20:59.763 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.761+0000 7efc57e00640 1 -- 172.21.15.121:0/1613069160 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1808090594 0 0) 0x7efc3c003520 con 0x7efc580d0380 2024-04-20T10:20:59.763 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.761+0000 7efc57e00640 1 -- 172.21.15.121:0/1613069160 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc580d4270 msgr2=0x7efc580d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:20:59.763 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.761+0000 7efc57e00640 1 --2- 172.21.15.121:0/1613069160 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc580d4270 0x7efc580d1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:59.763 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.761+0000 7efc57e00640 1 -- 172.21.15.121:0/1613069160 >> v1:172.21.15.123:6789/0 conn(0x7efc580d4650 legacy=0x7efc580b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:59.763 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.761+0000 7efc57e00640 1 -- 172.21.15.121:0/1613069160 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc580d3de0 con 0x7efc580d0380 2024-04-20T10:20:59.763 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.761+0000 7efc57e00640 1 -- 172.21.15.121:0/1613069160 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7efc500019f0 con 0x7efc580d0380 2024-04-20T10:20:59.763 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.761+0000 7efc57e00640 1 -- 172.21.15.121:0/1613069160 >> v1:172.21.15.121:6789/0 conn(0x7efc580d0380 legacy=0x7efc580d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:59.763 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.761+0000 7efc57e00640 1 --2- 172.21.15.121:0/1613069160 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc3c0039e0 0x7efc3c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:59.763 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.761+0000 7efc57e00640 1 --2- 172.21.15.121:0/1613069160 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc3c004800 0x7efc3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:59.763 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.761+0000 7efc57e00640 1 --2- 172.21.15.121:0/1613069160 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc3c005210 0x7efc3c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:59.763 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.761+0000 7efc57e00640 1 -- 172.21.15.121:0/1613069160 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efc580d40c0 con 0x7efc3c004800 2024-04-20T10:20:59.763 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.761+0000 7efc57e00640 1 -- 172.21.15.121:0/1613069160 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efc3c00c0c0 con 0x7efc3c0039e0 2024-04-20T10:20:59.763 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.761+0000 7efc57e00640 1 -- 172.21.15.121:0/1613069160 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7efc3c00c2c0 con 0x7efc3c005210 2024-04-20T10:20:59.764 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.761+0000 7efc5d000640 1 --2- 172.21.15.121:0/1613069160 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc580d4270 0x7efc580d1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:20:59.764 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.761+0000 7efc4fe00640 1 --2- 172.21.15.121:0/1613069160 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc3c004800 0x7efc3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:59.764 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.762+0000 7efc4fe00640 1 -- 172.21.15.121:0/1613069160 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc3c005210 msgr2=0x7efc3c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:20:59.764 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.762+0000 7efc4fe00640 1 --2- 172.21.15.121:0/1613069160 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc3c005210 0x7efc3c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:59.764 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.762+0000 7efc4fe00640 1 -- 172.21.15.121:0/1613069160 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc3c0039e0 msgr2=0x7efc3c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:20:59.764 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.762+0000 7efc4fe00640 1 --2- 172.21.15.121:0/1613069160 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc3c0039e0 0x7efc3c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:59.764 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.762+0000 7efc4fe00640 1 -- 172.21.15.121:0/1613069160 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc3c003520 con 0x7efc3c004800 2024-04-20T10:20:59.765 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.763+0000 7efc4fe00640 1 --2- 172.21.15.121:0/1613069160 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc3c004800 0x7efc3c004c80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efc40001750 tx=0x7efc3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b522ac951b08d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:59.765 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.763+0000 7efc57e00640 1 -- 172.21.15.121:0/1613069160 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc40001370 con 0x7efc3c004800 2024-04-20T10:20:59.765 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.763+0000 7efc57e00640 1 -- 172.21.15.121:0/1613069160 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc400052d0 con 0x7efc3c004800 2024-04-20T10:20:59.766 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.763+0000 7efc57e00640 1 -- 172.21.15.121:0/1613069160 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc40003830 con 0x7efc3c004800 2024-04-20T10:20:59.766 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.764+0000 7efc5fa00640 1 -- 172.21.15.121:0/1613069160 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc3c004800 msgr2=0x7efc3c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:59.766 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.764+0000 7efc5fa00640 1 --2- 172.21.15.121:0/1613069160 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc3c004800 0x7efc3c004c80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efc40001750 tx=0x7efc3c005830 comp rx=0 tx=0).stop 2024-04-20T10:20:59.766 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.764+0000 7efc5da00640 1 -- 172.21.15.121:0/1613069160 reap_dead start 2024-04-20T10:20:59.766 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.764+0000 7efc5fa00640 1 -- 172.21.15.121:0/1613069160 shutdown_connections 2024-04-20T10:20:59.766 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.764+0000 7efc5fa00640 1 -- 172.21.15.121:0/1613069160 >> 172.21.15.121:0/1613069160 conn(0x7efc580b99c0 msgr2=0x7efc580b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:20:59.766 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.764+0000 7efc5fa00640 1 -- 172.21.15.121:0/1613069160 shutdown_connections 2024-04-20T10:20:59.766 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.764+0000 7efc5fa00640 1 -- 172.21.15.121:0/1613069160 wait complete. 2024-04-20T10:20:59.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.765+0000 7efc5fa00640 1 Processor -- start 2024-04-20T10:20:59.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.766+0000 7efc5fa00640 1 -- start start 2024-04-20T10:20:59.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.766+0000 7efc5fa00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc3c004800 0x7efc581594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:59.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.766+0000 7efc5fa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc5806b900 0x7efc581e8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:59.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.766+0000 7efc5fa00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc581e8580 0x7efc580c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:59.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.766+0000 7efc5fa00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efc580d53b0 con 0x7efc581e8580 2024-04-20T10:20:59.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.766+0000 7efc5fa00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efc580b19a0 con 0x7efc5806b900 2024-04-20T10:20:59.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.766+0000 7efc5fa00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7efc58161f70 con 0x7efc3c004800 2024-04-20T10:20:59.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.767+0000 7efc5da00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc581e8580 0x7efc580c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:59.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.767+0000 7efc5da00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc581e8580 0x7efc580c8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44174/0 (socket says 172.21.15.121:44174) 2024-04-20T10:20:59.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.767+0000 7efc5da00640 1 -- 172.21.15.121:0/1163511832 learned_addr learned my addr 172.21.15.121:0/1163511832 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:20:59.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.767+0000 7efc5d000640 1 --2- 172.21.15.121:0/1163511832 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc3c004800 0x7efc581594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:59.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.767+0000 7efc5da00640 1 -- 172.21.15.121:0/1163511832 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc3c004800 msgr2=0x7efc581594f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:59.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.767+0000 7efc5da00640 1 --2- 172.21.15.121:0/1163511832 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc3c004800 0x7efc581594f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:59.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.767+0000 7efc4fe00640 1 --2- 172.21.15.121:0/1163511832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc5806b900 0x7efc581e8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:59.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.767+0000 7efc5da00640 1 -- 172.21.15.121:0/1163511832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc5806b900 msgr2=0x7efc581e8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:59.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.767+0000 7efc5da00640 1 --2- 172.21.15.121:0/1163511832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc5806b900 0x7efc581e8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:59.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.767+0000 7efc5da00640 1 -- 172.21.15.121:0/1163511832 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc580b5540 con 0x7efc581e8580 2024-04-20T10:20:59.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.767+0000 7efc5da00640 1 --2- 172.21.15.121:0/1163511832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc581e8580 0x7efc580c8a10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efc5808cc90 tx=0x7efc5000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:59.770 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.768+0000 7efc55600640 1 -- 172.21.15.121:0/1163511832 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5000a610 con 0x7efc581e8580 2024-04-20T10:20:59.770 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.768+0000 7efc55600640 1 -- 172.21.15.121:0/1163511832 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc5000a7b0 con 0x7efc581e8580 2024-04-20T10:20:59.770 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.768+0000 7efc55600640 1 -- 172.21.15.121:0/1163511832 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5000aba0 con 0x7efc581e8580 2024-04-20T10:20:59.770 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.768+0000 7efc5fa00640 1 -- 172.21.15.121:0/1163511832 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc580c9750 con 0x7efc581e8580 2024-04-20T10:20:59.770 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.768+0000 7efc5fa00640 1 -- 172.21.15.121:0/1163511832 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc580b5ae0 con 0x7efc581e8580 2024-04-20T10:20:59.770 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.769+0000 7efc5fa00640 1 -- 172.21.15.121:0/1163511832 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc58068980 con 0x7efc581e8580 2024-04-20T10:20:59.771 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.770+0000 7efc55600640 1 -- 172.21.15.121:0/1163511832 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efc5000ad40 con 0x7efc581e8580 2024-04-20T10:20:59.771 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.771+0000 7efc55600640 1 --2- 172.21.15.121:0/1163511832 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7efc3406c8f0 0x7efc3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:20:59.771 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.771+0000 7efc55600640 1 -- 172.21.15.121:0/1163511832 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7efc5008b190 con 0x7efc581e8580 2024-04-20T10:20:59.771 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.771+0000 7efc5d000640 1 --2- 172.21.15.121:0/1163511832 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7efc3406c8f0 0x7efc3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:20:59.772 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.772+0000 7efc5d000640 1 --2- 172.21.15.121:0/1163511832 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7efc3406c8f0 0x7efc3406edb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efc48004790 tx=0x7efc48008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:20:59.776 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.775+0000 7efc55600640 1 -- 172.21.15.121:0/1163511832 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7efc50059870 con 0x7efc581e8580 2024-04-20T10:20:59.978 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.977+0000 7efc5fa00640 1 -- 172.21.15.121:0/1163511832 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7efc58067910 con 0x7efc581e8580 2024-04-20T10:20:59.979 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.978+0000 7efc55600640 1 -- 172.21.15.121:0/1163511832 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7efc50059170 con 0x7efc581e8580 2024-04-20T10:20:59.979 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:20:59.979 INFO:teuthology.orchestra.run.smithi121.stdout:[ 2024-04-20T10:20:59.979 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T10:20:59.979 INFO:teuthology.orchestra.run.smithi121.stdout: "poolnum": 1, 2024-04-20T10:20:59.979 INFO:teuthology.orchestra.run.smithi121.stdout: "poolname": ".mgr" 2024-04-20T10:20:59.979 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T10:20:59.979 INFO:teuthology.orchestra.run.smithi121.stdout:] 2024-04-20T10:20:59.982 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.981+0000 7efc5fa00640 1 -- 172.21.15.121:0/1163511832 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7efc3406c8f0 msgr2=0x7efc3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:59.982 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.981+0000 7efc5fa00640 1 --2- 172.21.15.121:0/1163511832 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7efc3406c8f0 0x7efc3406edb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efc48004790 tx=0x7efc48008040 comp rx=0 tx=0).stop 2024-04-20T10:20:59.982 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.981+0000 7efc5fa00640 1 -- 172.21.15.121:0/1163511832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc581e8580 msgr2=0x7efc580c8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:20:59.982 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.981+0000 7efc5fa00640 1 --2- 172.21.15.121:0/1163511832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc581e8580 0x7efc580c8a10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7efc5808cc90 tx=0x7efc5000ccd0 comp rx=0 tx=0).stop 2024-04-20T10:20:59.982 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.981+0000 7efc5fa00640 1 -- 172.21.15.121:0/1163511832 shutdown_connections 2024-04-20T10:20:59.982 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.981+0000 7efc5fa00640 1 --2- 172.21.15.121:0/1163511832 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc581e8580 0x7efc580c8a10 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:59.982 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.981+0000 7efc5fa00640 1 --2- 172.21.15.121:0/1163511832 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7efc3406c8f0 0x7efc3406edb0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:59.982 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.981+0000 7efc5fa00640 1 --2- 172.21.15.121:0/1163511832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc5806b900 0x7efc581e8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:59.983 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.981+0000 7efc5fa00640 1 --2- 172.21.15.121:0/1163511832 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc3c004800 0x7efc581594f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:20:59.983 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.981+0000 7efc5fa00640 1 -- 172.21.15.121:0/1163511832 >> 172.21.15.121:0/1163511832 conn(0x7efc580b99c0 msgr2=0x7efc580d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:20:59.983 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.981+0000 7efc5fa00640 1 -- 172.21.15.121:0/1163511832 shutdown_connections 2024-04-20T10:20:59.983 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:20:59.981+0000 7efc5fa00640 1 -- 172.21.15.121:0/1163511832 wait complete. 2024-04-20T10:21:00.001 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-20T10:21:00.001 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-20T10:21:00.206 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.206+0000 7f2380800640 1 Processor -- start 2024-04-20T10:21:00.206 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.206+0000 7f2380800640 1 -- start start 2024-04-20T10:21:00.208 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.207+0000 7f2380800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f23780d0360 0x7f23780d07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:00.208 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.207+0000 7f2380800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2378141c60 con 0x7f237815ca40 2024-04-20T10:21:00.208 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.207+0000 7f2380800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23780adab0 con 0x7f2378156c20 2024-04-20T10:21:00.208 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.207+0000 7f2380800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f23780d0d10 con 0x7f23780d0360 2024-04-20T10:21:00.208 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.208+0000 7f237e800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f23780d0360 0x7f23780d07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:00.208 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.208+0000 7f237d400640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f237815ca40 0x7f23780ad3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:58848/0 (socket says 172.21.15.121:58848) 2024-04-20T10:21:00.208 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.208+0000 7f237d400640 1 -- 172.21.15.121:0/845070880 learned_addr learned my addr 172.21.15.121:0/845070880 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:00.209 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.208+0000 7f237e800640 1 -- 172.21.15.121:0/845070880 >> v1:172.21.15.123:6789/0 conn(0x7f2378156c20 legacy=0x7f237806b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:00.209 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.208+0000 7f237e800640 1 -- 172.21.15.121:0/845070880 >> v1:172.21.15.121:6789/0 conn(0x7f237815ca40 legacy=0x7f23780ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:00.209 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.208+0000 7f237e800640 1 -- 172.21.15.121:0/845070880 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f237815cf90 con 0x7f23780d0360 2024-04-20T10:21:00.209 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.209+0000 7f237e800640 1 --2- 172.21.15.121:0/845070880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23780d0360 0x7f23780d07d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2374009d90 tx=0x7f237402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876239ea660a13b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:00.210 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.209+0000 7f237ca00640 1 -- 172.21.15.121:0/845070880 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2374039070 con 0x7f23780d0360 2024-04-20T10:21:00.210 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.210+0000 7f237ca00640 1 -- 172.21.15.121:0/845070880 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f237402cc60 con 0x7f23780d0360 2024-04-20T10:21:00.210 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.210+0000 7f237ca00640 1 -- 172.21.15.121:0/845070880 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2374034660 con 0x7f23780d0360 2024-04-20T10:21:00.211 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.210+0000 7f2380800640 1 -- 172.21.15.121:0/845070880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23780d0360 msgr2=0x7f23780d07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:00.211 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.210+0000 7f2380800640 1 --2- 172.21.15.121:0/845070880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23780d0360 0x7f23780d07d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2374009d90 tx=0x7f237402beb0 comp rx=0 tx=0).stop 2024-04-20T10:21:00.211 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.210+0000 7f2380800640 1 -- 172.21.15.121:0/845070880 shutdown_connections 2024-04-20T10:21:00.211 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.210+0000 7f2380800640 1 --2- 172.21.15.121:0/845070880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23780d0360 0x7f23780d07d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:00.211 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.210+0000 7f2380800640 1 -- 172.21.15.121:0/845070880 >> 172.21.15.121:0/845070880 conn(0x7f23780b3350 msgr2=0x7f23780b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:00.211 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.210+0000 7f2380800640 1 -- 172.21.15.121:0/845070880 shutdown_connections 2024-04-20T10:21:00.211 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.210+0000 7f2380800640 1 -- 172.21.15.121:0/845070880 wait complete. 2024-04-20T10:21:00.212 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.212+0000 7f2380800640 1 Processor -- start 2024-04-20T10:21:00.212 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.212+0000 7f2380800640 1 -- start start 2024-04-20T10:21:00.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.213+0000 7f2380800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23780d0360 0x7f237814f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:00.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.213+0000 7f2380800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2378156c20 0x7f2378147a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:00.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.213+0000 7f2380800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f237815ca40 0x7f23780cd540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:00.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.213+0000 7f2380800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f237815d550 con 0x7f23780d0360 2024-04-20T10:21:00.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.213+0000 7f2380800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f237806c010 con 0x7f237815ca40 2024-04-20T10:21:00.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.213+0000 7f2380800640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f237815d830 con 0x7f2378156c20 2024-04-20T10:21:00.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.213+0000 7f237de00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23780d0360 0x7f237814f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:00.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.213+0000 7f237de00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23780d0360 0x7f237814f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44188/0 (socket says 172.21.15.121:44188) 2024-04-20T10:21:00.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.213+0000 7f237de00640 1 -- 172.21.15.121:0/533395685 learned_addr learned my addr 172.21.15.121:0/533395685 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:00.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.214+0000 7f237e800640 1 --2- 172.21.15.121:0/533395685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f237815ca40 0x7f23780cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:00.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.214+0000 7f237d400640 1 --2- 172.21.15.121:0/533395685 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2378156c20 0x7f2378147a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:00.216 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.214+0000 7f237de00640 1 -- 172.21.15.121:0/533395685 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2378156c20 msgr2=0x7f2378147a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:00.216 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.214+0000 7f237de00640 1 --2- 172.21.15.121:0/533395685 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2378156c20 0x7f2378147a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:00.216 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.214+0000 7f237de00640 1 -- 172.21.15.121:0/533395685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f237815ca40 msgr2=0x7f23780cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:00.216 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.214+0000 7f237de00640 1 --2- 172.21.15.121:0/533395685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f237815ca40 0x7f23780cd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:00.216 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.214+0000 7f237de00640 1 -- 172.21.15.121:0/533395685 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23780cda80 con 0x7f23780d0360 2024-04-20T10:21:00.216 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.214+0000 7f237e800640 1 --2- 172.21.15.121:0/533395685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f237815ca40 0x7f23780cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:00.216 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.214+0000 7f237de00640 1 --2- 172.21.15.121:0/533395685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23780d0360 0x7f237814f0e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f236c00c430 tx=0x7f236c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:00.216 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.214+0000 7f2372000640 1 -- 172.21.15.121:0/533395685 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236c011790 con 0x7f23780d0360 2024-04-20T10:21:00.216 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.214+0000 7f2372000640 1 -- 172.21.15.121:0/533395685 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f236c017070 con 0x7f23780d0360 2024-04-20T10:21:00.216 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.214+0000 7f2372000640 1 -- 172.21.15.121:0/533395685 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236c011bb0 con 0x7f23780d0360 2024-04-20T10:21:00.216 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.214+0000 7f2380800640 1 -- 172.21.15.121:0/533395685 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23780cdd20 con 0x7f23780d0360 2024-04-20T10:21:00.216 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.215+0000 7f2380800640 1 -- 172.21.15.121:0/533395685 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f237806ee10 con 0x7f23780d0360 2024-04-20T10:21:00.217 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.216+0000 7f2380800640 1 -- 172.21.15.121:0/533395685 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2378050540 con 0x7f23780d0360 2024-04-20T10:21:00.217 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.217+0000 7f2372000640 1 -- 172.21.15.121:0/533395685 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f236c01c460 con 0x7f23780d0360 2024-04-20T10:21:00.218 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.217+0000 7f2372000640 1 --2- 172.21.15.121:0/533395685 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f235c06c8f0 0x7f235c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:00.218 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.218+0000 7f2372000640 1 -- 172.21.15.121:0/533395685 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f236c08b6b0 con 0x7f23780d0360 2024-04-20T10:21:00.218 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.218+0000 7f237d400640 1 --2- 172.21.15.121:0/533395685 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f235c06c8f0 0x7f235c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:00.219 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.219+0000 7f237d400640 1 --2- 172.21.15.121:0/533395685 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f235c06c8f0 0x7f235c06edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f23780b3ae0 tx=0x7f2368009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:00.222 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.222+0000 7f2372000640 1 -- 172.21.15.121:0/533395685 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2378050540 con 0x7f23780d0360 2024-04-20T10:21:00.422 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.422+0000 7f2380800640 1 -- 172.21.15.121:0/533395685 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f23780d07d0 con 0x7f23780d0360 2024-04-20T10:21:00.565 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.565+0000 7f2372000640 1 -- 172.21.15.121:0/533395685 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7f23780d07d0 con 0x7f23780d0360 2024-04-20T10:21:00.566 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'cephfs_metadata' created 2024-04-20T10:21:00.569 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.569+0000 7f2380800640 1 -- 172.21.15.121:0/533395685 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f235c06c8f0 msgr2=0x7f235c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:00.569 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.569+0000 7f2380800640 1 --2- 172.21.15.121:0/533395685 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f235c06c8f0 0x7f235c06edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f23780b3ae0 tx=0x7f2368009040 comp rx=0 tx=0).stop 2024-04-20T10:21:00.569 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.569+0000 7f2380800640 1 -- 172.21.15.121:0/533395685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23780d0360 msgr2=0x7f237814f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:00.570 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.569+0000 7f2380800640 1 --2- 172.21.15.121:0/533395685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23780d0360 0x7f237814f0e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f236c00c430 tx=0x7f236c00c8f0 comp rx=0 tx=0).stop 2024-04-20T10:21:00.570 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.569+0000 7f2380800640 1 -- 172.21.15.121:0/533395685 shutdown_connections 2024-04-20T10:21:00.570 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.569+0000 7f2380800640 1 --2- 172.21.15.121:0/533395685 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f235c06c8f0 0x7f235c06edb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:00.570 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.569+0000 7f2380800640 1 --2- 172.21.15.121:0/533395685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f237815ca40 0x7f23780cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:00.570 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.569+0000 7f2380800640 1 --2- 172.21.15.121:0/533395685 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2378156c20 0x7f2378147a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:00.571 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.569+0000 7f2380800640 1 --2- 172.21.15.121:0/533395685 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f23780d0360 0x7f237814f0e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:00.571 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.569+0000 7f2380800640 1 -- 172.21.15.121:0/533395685 >> 172.21.15.121:0/533395685 conn(0x7f23780b3350 msgr2=0x7f23780998c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:00.571 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.570+0000 7f2380800640 1 -- 172.21.15.121:0/533395685 shutdown_connections 2024-04-20T10:21:00.571 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.570+0000 7f2380800640 1 -- 172.21.15.121:0/533395685 wait complete. 2024-04-20T10:21:00.599 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-20T10:21:00.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.789+0000 7f5287200640 1 Processor -- start 2024-04-20T10:21:00.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.789+0000 7f5287200640 1 -- start start 2024-04-20T10:21:00.791 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.790+0000 7f5287200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f528815a1f0 0x7f5288072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:00.791 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.790+0000 7f5287200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5288072190 0x7f5288073420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:00.791 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.790+0000 7f5287200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f528815cc90 con 0x7f5288159e10 2024-04-20T10:21:00.791 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.790+0000 7f5287200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f5288072900 con 0x7f5288072190 2024-04-20T10:21:00.791 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.790+0000 7f5287200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f528815ca40 con 0x7f528815a1f0 2024-04-20T10:21:00.791 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.791+0000 7f5285e00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f5288159e10 0x7f5288071a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:58856/0 (socket says 172.21.15.121:58856) 2024-04-20T10:21:00.791 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.791+0000 7f5285e00640 1 -- 172.21.15.121:0/2556899271 learned_addr learned my addr 172.21.15.121:0/2556899271 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:00.791 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.791+0000 7f5286800640 1 --2- 172.21.15.121:0/2556899271 >> v2:172.21.15.121:3300/0 conn(0x7f5288072190 0x7f5288073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:00.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.791+0000 7f527fe00640 1 --2- 172.21.15.121:0/2556899271 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f528815a1f0 0x7f5288072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:00.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.791+0000 7f5285400640 1 -- 172.21.15.121:0/2556899271 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3228023508 0 0) 0x7f528815cc90 con 0x7f5288159e10 2024-04-20T10:21:00.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.791+0000 7f5286800640 1 -- 172.21.15.121:0/2556899271 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f528815a1f0 msgr2=0x7f5288072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:00.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.791+0000 7f5286800640 1 --2- 172.21.15.121:0/2556899271 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f528815a1f0 0x7f5288072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:00.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.791+0000 7f5286800640 1 -- 172.21.15.121:0/2556899271 >> v1:172.21.15.121:6789/0 conn(0x7f5288159e10 legacy=0x7f5288071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:00.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.791+0000 7f5286800640 1 -- 172.21.15.121:0/2556899271 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f528815cf70 con 0x7f5288072190 2024-04-20T10:21:00.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.792+0000 7f5286800640 1 --2- 172.21.15.121:0/2556899271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5288072190 0x7f5288073420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5280009d90 tx=0x7f528002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894e1f01aa751351 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:00.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.793+0000 7f5285400640 1 -- 172.21.15.121:0/2556899271 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5280039070 con 0x7f5288072190 2024-04-20T10:21:00.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.793+0000 7f5285400640 1 -- 172.21.15.121:0/2556899271 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f528002cc50 con 0x7f5288072190 2024-04-20T10:21:00.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.793+0000 7f5285400640 1 -- 172.21.15.121:0/2556899271 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5280034440 con 0x7f5288072190 2024-04-20T10:21:00.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.793+0000 7f5287200640 1 -- 172.21.15.121:0/2556899271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5288072190 msgr2=0x7f5288073420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:00.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.793+0000 7f5287200640 1 --2- 172.21.15.121:0/2556899271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5288072190 0x7f5288073420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5280009d90 tx=0x7f528002bea0 comp rx=0 tx=0).stop 2024-04-20T10:21:00.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.793+0000 7f5287200640 1 -- 172.21.15.121:0/2556899271 shutdown_connections 2024-04-20T10:21:00.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.793+0000 7f5287200640 1 --2- 172.21.15.121:0/2556899271 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5288072190 0x7f5288073420 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:00.795 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.793+0000 7f5287200640 1 --2- 172.21.15.121:0/2556899271 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f528815a1f0 0x7f5288072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:00.795 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.793+0000 7f5287200640 1 -- 172.21.15.121:0/2556899271 >> 172.21.15.121:0/2556899271 conn(0x7f52880d0040 msgr2=0x7f52880d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:00.795 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.794+0000 7f5287200640 1 -- 172.21.15.121:0/2556899271 shutdown_connections 2024-04-20T10:21:00.795 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.794+0000 7f5287200640 1 -- 172.21.15.121:0/2556899271 wait complete. 2024-04-20T10:21:00.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.795+0000 7f5287200640 1 Processor -- start 2024-04-20T10:21:00.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.795+0000 7f5287200640 1 -- start start 2024-04-20T10:21:00.798 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.796+0000 7f5287200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5288072190 0x7f52881503e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:00.798 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.796+0000 7f5287200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5288159e10 0x7f52880b3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:00.798 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.796+0000 7f5287200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f528815a1f0 0x7f528815ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:00.798 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.796+0000 7f5287200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f528815d810 con 0x7f5288159e10 2024-04-20T10:21:00.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.796+0000 7f5287200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f528815d530 con 0x7f5288072190 2024-04-20T10:21:00.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.796+0000 7f5287200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f528806da90 con 0x7f528815a1f0 2024-04-20T10:21:00.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.797+0000 7f527fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5288159e10 0x7f52880b3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:00.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.797+0000 7f527fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5288159e10 0x7f52880b3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44212/0 (socket says 172.21.15.121:44212) 2024-04-20T10:21:00.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.797+0000 7f527fe00640 1 -- 172.21.15.121:0/2041330635 learned_addr learned my addr 172.21.15.121:0/2041330635 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:00.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.797+0000 7f5286800640 1 --2- 172.21.15.121:0/2041330635 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f528815a1f0 0x7f528815ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:00.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.797+0000 7f5285e00640 1 --2- 172.21.15.121:0/2041330635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5288072190 0x7f52881503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:00.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.797+0000 7f527fe00640 1 -- 172.21.15.121:0/2041330635 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f528815a1f0 msgr2=0x7f528815ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:00.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.797+0000 7f527fe00640 1 --2- 172.21.15.121:0/2041330635 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f528815a1f0 0x7f528815ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:00.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.797+0000 7f527fe00640 1 -- 172.21.15.121:0/2041330635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5288072190 msgr2=0x7f52881503e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:00.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.797+0000 7f527fe00640 1 --2- 172.21.15.121:0/2041330635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5288072190 0x7f52881503e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:00.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.797+0000 7f527fe00640 1 -- 172.21.15.121:0/2041330635 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52880b3aa0 con 0x7f5288159e10 2024-04-20T10:21:00.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.797+0000 7f527fe00640 1 --2- 172.21.15.121:0/2041330635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5288159e10 0x7f52880b3070 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f527000d5f0 tx=0x7f527000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:00.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.798+0000 7f527e000640 1 -- 172.21.15.121:0/2041330635 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52700125f0 con 0x7f5288159e10 2024-04-20T10:21:00.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.798+0000 7f527e000640 1 -- 172.21.15.121:0/2041330635 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5270019070 con 0x7f5288159e10 2024-04-20T10:21:00.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.798+0000 7f527e000640 1 -- 172.21.15.121:0/2041330635 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52700129d0 con 0x7f5288159e10 2024-04-20T10:21:00.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.798+0000 7f5287200640 1 -- 172.21.15.121:0/2041330635 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52880b3d80 con 0x7f5288159e10 2024-04-20T10:21:00.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.798+0000 7f5287200640 1 -- 172.21.15.121:0/2041330635 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52880b4340 con 0x7f5288159e10 2024-04-20T10:21:00.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.799+0000 7f5287200640 1 -- 172.21.15.121:0/2041330635 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5288075b60 con 0x7f5288159e10 2024-04-20T10:21:00.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.800+0000 7f527e000640 1 -- 172.21.15.121:0/2041330635 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5270012b70 con 0x7f5288159e10 2024-04-20T10:21:00.802 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.802+0000 7f527e000640 1 --2- 172.21.15.121:0/2041330635 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f526406c8f0 0x7f526406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:00.802 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.802+0000 7f527e000640 1 -- 172.21.15.121:0/2041330635 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7f527008c900 con 0x7f5288159e10 2024-04-20T10:21:00.802 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.802+0000 7f5285e00640 1 --2- 172.21.15.121:0/2041330635 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f526406c8f0 0x7f526406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:00.803 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.802+0000 7f5285e00640 1 --2- 172.21.15.121:0/2041330635 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f526406c8f0 0x7f526406edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5278007fd0 tx=0x7f5278009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:00.806 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:00.805+0000 7f527e000640 1 -- 172.21.15.121:0/2041330635 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f527005ae60 con 0x7f5288159e10 2024-04-20T10:21:01.013 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.013+0000 7f5287200640 1 -- 172.21.15.121:0/2041330635 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f528806bc30 con 0x7f5288159e10 2024-04-20T10:21:01.572 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.571+0000 7f527e000640 1 -- 172.21.15.121:0/2041330635 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) ==== 171+0+0 (secure 0 0 0) 0x7f527005a760 con 0x7f5288159e10 2024-04-20T10:21:01.572 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'cephfs_data' created 2024-04-20T10:21:01.575 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.575+0000 7f5287200640 1 -- 172.21.15.121:0/2041330635 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f526406c8f0 msgr2=0x7f526406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:01.575 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.575+0000 7f5287200640 1 --2- 172.21.15.121:0/2041330635 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f526406c8f0 0x7f526406edb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5278007fd0 tx=0x7f5278009040 comp rx=0 tx=0).stop 2024-04-20T10:21:01.575 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.575+0000 7f5287200640 1 -- 172.21.15.121:0/2041330635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5288159e10 msgr2=0x7f52880b3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:01.576 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.575+0000 7f5287200640 1 --2- 172.21.15.121:0/2041330635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5288159e10 0x7f52880b3070 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f527000d5f0 tx=0x7f527000dab0 comp rx=0 tx=0).stop 2024-04-20T10:21:01.576 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.575+0000 7f5287200640 1 -- 172.21.15.121:0/2041330635 shutdown_connections 2024-04-20T10:21:01.576 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.575+0000 7f5287200640 1 --2- 172.21.15.121:0/2041330635 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f528815a1f0 0x7f528815ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:01.576 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.575+0000 7f5287200640 1 --2- 172.21.15.121:0/2041330635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5288159e10 0x7f52880b3070 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:01.576 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.575+0000 7f5287200640 1 --2- 172.21.15.121:0/2041330635 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f526406c8f0 0x7f526406edb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:01.576 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.575+0000 7f5287200640 1 --2- 172.21.15.121:0/2041330635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5288072190 0x7f52881503e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:01.576 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.575+0000 7f5287200640 1 -- 172.21.15.121:0/2041330635 >> 172.21.15.121:0/2041330635 conn(0x7f52880d0040 msgr2=0x7f528806cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:01.576 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.576+0000 7f5287200640 1 -- 172.21.15.121:0/2041330635 shutdown_connections 2024-04-20T10:21:01.577 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.576+0000 7f5287200640 1 -- 172.21.15.121:0/2041330635 wait complete. 2024-04-20T10:21:01.610 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-20T10:21:01.788 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.788+0000 7f969ee00640 1 Processor -- start 2024-04-20T10:21:01.788 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.788+0000 7f969ee00640 1 -- start start 2024-04-20T10:21:01.790 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.789+0000 7f969ee00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f96980d0360 0x7f96980d0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:01.790 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.789+0000 7f969ee00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96980ade40 con 0x7f969806dcb0 2024-04-20T10:21:01.790 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.789+0000 7f969ee00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96980ae0e0 con 0x7f969815b3b0 2024-04-20T10:21:01.790 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.789+0000 7f969ee00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f969806f1b0 con 0x7f96980d0360 2024-04-20T10:21:01.790 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.790+0000 7f968f400640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f969806dcb0 0x7f9698071b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:58866/0 (socket says 172.21.15.121:58866) 2024-04-20T10:21:01.790 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.790+0000 7f968f400640 1 -- 172.21.15.121:0/2521468980 learned_addr learned my addr 172.21.15.121:0/2521468980 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:01.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.790+0000 7f969ce00640 1 --2- 172.21.15.121:0/2521468980 >> v2:172.21.15.123:3300/0 conn(0x7f96980d0360 0x7f96980d0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:01.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.790+0000 7f9697400640 1 -- 172.21.15.121:0/2521468980 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3004573966 0 0) 0x7f96980ade40 con 0x7f969806dcb0 2024-04-20T10:21:01.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.790+0000 7f9697400640 1 -- 172.21.15.121:0/2521468980 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f967c003520 con 0x7f969806dcb0 2024-04-20T10:21:01.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.791+0000 7f9697400640 1 -- 172.21.15.121:0/2521468980 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 355769096 0 0) 0x7f967c003520 con 0x7f969806dcb0 2024-04-20T10:21:01.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.791+0000 7f9697400640 1 -- 172.21.15.121:0/2521468980 >> v2:172.21.15.123:3300/0 conn(0x7f96980d0360 msgr2=0x7f96980d0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:01.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.791+0000 7f9697400640 1 --2- 172.21.15.121:0/2521468980 >> v2:172.21.15.123:3300/0 conn(0x7f96980d0360 0x7f96980d0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:01.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.791+0000 7f9697400640 1 -- 172.21.15.121:0/2521468980 >> v1:172.21.15.123:6789/0 conn(0x7f969815b3b0 legacy=0x7f969806eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:01.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.791+0000 7f9697400640 1 -- 172.21.15.121:0/2521468980 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f969815b790 con 0x7f969806dcb0 2024-04-20T10:21:01.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.791+0000 7f9697400640 1 -- 172.21.15.121:0/2521468980 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f9680001670 con 0x7f969806dcb0 2024-04-20T10:21:01.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.791+0000 7f9697400640 1 -- 172.21.15.121:0/2521468980 >> v1:172.21.15.121:6789/0 conn(0x7f969806dcb0 legacy=0x7f9698071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:01.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.791+0000 7f9697400640 1 --2- 172.21.15.121:0/2521468980 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f967c0039b0 0x7f967c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:01.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.791+0000 7f9697400640 1 --2- 172.21.15.121:0/2521468980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f967c004800 0x7f967c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:01.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.791+0000 7f9697400640 1 --2- 172.21.15.121:0/2521468980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f967c0051c0 0x7f967c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:01.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.791+0000 7f9697400640 1 -- 172.21.15.121:0/2521468980 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f969806f1b0 con 0x7f967c004800 2024-04-20T10:21:01.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.791+0000 7f9697400640 1 -- 172.21.15.121:0/2521468980 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f967c00c060 con 0x7f967c0051c0 2024-04-20T10:21:01.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.791+0000 7f9697400640 1 -- 172.21.15.121:0/2521468980 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f967c00c260 con 0x7f967c0039b0 2024-04-20T10:21:01.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.791+0000 7f968f400640 1 --2- 172.21.15.121:0/2521468980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f967c004800 0x7f967c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:01.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.791+0000 7f9697e00640 1 --2- 172.21.15.121:0/2521468980 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f967c0039b0 0x7f967c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:01.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.792+0000 7f968f400640 1 -- 172.21.15.121:0/2521468980 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f967c0039b0 msgr2=0x7f967c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:01.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.792+0000 7f968f400640 1 --2- 172.21.15.121:0/2521468980 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f967c0039b0 0x7f967c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:01.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.792+0000 7f968f400640 1 -- 172.21.15.121:0/2521468980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f967c0051c0 msgr2=0x7f967c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:01.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.792+0000 7f969ce00640 1 --2- 172.21.15.121:0/2521468980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f967c0051c0 0x7f967c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:01.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.792+0000 7f968f400640 1 --2- 172.21.15.121:0/2521468980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f967c0051c0 0x7f967c00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:01.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.792+0000 7f968f400640 1 -- 172.21.15.121:0/2521468980 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f967c003520 con 0x7f967c004800 2024-04-20T10:21:01.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.792+0000 7f969ce00640 1 -- 172.21.15.121:0/2521468980 reap_dead start 2024-04-20T10:21:01.795 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.793+0000 7f968f400640 1 --2- 172.21.15.121:0/2521468980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f967c004800 0x7f967c004c80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f968000c240 tx=0x7f967c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd5e9b2426eb2f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:01.795 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.793+0000 7f9697400640 1 -- 172.21.15.121:0/2521468980 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9680007d10 con 0x7f967c004800 2024-04-20T10:21:01.795 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.793+0000 7f9697400640 1 -- 172.21.15.121:0/2521468980 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9680030070 con 0x7f967c004800 2024-04-20T10:21:01.795 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.793+0000 7f9697400640 1 -- 172.21.15.121:0/2521468980 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9680005d70 con 0x7f967c004800 2024-04-20T10:21:01.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.794+0000 7f969ee00640 1 -- 172.21.15.121:0/2521468980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f967c004800 msgr2=0x7f967c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:01.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.794+0000 7f969ee00640 1 --2- 172.21.15.121:0/2521468980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f967c004800 0x7f967c004c80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f968000c240 tx=0x7f967c005830 comp rx=0 tx=0).stop 2024-04-20T10:21:01.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.794+0000 7f969ee00640 1 -- 172.21.15.121:0/2521468980 shutdown_connections 2024-04-20T10:21:01.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.794+0000 7f969ee00640 1 --2- 172.21.15.121:0/2521468980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f967c004800 0x7f967c004c80 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:01.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.794+0000 7f969ee00640 1 -- 172.21.15.121:0/2521468980 >> 172.21.15.121:0/2521468980 conn(0x7f96980b3350 msgr2=0x7f96980b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:01.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.794+0000 7f969ee00640 1 -- 172.21.15.121:0/2521468980 shutdown_connections 2024-04-20T10:21:01.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.794+0000 7f969ee00640 1 -- 172.21.15.121:0/2521468980 wait complete. 2024-04-20T10:21:01.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.796+0000 7f969ee00640 1 Processor -- start 2024-04-20T10:21:01.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.796+0000 7f969ee00640 1 -- start start 2024-04-20T10:21:01.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.797+0000 7f969ee00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f967c004800 0x7f969814aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:01.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.797+0000 7f969ee00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96980c6df0 0x7f96980c7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:01.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.797+0000 7f969ee00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f969815b960 0x7f969815bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:01.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.797+0000 7f969ee00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f96980ae940 con 0x7f969815b960 2024-04-20T10:21:01.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.797+0000 7f969ee00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f96980ae660 con 0x7f96980c6df0 2024-04-20T10:21:01.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.797+0000 7f969ee00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f96980aec20 con 0x7f967c004800 2024-04-20T10:21:01.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.797+0000 7f969ce00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f969815b960 0x7f969815bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:01.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.797+0000 7f969ce00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f969815b960 0x7f969815bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44230/0 (socket says 172.21.15.121:44230) 2024-04-20T10:21:01.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.797+0000 7f969ce00640 1 -- 172.21.15.121:0/3133623786 learned_addr learned my addr 172.21.15.121:0/3133623786 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:01.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.797+0000 7f968f400640 1 --2- 172.21.15.121:0/3133623786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96980c6df0 0x7f96980c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:01.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.797+0000 7f9697e00640 1 --2- 172.21.15.121:0/3133623786 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f967c004800 0x7f969814aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:01.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.798+0000 7f969ce00640 1 -- 172.21.15.121:0/3133623786 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f967c004800 msgr2=0x7f969814aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:01.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.798+0000 7f969ce00640 1 --2- 172.21.15.121:0/3133623786 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f967c004800 0x7f969814aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:01.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.798+0000 7f969ce00640 1 -- 172.21.15.121:0/3133623786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96980c6df0 msgr2=0x7f96980c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:01.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.798+0000 7f969ce00640 1 --2- 172.21.15.121:0/3133623786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96980c6df0 0x7f96980c7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:01.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.798+0000 7f969ce00640 1 -- 172.21.15.121:0/3133623786 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f969815c670 con 0x7f969815b960 2024-04-20T10:21:01.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.798+0000 7f9697e00640 1 --2- 172.21.15.121:0/3133623786 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f967c004800 0x7f969814aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:01.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.798+0000 7f968f400640 1 --2- 172.21.15.121:0/3133623786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96980c6df0 0x7f96980c7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:01.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.798+0000 7f969ce00640 1 --2- 172.21.15.121:0/3133623786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f969815b960 0x7f969815bdf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f96980ad6d0 tx=0x7f969000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:01.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.798+0000 7f9694c00640 1 -- 172.21.15.121:0/3133623786 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9690016420 con 0x7f969815b960 2024-04-20T10:21:01.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.798+0000 7f9694c00640 1 -- 172.21.15.121:0/3133623786 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f969001b070 con 0x7f969815b960 2024-04-20T10:21:01.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.798+0000 7f9694c00640 1 -- 172.21.15.121:0/3133623786 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9690016820 con 0x7f969815b960 2024-04-20T10:21:01.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.798+0000 7f969ee00640 1 -- 172.21.15.121:0/3133623786 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f969815c950 con 0x7f969815b960 2024-04-20T10:21:01.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.798+0000 7f969ee00640 1 -- 172.21.15.121:0/3133623786 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f969815cf10 con 0x7f969815b960 2024-04-20T10:21:01.801 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.799+0000 7f969ee00640 1 -- 172.21.15.121:0/3133623786 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9698068980 con 0x7f969815b960 2024-04-20T10:21:01.801 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.800+0000 7f9694c00640 1 -- 172.21.15.121:0/3133623786 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9690005d10 con 0x7f969815b960 2024-04-20T10:21:01.802 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.801+0000 7f9694c00640 1 --2- 172.21.15.121:0/3133623786 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f967406c8f0 0x7f967406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:01.802 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.801+0000 7f9694c00640 1 -- 172.21.15.121:0/3133623786 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7f969008c6d0 con 0x7f969815b960 2024-04-20T10:21:01.802 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.802+0000 7f9697e00640 1 --2- 172.21.15.121:0/3133623786 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f967406c8f0 0x7f967406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:01.802 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.802+0000 7f9697e00640 1 --2- 172.21.15.121:0/3133623786 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f967406c8f0 0x7f967406edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f96980ae2b0 tx=0x7f9688009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:01.807 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:01.806+0000 7f9694c00640 1 -- 172.21.15.121:0/3133623786 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9690025020 con 0x7f969815b960 2024-04-20T10:21:02.050 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.046+0000 7f969ee00640 1 -- 172.21.15.121:0/3133623786 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f9698156ea0 con 0x7f969815b960 2024-04-20T10:21:02.599 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.598+0000 7f9694c00640 1 -- 172.21.15.121:0/3133623786 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f969005aac0 con 0x7f969815b960 2024-04-20T10:21:02.599 INFO:teuthology.orchestra.run.smithi121.stderr:new fs with metadata pool 2 and data pool 3 2024-04-20T10:21:02.602 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.601+0000 7f969ee00640 1 -- 172.21.15.121:0/3133623786 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f967406c8f0 msgr2=0x7f967406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:02.602 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.601+0000 7f969ee00640 1 --2- 172.21.15.121:0/3133623786 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f967406c8f0 0x7f967406edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f96980ae2b0 tx=0x7f9688009040 comp rx=0 tx=0).stop 2024-04-20T10:21:02.602 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.601+0000 7f969ee00640 1 -- 172.21.15.121:0/3133623786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f969815b960 msgr2=0x7f969815bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:02.602 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.601+0000 7f969ee00640 1 --2- 172.21.15.121:0/3133623786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f969815b960 0x7f969815bdf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f96980ad6d0 tx=0x7f969000f520 comp rx=0 tx=0).stop 2024-04-20T10:21:02.602 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.601+0000 7f969ee00640 1 -- 172.21.15.121:0/3133623786 shutdown_connections 2024-04-20T10:21:02.602 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.601+0000 7f969ee00640 1 --2- 172.21.15.121:0/3133623786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f969815b960 0x7f969815bdf0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:02.602 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.601+0000 7f969ee00640 1 --2- 172.21.15.121:0/3133623786 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f967406c8f0 0x7f967406edb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:02.602 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.602+0000 7f969ee00640 1 --2- 172.21.15.121:0/3133623786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96980c6df0 0x7f96980c7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:02.602 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.602+0000 7f969ee00640 1 --2- 172.21.15.121:0/3133623786 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f967c004800 0x7f969814aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:02.602 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.602+0000 7f969ee00640 1 -- 172.21.15.121:0/3133623786 >> 172.21.15.121:0/3133623786 conn(0x7f96980b3350 msgr2=0x7f9698158e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:02.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.602+0000 7f969ee00640 1 -- 172.21.15.121:0/3133623786 shutdown_connections 2024-04-20T10:21:02.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.602+0000 7f969ee00640 1 -- 172.21.15.121:0/3133623786 wait complete. 2024-04-20T10:21:02.642 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-20T10:21:02.642 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-20T10:21:02.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.821+0000 7f38de200640 1 Processor -- start 2024-04-20T10:21:02.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.821+0000 7f38de200640 1 -- start start 2024-04-20T10:21:02.823 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.822+0000 7f38de200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f38d806dcb0 0x7f38d8071b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:02.823 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.823+0000 7f38de200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f38d8072090 0x7f38d80d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:02.823 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.823+0000 7f38de200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38d80ade40 con 0x7f38d815b3b0 2024-04-20T10:21:02.823 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.823+0000 7f38de200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f38d806f1b0 con 0x7f38d8072090 2024-04-20T10:21:02.823 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.823+0000 7f38de200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f38d80ae0e0 con 0x7f38d806dcb0 2024-04-20T10:21:02.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.823+0000 7f38d7e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f38d8072090 0x7f38d80d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:02.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.823+0000 7f38d7e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f38d8072090 0x7f38d80d0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44236/0 (socket says 172.21.15.121:44236) 2024-04-20T10:21:02.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.823+0000 7f38d7e00640 1 -- 172.21.15.121:0/564375319 learned_addr learned my addr 172.21.15.121:0/564375319 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:02.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.823+0000 7f38d7e00640 1 -- 172.21.15.121:0/564375319 >> v2:172.21.15.123:3300/0 conn(0x7f38d806dcb0 msgr2=0x7f38d8071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:02.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.823+0000 7f38d7e00640 1 --2- 172.21.15.121:0/564375319 >> v2:172.21.15.123:3300/0 conn(0x7f38d806dcb0 0x7f38d8071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:02.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.823+0000 7f38d7e00640 1 -- 172.21.15.121:0/564375319 >> v1:172.21.15.123:6789/0 conn(0x7f38d815b3b0 legacy=0x7f38d806eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:02.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.823+0000 7f38d7e00640 1 -- 172.21.15.121:0/564375319 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38d815b790 con 0x7f38d8072090 2024-04-20T10:21:02.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.824+0000 7f38d7e00640 1 --2- 172.21.15.121:0/564375319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38d8072090 0x7f38d80d0360 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f38d80ad8d0 tx=0x7f38d002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626de602b07afb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:02.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.825+0000 7f38d6a00640 1 -- 172.21.15.121:0/564375319 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d002ca80 con 0x7f38d8072090 2024-04-20T10:21:02.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.825+0000 7f38d6a00640 1 -- 172.21.15.121:0/564375319 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38d002cc20 con 0x7f38d8072090 2024-04-20T10:21:02.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.825+0000 7f38d6a00640 1 -- 172.21.15.121:0/564375319 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d00343f0 con 0x7f38d8072090 2024-04-20T10:21:02.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.825+0000 7f38de200640 1 -- 172.21.15.121:0/564375319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38d8072090 msgr2=0x7f38d80d0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:02.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.825+0000 7f38de200640 1 --2- 172.21.15.121:0/564375319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38d8072090 0x7f38d80d0360 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f38d80ad8d0 tx=0x7f38d002bee0 comp rx=0 tx=0).stop 2024-04-20T10:21:02.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.825+0000 7f38de200640 1 -- 172.21.15.121:0/564375319 shutdown_connections 2024-04-20T10:21:02.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.825+0000 7f38de200640 1 --2- 172.21.15.121:0/564375319 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38d8072090 0x7f38d80d0360 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:02.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.825+0000 7f38de200640 1 --2- 172.21.15.121:0/564375319 >> v2:172.21.15.123:3300/0 conn(0x7f38d806dcb0 0x7f38d8071b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:02.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.825+0000 7f38de200640 1 -- 172.21.15.121:0/564375319 >> 172.21.15.121:0/564375319 conn(0x7f38d80b3350 msgr2=0x7f38d80b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:02.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.825+0000 7f38de200640 1 -- 172.21.15.121:0/564375319 shutdown_connections 2024-04-20T10:21:02.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.825+0000 7f38de200640 1 -- 172.21.15.121:0/564375319 wait complete. 2024-04-20T10:21:02.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.827+0000 7f38de200640 1 Processor -- start 2024-04-20T10:21:02.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.827+0000 7f38de200640 1 -- start start 2024-04-20T10:21:02.829 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.828+0000 7f38de200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38d806dcb0 0x7f38d809dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:02.829 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.828+0000 7f38de200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38d815b3b0 0x7f38d81f1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:02.829 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.828+0000 7f38de200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38d815b9b0 0x7f38d81f2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:02.829 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.828+0000 7f38de200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f38d80ae940 con 0x7f38d815b3b0 2024-04-20T10:21:02.829 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.828+0000 7f38de200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f38d80ae660 con 0x7f38d806dcb0 2024-04-20T10:21:02.829 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.828+0000 7f38de200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f38d80aec20 con 0x7f38d815b9b0 2024-04-20T10:21:02.829 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.828+0000 7f38cea00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38d815b3b0 0x7f38d81f1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:02.829 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.828+0000 7f38cea00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38d815b3b0 0x7f38d81f1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44238/0 (socket says 172.21.15.121:44238) 2024-04-20T10:21:02.829 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.829+0000 7f38cea00640 1 -- 172.21.15.121:0/2139568750 learned_addr learned my addr 172.21.15.121:0/2139568750 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:02.830 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.829+0000 7f38d7e00640 1 --2- 172.21.15.121:0/2139568750 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38d815b9b0 0x7f38d81f2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:02.830 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.829+0000 7f38d7400640 1 --2- 172.21.15.121:0/2139568750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38d806dcb0 0x7f38d809dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:02.831 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.829+0000 7f38cea00640 1 -- 172.21.15.121:0/2139568750 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38d815b9b0 msgr2=0x7f38d81f2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:02.831 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.829+0000 7f38cea00640 1 --2- 172.21.15.121:0/2139568750 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38d815b9b0 0x7f38d81f2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:02.831 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.829+0000 7f38cea00640 1 -- 172.21.15.121:0/2139568750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38d806dcb0 msgr2=0x7f38d809dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:02.831 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.829+0000 7f38cea00640 1 --2- 172.21.15.121:0/2139568750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38d806dcb0 0x7f38d809dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:02.831 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.829+0000 7f38cea00640 1 -- 172.21.15.121:0/2139568750 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38d81477e0 con 0x7f38d815b3b0 2024-04-20T10:21:02.831 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.829+0000 7f38cea00640 1 --2- 172.21.15.121:0/2139568750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38d815b3b0 0x7f38d81f1fe0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f38c000b4a0 tx=0x7f38c000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:02.831 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.830+0000 7f38cfe00640 1 -- 172.21.15.121:0/2139568750 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c0014070 con 0x7f38d815b3b0 2024-04-20T10:21:02.831 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.830+0000 7f38cfe00640 1 -- 172.21.15.121:0/2139568750 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38c0010610 con 0x7f38d815b3b0 2024-04-20T10:21:02.831 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.830+0000 7f38cfe00640 1 -- 172.21.15.121:0/2139568750 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c0010a00 con 0x7f38d815b3b0 2024-04-20T10:21:02.831 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.830+0000 7f38de200640 1 -- 172.21.15.121:0/2139568750 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38d81f2b50 con 0x7f38d815b3b0 2024-04-20T10:21:02.831 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.830+0000 7f38de200640 1 -- 172.21.15.121:0/2139568750 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38d8151ea0 con 0x7f38d815b3b0 2024-04-20T10:21:02.832 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.831+0000 7f38de200640 1 -- 172.21.15.121:0/2139568750 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38d8068980 con 0x7f38d815b3b0 2024-04-20T10:21:02.832 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.832+0000 7f38cfe00640 1 -- 172.21.15.121:0/2139568750 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f38c0017070 con 0x7f38d815b3b0 2024-04-20T10:21:02.833 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.833+0000 7f38cfe00640 1 --2- 172.21.15.121:0/2139568750 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f38b406c8f0 0x7f38b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:02.833 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.833+0000 7f38cfe00640 1 -- 172.21.15.121:0/2139568750 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f38c008b600 con 0x7f38d815b3b0 2024-04-20T10:21:02.834 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.833+0000 7f38d7400640 1 --2- 172.21.15.121:0/2139568750 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f38b406c8f0 0x7f38b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:02.834 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.834+0000 7f38d7400640 1 --2- 172.21.15.121:0/2139568750 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f38b406c8f0 0x7f38b406edb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f38c8006fd0 tx=0x7f38c8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:02.839 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:02.838+0000 7f38cfe00640 1 -- 172.21.15.121:0/2139568750 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f38c0059a20 con 0x7f38d815b3b0 2024-04-20T10:21:03.042 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.042+0000 7f38de200640 1 -- 172.21.15.121:0/2139568750 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f38d80cf320 con 0x7f38d815b3b0 2024-04-20T10:21:03.555 INFO:tasks.ceph.mds.d.smithi123.stderr:2024-04-20T10:21:03.555+0000 7f52c3800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T10:21:03.633 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.632+0000 7f38cfe00640 1 -- 172.21.15.121:0/2139568750 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) ==== 157+0+0 (secure 0 0 0) 0x7f38c0059320 con 0x7f38d815b3b0 2024-04-20T10:21:03.636 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.636+0000 7f38de200640 1 -- 172.21.15.121:0/2139568750 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f38b406c8f0 msgr2=0x7f38b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:03.637 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.636+0000 7f38de200640 1 --2- 172.21.15.121:0/2139568750 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f38b406c8f0 0x7f38b406edb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f38c8006fd0 tx=0x7f38c8008040 comp rx=0 tx=0).stop 2024-04-20T10:21:03.637 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.636+0000 7f38de200640 1 -- 172.21.15.121:0/2139568750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38d815b3b0 msgr2=0x7f38d81f1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:03.637 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.636+0000 7f38de200640 1 --2- 172.21.15.121:0/2139568750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38d815b3b0 0x7f38d81f1fe0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f38c000b4a0 tx=0x7f38c000b960 comp rx=0 tx=0).stop 2024-04-20T10:21:03.637 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.637+0000 7f38de200640 1 -- 172.21.15.121:0/2139568750 shutdown_connections 2024-04-20T10:21:03.637 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.637+0000 7f38de200640 1 --2- 172.21.15.121:0/2139568750 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f38d815b9b0 0x7f38d81f2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:03.637 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.637+0000 7f38de200640 1 --2- 172.21.15.121:0/2139568750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f38d815b3b0 0x7f38d81f1fe0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:03.637 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.637+0000 7f38de200640 1 --2- 172.21.15.121:0/2139568750 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f38b406c8f0 0x7f38b406edb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:03.637 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.637+0000 7f38de200640 1 --2- 172.21.15.121:0/2139568750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38d806dcb0 0x7f38d809dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:03.638 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.637+0000 7f38de200640 1 -- 172.21.15.121:0/2139568750 >> 172.21.15.121:0/2139568750 conn(0x7f38d80b3350 msgr2=0x7f38d80b3730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:03.638 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.637+0000 7f38de200640 1 -- 172.21.15.121:0/2139568750 shutdown_connections 2024-04-20T10:21:03.638 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.637+0000 7f38de200640 1 -- 172.21.15.121:0/2139568750 wait complete. 2024-04-20T10:21:03.659 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-20T10:21:03.859 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.858+0000 7f0ef0600640 1 Processor -- start 2024-04-20T10:21:03.859 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.858+0000 7f0ef0600640 1 -- start start 2024-04-20T10:21:03.860 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.860+0000 7f0ef0600640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ee806eb90 0x7f0ee806f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:03.860 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.860+0000 7f0ef0600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ee80ccf10 con 0x7f0ee80cc8a0 2024-04-20T10:21:03.861 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.860+0000 7f0ef0600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ee80cd1f0 con 0x7f0ee815c4e0 2024-04-20T10:21:03.861 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.860+0000 7f0ef0600640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f0ee80ca800 con 0x7f0ee806eb90 2024-04-20T10:21:03.861 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.860+0000 7f0eed200640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f0ee80cc8a0 0x7f0ee80caf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:58870/0 (socket says 172.21.15.121:58870) 2024-04-20T10:21:03.861 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.860+0000 7f0eed200640 1 -- 172.21.15.121:0/3804317132 learned_addr learned my addr 172.21.15.121:0/3804317132 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:03.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.860+0000 7f0ee7e00640 1 -- 172.21.15.121:0/3804317132 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 453601428 0 0) 0x7f0ee80ccf10 con 0x7f0ee80cc8a0 2024-04-20T10:21:03.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.860+0000 7f0ee7e00640 1 -- 172.21.15.121:0/3804317132 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ed0003520 con 0x7f0ee80cc8a0 2024-04-20T10:21:03.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.861+0000 7f0eee600640 1 --2- 172.21.15.121:0/3804317132 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ee806eb90 0x7f0ee806f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:03.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.861+0000 7f0ee7e00640 1 -- 172.21.15.121:0/3804317132 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3987745626 0 0) 0x7f0ed0003520 con 0x7f0ee80cc8a0 2024-04-20T10:21:03.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.861+0000 7f0ee7e00640 1 -- 172.21.15.121:0/3804317132 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ee806eb90 msgr2=0x7f0ee806f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:03.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.861+0000 7f0ee7e00640 1 --2- 172.21.15.121:0/3804317132 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ee806eb90 0x7f0ee806f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:03.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.861+0000 7f0ee7e00640 1 -- 172.21.15.121:0/3804317132 >> v1:172.21.15.123:6789/0 conn(0x7f0ee815c4e0 legacy=0x7f0ee8160ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:03.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.861+0000 7f0ee7e00640 1 -- 172.21.15.121:0/3804317132 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ee80cd4d0 con 0x7f0ee80cc8a0 2024-04-20T10:21:03.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.861+0000 7f0ee7e00640 1 -- 172.21.15.121:0/3804317132 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f0ed4001670 con 0x7f0ee80cc8a0 2024-04-20T10:21:03.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.861+0000 7f0eee600640 1 --2- 172.21.15.121:0/3804317132 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ee806eb90 0x7f0ee806f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:03.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.861+0000 7f0ee7e00640 1 -- 172.21.15.121:0/3804317132 >> v1:172.21.15.121:6789/0 conn(0x7f0ee80cc8a0 legacy=0x7f0ee80caf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:03.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.861+0000 7f0ee7e00640 1 --2- 172.21.15.121:0/3804317132 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ed00039e0 0x7f0ed0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:03.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.861+0000 7f0ee7e00640 1 --2- 172.21.15.121:0/3804317132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ed0004800 0x7f0ed0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:03.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.861+0000 7f0ee7e00640 1 --2- 172.21.15.121:0/3804317132 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ee806eb90 0x7f0ed000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:03.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.861+0000 7f0ee7e00640 1 -- 172.21.15.121:0/3804317132 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee80ca800 con 0x7f0ed00039e0 2024-04-20T10:21:03.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.861+0000 7f0ee7e00640 1 -- 172.21.15.121:0/3804317132 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed00055c0 con 0x7f0ed0004800 2024-04-20T10:21:03.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.861+0000 7f0ee7e00640 1 -- 172.21.15.121:0/3804317132 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f0ed000bf00 con 0x7f0ee806eb90 2024-04-20T10:21:03.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.861+0000 7f0eedc00640 1 --2- 172.21.15.121:0/3804317132 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ed00039e0 0x7f0ed0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:03.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.862+0000 7f0eee600640 1 --2- 172.21.15.121:0/3804317132 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ee806eb90 0x7f0ed000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:03.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.862+0000 7f0eedc00640 1 -- 172.21.15.121:0/3804317132 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ee806eb90 msgr2=0x7f0ed000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:03.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.862+0000 7f0eedc00640 1 --2- 172.21.15.121:0/3804317132 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ee806eb90 0x7f0ed000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:03.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.862+0000 7f0eed200640 1 --2- 172.21.15.121:0/3804317132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ed0004800 0x7f0ed0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:03.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.862+0000 7f0eedc00640 1 -- 172.21.15.121:0/3804317132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ed0004800 msgr2=0x7f0ed0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:03.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.862+0000 7f0eedc00640 1 --2- 172.21.15.121:0/3804317132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ed0004800 0x7f0ed0004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:03.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.862+0000 7f0eedc00640 1 -- 172.21.15.121:0/3804317132 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed0003520 con 0x7f0ed00039e0 2024-04-20T10:21:03.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.862+0000 7f0eee600640 1 --2- 172.21.15.121:0/3804317132 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ee806eb90 0x7f0ed000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:03.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.863+0000 7f0eedc00640 1 --2- 172.21.15.121:0/3804317132 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ed00039e0 0x7f0ed0003de0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0edc003db0 tx=0x7f0ed0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35c19b451c6beb62 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:03.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.863+0000 7f0ee7e00640 1 -- 172.21.15.121:0/3804317132 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0edc037070 con 0x7f0ed00039e0 2024-04-20T10:21:03.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.863+0000 7f0ee7e00640 1 -- 172.21.15.121:0/3804317132 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0edc032600 con 0x7f0ed00039e0 2024-04-20T10:21:03.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.863+0000 7f0ee7e00640 1 -- 172.21.15.121:0/3804317132 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0edc003980 con 0x7f0ed00039e0 2024-04-20T10:21:03.869 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.864+0000 7f0ef0600640 1 -- 172.21.15.121:0/3804317132 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ed00039e0 msgr2=0x7f0ed0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:03.869 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.864+0000 7f0ef0600640 1 --2- 172.21.15.121:0/3804317132 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ed00039e0 0x7f0ed0003de0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0edc003db0 tx=0x7f0ed0005830 comp rx=0 tx=0).stop 2024-04-20T10:21:03.869 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.864+0000 7f0eee600640 1 -- 172.21.15.121:0/3804317132 reap_dead start 2024-04-20T10:21:03.869 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.864+0000 7f0ef0600640 1 -- 172.21.15.121:0/3804317132 shutdown_connections 2024-04-20T10:21:03.869 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.864+0000 7f0ef0600640 1 -- 172.21.15.121:0/3804317132 >> 172.21.15.121:0/3804317132 conn(0x7f0ee80b1e80 msgr2=0x7f0ee80b2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:03.869 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.864+0000 7f0ef0600640 1 -- 172.21.15.121:0/3804317132 shutdown_connections 2024-04-20T10:21:03.869 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.864+0000 7f0ef0600640 1 -- 172.21.15.121:0/3804317132 wait complete. 2024-04-20T10:21:03.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.866+0000 7f0ef0600640 1 Processor -- start 2024-04-20T10:21:03.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.866+0000 7f0ef0600640 1 -- start start 2024-04-20T10:21:03.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.867+0000 7f0ef0600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ed00039e0 0x7f0ee80ae420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:03.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.867+0000 7f0ef0600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ee809da10 0x7f0ee815ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:03.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.867+0000 7f0ef0600640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ee809de90 0x7f0ee815f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:03.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.867+0000 7f0ef0600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee80cb680 con 0x7f0ee809da10 2024-04-20T10:21:03.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.867+0000 7f0ef0600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee80caae0 con 0x7f0ed00039e0 2024-04-20T10:21:03.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.867+0000 7f0ef0600640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f0ee80cb800 con 0x7f0ee809de90 2024-04-20T10:21:03.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.867+0000 7f0eed200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ee809da10 0x7f0ee815ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:03.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.867+0000 7f0eed200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ee809da10 0x7f0ee815ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44248/0 (socket says 172.21.15.121:44248) 2024-04-20T10:21:03.871 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.867+0000 7f0eed200640 1 -- 172.21.15.121:0/2126285380 learned_addr learned my addr 172.21.15.121:0/2126285380 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:03.871 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.867+0000 7f0eed200640 1 -- 172.21.15.121:0/2126285380 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ee809de90 msgr2=0x7f0ee815f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:03.871 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.867+0000 7f0eed200640 1 --2- 172.21.15.121:0/2126285380 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ee809de90 0x7f0ee815f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:03.871 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.867+0000 7f0eed200640 1 -- 172.21.15.121:0/2126285380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ed00039e0 msgr2=0x7f0ee80ae420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:03.871 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.867+0000 7f0eed200640 1 --2- 172.21.15.121:0/2126285380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ed00039e0 0x7f0ee80ae420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:03.871 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.867+0000 7f0eed200640 1 -- 172.21.15.121:0/2126285380 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ee81f2fa0 con 0x7f0ee809da10 2024-04-20T10:21:03.871 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.867+0000 7f0eed200640 1 --2- 172.21.15.121:0/2126285380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ee809da10 0x7f0ee815ef90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0ed400d7d0 tx=0x7f0ed400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:03.871 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.868+0000 7f0ee5600640 1 -- 172.21.15.121:0/2126285380 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed4019070 con 0x7f0ee809da10 2024-04-20T10:21:03.871 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.868+0000 7f0ee5600640 1 -- 172.21.15.121:0/2126285380 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ed40074e0 con 0x7f0ee809da10 2024-04-20T10:21:03.871 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.868+0000 7f0ee5600640 1 -- 172.21.15.121:0/2126285380 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed4004a00 con 0x7f0ee809da10 2024-04-20T10:21:03.871 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.868+0000 7f0ef0600640 1 -- 172.21.15.121:0/2126285380 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ee809cf30 con 0x7f0ee809da10 2024-04-20T10:21:03.871 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.868+0000 7f0ef0600640 1 -- 172.21.15.121:0/2126285380 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ee809d4f0 con 0x7f0ee809da10 2024-04-20T10:21:03.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.869+0000 7f0ef0600640 1 -- 172.21.15.121:0/2126285380 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ee8068980 con 0x7f0ee809da10 2024-04-20T10:21:03.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.870+0000 7f0ee5600640 1 -- 172.21.15.121:0/2126285380 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ed401c070 con 0x7f0ee809da10 2024-04-20T10:21:03.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.871+0000 7f0ee5600640 1 --2- 172.21.15.121:0/2126285380 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f0ec806c970 0x7f0ec806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:03.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.871+0000 7f0ee5600640 1 -- 172.21.15.121:0/2126285380 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 6007+0+0 (secure 0 0 0) 0x7f0ed408b670 con 0x7f0ee809da10 2024-04-20T10:21:03.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.871+0000 7f0eedc00640 1 --2- 172.21.15.121:0/2126285380 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f0ec806c970 0x7f0ec806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:03.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.872+0000 7f0eedc00640 1 --2- 172.21.15.121:0/2126285380 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f0ec806c970 0x7f0ec806ee30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0edc034000 tx=0x7f0edc02f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:03.875 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:03.875+0000 7f0ee5600640 1 -- 172.21.15.121:0/2126285380 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0ed40593c0 con 0x7f0ee809da10 2024-04-20T10:21:04.079 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:04.079+0000 7f0ef0600640 1 -- 172.21.15.121:0/2126285380 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f0ee815bc90 con 0x7f0ee809da10 2024-04-20T10:21:05.852 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:05.851+0000 7f0ee5600640 1 -- 172.21.15.121:0/2126285380 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) ==== 228+0+0 (secure 0 0 0) 0x7f0ed405d6c0 con 0x7f0ee809da10 2024-04-20T10:21:05.852 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'cephfs_data_ec' created 2024-04-20T10:21:05.856 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:05.856+0000 7f0ef0600640 1 -- 172.21.15.121:0/2126285380 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f0ec806c970 msgr2=0x7f0ec806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:05.856 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:05.856+0000 7f0ef0600640 1 --2- 172.21.15.121:0/2126285380 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f0ec806c970 0x7f0ec806ee30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0edc034000 tx=0x7f0edc02f000 comp rx=0 tx=0).stop 2024-04-20T10:21:05.857 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:05.856+0000 7f0ef0600640 1 -- 172.21.15.121:0/2126285380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ee809da10 msgr2=0x7f0ee815ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:05.857 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:05.856+0000 7f0ef0600640 1 --2- 172.21.15.121:0/2126285380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ee809da10 0x7f0ee815ef90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0ed400d7d0 tx=0x7f0ed400dc90 comp rx=0 tx=0).stop 2024-04-20T10:21:05.857 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:05.856+0000 7f0ef0600640 1 -- 172.21.15.121:0/2126285380 shutdown_connections 2024-04-20T10:21:05.857 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:05.856+0000 7f0ef0600640 1 --2- 172.21.15.121:0/2126285380 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0ee809de90 0x7f0ee815f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:05.857 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:05.856+0000 7f0ef0600640 1 --2- 172.21.15.121:0/2126285380 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0ee809da10 0x7f0ee815ef90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:05.857 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:05.856+0000 7f0ef0600640 1 --2- 172.21.15.121:0/2126285380 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f0ec806c970 0x7f0ec806ee30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:05.857 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:05.856+0000 7f0ef0600640 1 --2- 172.21.15.121:0/2126285380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ed00039e0 0x7f0ee80ae420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:05.857 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:05.856+0000 7f0ef0600640 1 -- 172.21.15.121:0/2126285380 >> 172.21.15.121:0/2126285380 conn(0x7f0ee80b1e80 msgr2=0x7f0ee815a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:05.858 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:05.857+0000 7f0ef0600640 1 -- 172.21.15.121:0/2126285380 shutdown_connections 2024-04-20T10:21:05.858 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:05.857+0000 7f0ef0600640 1 -- 172.21.15.121:0/2126285380 wait complete. 2024-04-20T10:21:05.884 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-20T10:21:06.063 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.063+0000 7f59bfe00640 1 Processor -- start 2024-04-20T10:21:06.063 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.063+0000 7f59bfe00640 1 -- start start 2024-04-20T10:21:06.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.064+0000 7f59bfe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f59b8156c20 0x7f59b806b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:06.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.064+0000 7f59bfe00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59b80d0360 0x7f59b80d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:06.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.064+0000 7f59bfe00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59b8141c60 con 0x7f59b815ca40 2024-04-20T10:21:06.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.064+0000 7f59bfe00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f59b80adab0 con 0x7f59b8156c20 2024-04-20T10:21:06.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.064+0000 7f59bfe00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f59b815cf90 con 0x7f59b80d0360 2024-04-20T10:21:06.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.065+0000 7f59bde00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59b80d0360 0x7f59b80d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:06.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.065+0000 7f59bde00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59b80d0360 0x7f59b80d07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.121:47680/0 (socket says 172.21.15.121:47680) 2024-04-20T10:21:06.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.065+0000 7f59bde00640 1 -- 172.21.15.121:0/1887711286 learned_addr learned my addr 172.21.15.121:0/1887711286 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:06.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.065+0000 7f59bd400640 1 --2- 172.21.15.121:0/1887711286 >> v2:172.21.15.123:3300/0 conn(0x7f59b8156c20 0x7f59b806b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:06.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.066+0000 7f59bde00640 1 -- 172.21.15.121:0/1887711286 >> v2:172.21.15.123:3300/0 conn(0x7f59b8156c20 msgr2=0x7f59b806b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:06.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.066+0000 7f59bde00640 1 --2- 172.21.15.121:0/1887711286 >> v2:172.21.15.123:3300/0 conn(0x7f59b8156c20 0x7f59b806b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:06.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.066+0000 7f59bde00640 1 -- 172.21.15.121:0/1887711286 >> v1:172.21.15.123:6789/0 conn(0x7f59b815ca40 legacy=0x7f59b80ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:06.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.066+0000 7f59bde00640 1 -- 172.21.15.121:0/1887711286 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b815d270 con 0x7f59b80d0360 2024-04-20T10:21:06.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.067+0000 7f59bde00640 1 --2- 172.21.15.121:0/1887711286 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59b80d0360 0x7f59b80d07f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f59b0009e50 tx=0x7f59b002bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e801dcb4047ce34 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:06.068 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.068+0000 7f59b7e00640 1 -- 172.21.15.121:0/1887711286 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b002cb00 con 0x7f59b80d0360 2024-04-20T10:21:06.068 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.068+0000 7f59b7e00640 1 -- 172.21.15.121:0/1887711286 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59b002cca0 con 0x7f59b80d0360 2024-04-20T10:21:06.068 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.068+0000 7f59b7e00640 1 -- 172.21.15.121:0/1887711286 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b0034610 con 0x7f59b80d0360 2024-04-20T10:21:06.069 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.068+0000 7f59bfe00640 1 -- 172.21.15.121:0/1887711286 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59b80d0360 msgr2=0x7f59b80d07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:06.069 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.068+0000 7f59bfe00640 1 --2- 172.21.15.121:0/1887711286 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59b80d0360 0x7f59b80d07f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f59b0009e50 tx=0x7f59b002bf60 comp rx=0 tx=0).stop 2024-04-20T10:21:06.069 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.068+0000 7f59bfe00640 1 -- 172.21.15.121:0/1887711286 shutdown_connections 2024-04-20T10:21:06.069 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.068+0000 7f59bfe00640 1 --2- 172.21.15.121:0/1887711286 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59b80d0360 0x7f59b80d07f0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:06.069 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.068+0000 7f59bfe00640 1 --2- 172.21.15.121:0/1887711286 >> v2:172.21.15.123:3300/0 conn(0x7f59b8156c20 0x7f59b806b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:06.069 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.068+0000 7f59bfe00640 1 -- 172.21.15.121:0/1887711286 >> 172.21.15.121:0/1887711286 conn(0x7f59b80b3350 msgr2=0x7f59b80b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:06.069 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.068+0000 7f59bfe00640 1 -- 172.21.15.121:0/1887711286 shutdown_connections 2024-04-20T10:21:06.069 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.069+0000 7f59bfe00640 1 -- 172.21.15.121:0/1887711286 wait complete. 2024-04-20T10:21:06.071 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.070+0000 7f59bfe00640 1 Processor -- start 2024-04-20T10:21:06.071 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.070+0000 7f59bfe00640 1 -- start start 2024-04-20T10:21:06.072 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.071+0000 7f59bfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59b80d0360 0x7f59b81e5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:06.073 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.071+0000 7f59bfe00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59b8156c20 0x7f59b80cf200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:06.073 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.071+0000 7f59bfe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59b815ca40 0x7f59b814eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:06.073 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.071+0000 7f59bfe00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f59b806be40 con 0x7f59b815ca40 2024-04-20T10:21:06.073 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.071+0000 7f59bfe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f59b815d830 con 0x7f59b80d0360 2024-04-20T10:21:06.073 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.071+0000 7f59bfe00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f59b806bfc0 con 0x7f59b8156c20 2024-04-20T10:21:06.073 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.072+0000 7f59bde00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59b815ca40 0x7f59b814eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:06.073 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.072+0000 7f59bde00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59b815ca40 0x7f59b814eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:44260/0 (socket says 172.21.15.121:44260) 2024-04-20T10:21:06.073 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.072+0000 7f59bde00640 1 -- 172.21.15.121:0/2009182668 learned_addr learned my addr 172.21.15.121:0/2009182668 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:06.073 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.072+0000 7f59bd400640 1 --2- 172.21.15.121:0/2009182668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59b80d0360 0x7f59b81e5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:06.073 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.072+0000 7f59bca00640 1 --2- 172.21.15.121:0/2009182668 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59b8156c20 0x7f59b80cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:06.073 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.072+0000 7f59bde00640 1 -- 172.21.15.121:0/2009182668 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59b8156c20 msgr2=0x7f59b80cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:06.073 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.072+0000 7f59bde00640 1 --2- 172.21.15.121:0/2009182668 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59b8156c20 0x7f59b80cf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:06.074 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.072+0000 7f59bde00640 1 -- 172.21.15.121:0/2009182668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59b80d0360 msgr2=0x7f59b81e5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:06.074 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.072+0000 7f59bde00640 1 --2- 172.21.15.121:0/2009182668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59b80d0360 0x7f59b81e5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:06.074 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.072+0000 7f59bde00640 1 -- 172.21.15.121:0/2009182668 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b8152310 con 0x7f59b815ca40 2024-04-20T10:21:06.074 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.072+0000 7f59bd400640 1 --2- 172.21.15.121:0/2009182668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59b80d0360 0x7f59b81e5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:21:06.074 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.072+0000 7f59bde00640 1 --2- 172.21.15.121:0/2009182668 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59b815ca40 0x7f59b814eda0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f59b0036040 tx=0x7f59b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:06.074 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.073+0000 7f59b5600640 1 -- 172.21.15.121:0/2009182668 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b0043020 con 0x7f59b815ca40 2024-04-20T10:21:06.075 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.073+0000 7f59b5600640 1 -- 172.21.15.121:0/2009182668 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59b002ce50 con 0x7f59b815ca40 2024-04-20T10:21:06.075 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.073+0000 7f59b5600640 1 -- 172.21.15.121:0/2009182668 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b0009510 con 0x7f59b815ca40 2024-04-20T10:21:06.075 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.073+0000 7f59bfe00640 1 -- 172.21.15.121:0/2009182668 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59b81525f0 con 0x7f59b815ca40 2024-04-20T10:21:06.075 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.073+0000 7f59bfe00640 1 -- 172.21.15.121:0/2009182668 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59b81e8810 con 0x7f59b815ca40 2024-04-20T10:21:06.075 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.074+0000 7f59bfe00640 1 -- 172.21.15.121:0/2009182668 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59b80704f0 con 0x7f59b815ca40 2024-04-20T10:21:06.075 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.075+0000 7f59b5600640 1 -- 172.21.15.121:0/2009182668 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f59b0046020 con 0x7f59b815ca40 2024-04-20T10:21:06.076 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.076+0000 7f59b5600640 1 --2- 172.21.15.121:0/2009182668 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f599c06c8f0 0x7f599c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:06.077 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.076+0000 7f59b5600640 1 -- 172.21.15.121:0/2009182668 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7f59b00ae430 con 0x7f59b815ca40 2024-04-20T10:21:06.077 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.077+0000 7f59bd400640 1 --2- 172.21.15.121:0/2009182668 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f599c06c8f0 0x7f599c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:06.078 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.077+0000 7f59bd400640 1 --2- 172.21.15.121:0/2009182668 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f599c06c8f0 0x7f599c06edb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f59ac006fd0 tx=0x7f59ac008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:06.081 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.081+0000 7f59b5600640 1 -- 172.21.15.121:0/2009182668 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f59b80704f0 con 0x7f59b815ca40 2024-04-20T10:21:06.283 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.283+0000 7f59bfe00640 1 -- 172.21.15.121:0/2009182668 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f59b8068980 con 0x7f59b815ca40 2024-04-20T10:21:06.857 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.856+0000 7f59b5600640 1 -- 172.21.15.121:0/2009182668 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) ==== 169+0+0 (secure 0 0 0) 0x7f59b004d030 con 0x7f59b815ca40 2024-04-20T10:21:06.857 INFO:teuthology.orchestra.run.smithi121.stderr:set pool 4 allow_ec_overwrites to true 2024-04-20T10:21:06.861 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.860+0000 7f59bfe00640 1 -- 172.21.15.121:0/2009182668 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f599c06c8f0 msgr2=0x7f599c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:06.861 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.860+0000 7f59bfe00640 1 --2- 172.21.15.121:0/2009182668 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f599c06c8f0 0x7f599c06edb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f59ac006fd0 tx=0x7f59ac008040 comp rx=0 tx=0).stop 2024-04-20T10:21:06.861 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.861+0000 7f59bfe00640 1 -- 172.21.15.121:0/2009182668 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59b815ca40 msgr2=0x7f59b814eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:06.861 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.861+0000 7f59bfe00640 1 --2- 172.21.15.121:0/2009182668 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59b815ca40 0x7f59b814eda0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f59b0036040 tx=0x7f59b0004a40 comp rx=0 tx=0).stop 2024-04-20T10:21:06.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.862+0000 7f59bfe00640 1 -- 172.21.15.121:0/2009182668 shutdown_connections 2024-04-20T10:21:06.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.862+0000 7f59bfe00640 1 --2- 172.21.15.121:0/2009182668 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59b815ca40 0x7f59b814eda0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:06.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.862+0000 7f59bfe00640 1 --2- 172.21.15.121:0/2009182668 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59b8156c20 0x7f59b80cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:06.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.862+0000 7f59bfe00640 1 --2- 172.21.15.121:0/2009182668 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f599c06c8f0 0x7f599c06edb0 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f59ac006fd0 tx=0x7f59ac008040 comp rx=0 tx=0).stop 2024-04-20T10:21:06.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.862+0000 7f59bfe00640 1 --2- 172.21.15.121:0/2009182668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59b80d0360 0x7f59b81e5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:06.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.862+0000 7f59bfe00640 1 -- 172.21.15.121:0/2009182668 >> 172.21.15.121:0/2009182668 conn(0x7f59b80b3350 msgr2=0x7f59b8073560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:06.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.864+0000 7f59bfe00640 1 -- 172.21.15.121:0/2009182668 shutdown_connections 2024-04-20T10:21:06.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:06.864+0000 7f59bfe00640 1 -- 172.21.15.121:0/2009182668 wait complete. 2024-04-20T10:21:06.910 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-20T10:21:07.086 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.085+0000 7fc49da00640 1 Processor -- start 2024-04-20T10:21:07.086 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.085+0000 7fc49da00640 1 -- start start 2024-04-20T10:21:07.088 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.087+0000 7fc49da00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc49815e8a0 0x7fc49806e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:07.088 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.087+0000 7fc49da00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc49806dbb0 0x7fc4980d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:07.088 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.087+0000 7fc49da00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc49806f060 con 0x7fc4980d08a0 2024-04-20T10:21:07.088 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.087+0000 7fc49da00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fc49806e7c0 con 0x7fc49815e8a0 2024-04-20T10:21:07.088 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.087+0000 7fc49da00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fc49806eaa0 con 0x7fc49806dbb0 2024-04-20T10:21:07.088 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.087+0000 7fc497400640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fc4980d08a0 0x7fc49806b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:35854/0 (socket says 172.21.15.121:35854) 2024-04-20T10:21:07.088 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.087+0000 7fc497400640 1 -- 172.21.15.121:0/213093847 learned_addr learned my addr 172.21.15.121:0/213093847 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:07.088 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.087+0000 7fc496a00640 1 --2- 172.21.15.121:0/213093847 >> v2:172.21.15.123:3300/0 conn(0x7fc49815e8a0 0x7fc49806e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:07.088 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.087+0000 7fc48fe00640 1 --2- 172.21.15.121:0/213093847 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc49806dbb0 0x7fc4980d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:07.088 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.087+0000 7fc496000640 1 -- 172.21.15.121:0/213093847 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3706053366 0 0) 0x7fc49806f060 con 0x7fc4980d08a0 2024-04-20T10:21:07.089 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.087+0000 7fc496000640 1 -- 172.21.15.121:0/213093847 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc478003520 con 0x7fc4980d08a0 2024-04-20T10:21:07.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.088+0000 7fc496000640 1 -- 172.21.15.121:0/213093847 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3151783296 0 0) 0x7fc478003520 con 0x7fc4980d08a0 2024-04-20T10:21:07.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.088+0000 7fc496000640 1 -- 172.21.15.121:0/213093847 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc49806dbb0 msgr2=0x7fc4980d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:07.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.088+0000 7fc496000640 1 --2- 172.21.15.121:0/213093847 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc49806dbb0 0x7fc4980d0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:07.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.088+0000 7fc496000640 1 -- 172.21.15.121:0/213093847 >> v2:172.21.15.123:3300/0 conn(0x7fc49815e8a0 msgr2=0x7fc49806e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:07.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.088+0000 7fc496000640 1 --2- 172.21.15.121:0/213093847 >> v2:172.21.15.123:3300/0 conn(0x7fc49815e8a0 0x7fc49806e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:07.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.088+0000 7fc496000640 1 -- 172.21.15.121:0/213093847 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc49806ed80 con 0x7fc4980d08a0 2024-04-20T10:21:07.093 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.088+0000 7fc496000640 1 -- 172.21.15.121:0/213093847 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7fc4900019f0 con 0x7fc4980d08a0 2024-04-20T10:21:07.093 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.088+0000 7fc496a00640 1 --2- 172.21.15.121:0/213093847 >> v2:172.21.15.123:3300/0 conn(0x7fc49815e8a0 0x7fc49806e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:21:07.093 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.088+0000 7fc496000640 1 -- 172.21.15.121:0/213093847 >> v1:172.21.15.121:6789/0 conn(0x7fc4980d08a0 legacy=0x7fc49806b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:07.093 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.088+0000 7fc496000640 1 --2- 172.21.15.121:0/213093847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4780039b0 0x7fc478003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:07.093 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.088+0000 7fc496000640 1 --2- 172.21.15.121:0/213093847 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc49806dbb0 0x7fc478004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:07.093 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.088+0000 7fc496000640 1 --2- 172.21.15.121:0/213093847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc478004d40 0x7fc4780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:07.093 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.088+0000 7fc496000640 1 -- 172.21.15.121:0/213093847 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc49806e7c0 con 0x7fc4780039b0 2024-04-20T10:21:07.094 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.088+0000 7fc496000640 1 -- 172.21.15.121:0/213093847 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc49806eaa0 con 0x7fc478004d40 2024-04-20T10:21:07.094 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.088+0000 7fc496000640 1 -- 172.21.15.121:0/213093847 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fc47800b9c0 con 0x7fc49806dbb0 2024-04-20T10:21:07.094 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.088+0000 7fc496a00640 1 --2- 172.21.15.121:0/213093847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4780039b0 0x7fc478003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:07.094 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.089+0000 7fc497400640 1 --2- 172.21.15.121:0/213093847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc478004d40 0x7fc4780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:07.094 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.089+0000 7fc48fe00640 1 --2- 172.21.15.121:0/213093847 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc49806dbb0 0x7fc478004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:07.094 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.089+0000 7fc496a00640 1 -- 172.21.15.121:0/213093847 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc49806dbb0 msgr2=0x7fc478004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:07.094 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.089+0000 7fc496a00640 1 --2- 172.21.15.121:0/213093847 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc49806dbb0 0x7fc478004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:07.094 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.089+0000 7fc496a00640 1 -- 172.21.15.121:0/213093847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc478004d40 msgr2=0x7fc4780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:07.095 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.089+0000 7fc496a00640 1 --2- 172.21.15.121:0/213093847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc478004d40 0x7fc4780051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:07.095 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.089+0000 7fc496a00640 1 -- 172.21.15.121:0/213093847 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc478003520 con 0x7fc4780039b0 2024-04-20T10:21:07.095 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.089+0000 7fc497400640 1 --2- 172.21.15.121:0/213093847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc478004d40 0x7fc4780051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:07.095 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.090+0000 7fc496a00640 1 --2- 172.21.15.121:0/213093847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4780039b0 0x7fc478003dd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc488009c90 tx=0x7fc478005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d175b681c88ae44 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:07.095 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.090+0000 7fc496000640 1 -- 172.21.15.121:0/213093847 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc488039070 con 0x7fc4780039b0 2024-04-20T10:21:07.095 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.090+0000 7fc496000640 1 -- 172.21.15.121:0/213093847 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc48802c980 con 0x7fc4780039b0 2024-04-20T10:21:07.095 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.090+0000 7fc496000640 1 -- 172.21.15.121:0/213093847 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc488009860 con 0x7fc4780039b0 2024-04-20T10:21:07.095 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.091+0000 7fc49da00640 1 -- 172.21.15.121:0/213093847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4780039b0 msgr2=0x7fc478003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:07.095 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.091+0000 7fc49da00640 1 --2- 172.21.15.121:0/213093847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4780039b0 0x7fc478003dd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc488009c90 tx=0x7fc478005830 comp rx=0 tx=0).stop 2024-04-20T10:21:07.095 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.091+0000 7fc497400640 1 -- 172.21.15.121:0/213093847 reap_dead start 2024-04-20T10:21:07.095 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.091+0000 7fc49da00640 1 -- 172.21.15.121:0/213093847 shutdown_connections 2024-04-20T10:21:07.095 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.091+0000 7fc49da00640 1 -- 172.21.15.121:0/213093847 >> 172.21.15.121:0/213093847 conn(0x7fc4980b3350 msgr2=0x7fc4980b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:07.095 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.091+0000 7fc49da00640 1 -- 172.21.15.121:0/213093847 shutdown_connections 2024-04-20T10:21:07.096 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.091+0000 7fc49da00640 1 -- 172.21.15.121:0/213093847 wait complete. 2024-04-20T10:21:07.098 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.092+0000 7fc49da00640 1 Processor -- start 2024-04-20T10:21:07.098 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.093+0000 7fc49da00640 1 -- start start 2024-04-20T10:21:07.098 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.093+0000 7fc49da00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4780039b0 0x7fc498155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:07.098 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.094+0000 7fc49da00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc4981f5fb0 0x7fc4981f45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:07.098 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.094+0000 7fc49da00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4980b0f60 0x7fc498161070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:07.098 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.094+0000 7fc49da00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc49806fef0 con 0x7fc4980b0f60 2024-04-20T10:21:07.098 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.094+0000 7fc49da00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc49806fd70 con 0x7fc4780039b0 2024-04-20T10:21:07.099 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.094+0000 7fc49da00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fc4980ae690 con 0x7fc4981f5fb0 2024-04-20T10:21:07.099 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.094+0000 7fc497400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4980b0f60 0x7fc498161070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:07.099 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.094+0000 7fc497400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4980b0f60 0x7fc498161070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42206/0 (socket says 172.21.15.121:42206) 2024-04-20T10:21:07.099 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.094+0000 7fc497400640 1 -- 172.21.15.121:0/2929513223 learned_addr learned my addr 172.21.15.121:0/2929513223 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:07.099 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.094+0000 7fc496a00640 1 --2- 172.21.15.121:0/2929513223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4780039b0 0x7fc498155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:07.099 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.094+0000 7fc48fe00640 1 --2- 172.21.15.121:0/2929513223 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc4981f5fb0 0x7fc4981f45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:07.099 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.094+0000 7fc497400640 1 -- 172.21.15.121:0/2929513223 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc4981f5fb0 msgr2=0x7fc4981f45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:07.099 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.094+0000 7fc497400640 1 --2- 172.21.15.121:0/2929513223 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc4981f5fb0 0x7fc4981f45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:07.099 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.094+0000 7fc497400640 1 -- 172.21.15.121:0/2929513223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4780039b0 msgr2=0x7fc498155c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:07.099 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.094+0000 7fc497400640 1 --2- 172.21.15.121:0/2929513223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4780039b0 0x7fc498155c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:07.099 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.094+0000 7fc497400640 1 -- 172.21.15.121:0/2929513223 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc498156270 con 0x7fc4980b0f60 2024-04-20T10:21:07.099 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.094+0000 7fc48fe00640 1 --2- 172.21.15.121:0/2929513223 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc4981f5fb0 0x7fc4981f45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:07.099 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.095+0000 7fc497400640 1 --2- 172.21.15.121:0/2929513223 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4980b0f60 0x7fc498161070 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc4980719a0 tx=0x7fc49000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:07.100 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.095+0000 7fc48ea00640 1 -- 172.21.15.121:0/2929513223 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc490004d60 con 0x7fc4980b0f60 2024-04-20T10:21:07.100 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.095+0000 7fc48ea00640 1 -- 172.21.15.121:0/2929513223 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc49000bb80 con 0x7fc4980b0f60 2024-04-20T10:21:07.100 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.095+0000 7fc48ea00640 1 -- 172.21.15.121:0/2929513223 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc490007110 con 0x7fc4980b0f60 2024-04-20T10:21:07.100 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.095+0000 7fc49da00640 1 -- 172.21.15.121:0/2929513223 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc498156550 con 0x7fc4980b0f60 2024-04-20T10:21:07.100 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.095+0000 7fc49da00640 1 -- 172.21.15.121:0/2929513223 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc498156b10 con 0x7fc4980b0f60 2024-04-20T10:21:07.100 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.096+0000 7fc49da00640 1 -- 172.21.15.121:0/2929513223 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc498068980 con 0x7fc4980b0f60 2024-04-20T10:21:07.100 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.098+0000 7fc48ea00640 1 -- 172.21.15.121:0/2929513223 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc49001b020 con 0x7fc4980b0f60 2024-04-20T10:21:07.101 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.099+0000 7fc48ea00640 1 --2- 172.21.15.121:0/2929513223 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fc47406c8f0 0x7fc47406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:07.101 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.099+0000 7fc48ea00640 1 -- 172.21.15.121:0/2929513223 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6500+0+0 (secure 0 0 0) 0x7fc49008c110 con 0x7fc4980b0f60 2024-04-20T10:21:07.101 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.099+0000 7fc496a00640 1 --2- 172.21.15.121:0/2929513223 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fc47406c8f0 0x7fc47406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:07.101 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.099+0000 7fc496a00640 1 --2- 172.21.15.121:0/2929513223 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fc47406c8f0 0x7fc47406edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc488036000 tx=0x7fc488030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:07.103 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.103+0000 7fc48ea00640 1 -- 172.21.15.121:0/2929513223 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc49005a180 con 0x7fc4980b0f60 2024-04-20T10:21:07.335 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.334+0000 7fc49da00640 1 -- 172.21.15.121:0/2929513223 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fc49806d160 con 0x7fc4980b0f60 2024-04-20T10:21:07.889 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.888+0000 7fc48ea00640 1 -- 172.21.15.121:0/2929513223 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v6) ==== 137+0+0 (secure 0 0 0) 0x7fc490059a80 con 0x7fc4980b0f60 2024-04-20T10:21:07.889 INFO:teuthology.orchestra.run.smithi121.stderr:added data pool 4 to fsmap 2024-04-20T10:21:07.892 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.892+0000 7fc49da00640 1 -- 172.21.15.121:0/2929513223 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fc47406c8f0 msgr2=0x7fc47406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:07.892 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.892+0000 7fc49da00640 1 --2- 172.21.15.121:0/2929513223 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fc47406c8f0 0x7fc47406edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc488036000 tx=0x7fc488030000 comp rx=0 tx=0).stop 2024-04-20T10:21:07.892 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.892+0000 7fc49da00640 1 -- 172.21.15.121:0/2929513223 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4980b0f60 msgr2=0x7fc498161070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:07.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.892+0000 7fc49da00640 1 --2- 172.21.15.121:0/2929513223 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4980b0f60 0x7fc498161070 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc4980719a0 tx=0x7fc49000b4a0 comp rx=0 tx=0).stop 2024-04-20T10:21:07.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.892+0000 7fc49da00640 1 -- 172.21.15.121:0/2929513223 shutdown_connections 2024-04-20T10:21:07.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.892+0000 7fc49da00640 1 --2- 172.21.15.121:0/2929513223 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc4980b0f60 0x7fc498161070 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:07.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.892+0000 7fc49da00640 1 --2- 172.21.15.121:0/2929513223 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc4981f5fb0 0x7fc4981f45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:07.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.892+0000 7fc49da00640 1 --2- 172.21.15.121:0/2929513223 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fc47406c8f0 0x7fc47406edb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:07.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.892+0000 7fc49da00640 1 --2- 172.21.15.121:0/2929513223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4780039b0 0x7fc498155c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:07.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.892+0000 7fc49da00640 1 -- 172.21.15.121:0/2929513223 >> 172.21.15.121:0/2929513223 conn(0x7fc4980b3350 msgr2=0x7fc498143ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:07.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.892+0000 7fc49da00640 1 -- 172.21.15.121:0/2929513223 shutdown_connections 2024-04-20T10:21:07.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:07.892+0000 7fc49da00640 1 -- 172.21.15.121:0/2929513223 wait complete. 2024-04-20T10:21:07.910 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:21:08.089 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.088+0000 7fd4f9a00640 1 Processor -- start 2024-04-20T10:21:08.089 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.089+0000 7fd4f9a00640 1 -- start start 2024-04-20T10:21:08.091 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.090+0000 7fd4f9a00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd4f40d4270 0x7fd4f40d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:08.091 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.090+0000 7fd4f9a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd4f40d4650 0x7fd4f40b1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:08.091 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.090+0000 7fd4f9a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd4f40b17d0 0x7fd4f40d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:08.091 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.090+0000 7fd4f9a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fd4f40d40c0 con 0x7fd4f40b17d0 2024-04-20T10:21:08.091 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.090+0000 7fd4f9a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fd4f40d3820 con 0x7fd4f40d4650 2024-04-20T10:21:08.091 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.090+0000 7fd4f9a00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fd4f40d3b00 con 0x7fd4f40d4270 2024-04-20T10:21:08.091 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.090+0000 7fd4f3400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd4f40b17d0 0x7fd4f40d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:08.091 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.090+0000 7fd4f3400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd4f40b17d0 0x7fd4f40d0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42216/0 (socket says 172.21.15.121:42216) 2024-04-20T10:21:08.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.090+0000 7fd4f3400640 1 -- 172.21.15.121:0/4186362641 learned_addr learned my addr 172.21.15.121:0/4186362641 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:08.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.090+0000 7fd4f3400640 1 -- 172.21.15.121:0/4186362641 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd4f40d4270 msgr2=0x7fd4f40d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:08.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.090+0000 7fd4f3400640 1 --2- 172.21.15.121:0/4186362641 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd4f40d4270 0x7fd4f40d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:08.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.090+0000 7fd4f3400640 1 -- 172.21.15.121:0/4186362641 >> v2:172.21.15.123:3300/0 conn(0x7fd4f40d4650 msgr2=0x7fd4f40b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:08.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.090+0000 7fd4f2000640 1 --2- 172.21.15.121:0/4186362641 >> v2:172.21.15.123:3300/0 conn(0x7fd4f40d4650 0x7fd4f40b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:08.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.091+0000 7fd4f3400640 1 --2- 172.21.15.121:0/4186362641 >> v2:172.21.15.123:3300/0 conn(0x7fd4f40d4650 0x7fd4f40b1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:08.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.091+0000 7fd4f3400640 1 -- 172.21.15.121:0/4186362641 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f40d3de0 con 0x7fd4f40b17d0 2024-04-20T10:21:08.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.091+0000 7fd4f2000640 1 --2- 172.21.15.121:0/4186362641 >> v2:172.21.15.123:3300/0 conn(0x7fd4f40d4650 0x7fd4f40b1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:08.093 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.092+0000 7fd4f3400640 1 --2- 172.21.15.121:0/4186362641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4f40b17d0 0x7fd4f40d0380 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd4f40683c0 tx=0x7fd4ec02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a1af439b6c65e53 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:08.093 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.092+0000 7fd4f1600640 1 -- 172.21.15.121:0/4186362641 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec035020 con 0x7fd4f40b17d0 2024-04-20T10:21:08.093 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.092+0000 7fd4f1600640 1 -- 172.21.15.121:0/4186362641 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4ec031870 con 0x7fd4f40b17d0 2024-04-20T10:21:08.093 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.092+0000 7fd4f1600640 1 -- 172.21.15.121:0/4186362641 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec031c40 con 0x7fd4f40b17d0 2024-04-20T10:21:08.093 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.092+0000 7fd4f9a00640 1 -- 172.21.15.121:0/4186362641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4f40b17d0 msgr2=0x7fd4f40d0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:08.093 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.092+0000 7fd4f9a00640 1 --2- 172.21.15.121:0/4186362641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4f40b17d0 0x7fd4f40d0380 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd4f40683c0 tx=0x7fd4ec02b910 comp rx=0 tx=0).stop 2024-04-20T10:21:08.094 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.092+0000 7fd4f9a00640 1 -- 172.21.15.121:0/4186362641 shutdown_connections 2024-04-20T10:21:08.094 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.092+0000 7fd4f9a00640 1 --2- 172.21.15.121:0/4186362641 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4f40b17d0 0x7fd4f40d0380 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:08.094 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.092+0000 7fd4f9a00640 1 --2- 172.21.15.121:0/4186362641 >> v2:172.21.15.123:3300/0 conn(0x7fd4f40d4650 0x7fd4f40b1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:08.094 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.092+0000 7fd4f9a00640 1 --2- 172.21.15.121:0/4186362641 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd4f40d4270 0x7fd4f40d1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:08.094 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.092+0000 7fd4f9a00640 1 -- 172.21.15.121:0/4186362641 >> 172.21.15.121:0/4186362641 conn(0x7fd4f40b99c0 msgr2=0x7fd4f40b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:08.094 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.093+0000 7fd4f9a00640 1 -- 172.21.15.121:0/4186362641 shutdown_connections 2024-04-20T10:21:08.094 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.093+0000 7fd4f9a00640 1 -- 172.21.15.121:0/4186362641 wait complete. 2024-04-20T10:21:08.094 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.094+0000 7fd4f9a00640 1 Processor -- start 2024-04-20T10:21:08.095 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.094+0000 7fd4f9a00640 1 -- start start 2024-04-20T10:21:08.097 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.095+0000 7fd4f9a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4f40b17d0 0x7fd4f40cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:08.097 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.095+0000 7fd4f9a00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd4f40d4270 0x7fd4f40cde00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:08.097 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.095+0000 7fd4f9a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4f40d4650 0x7fd4f41555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:08.097 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.095+0000 7fd4f9a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f4161f70 con 0x7fd4f40d4650 2024-04-20T10:21:08.097 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.095+0000 7fd4f9a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f40d53b0 con 0x7fd4f40b17d0 2024-04-20T10:21:08.097 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.095+0000 7fd4f9a00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fd4f41620f0 con 0x7fd4f40d4270 2024-04-20T10:21:08.097 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.095+0000 7fd4f3400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4f40d4650 0x7fd4f41555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:08.097 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.095+0000 7fd4f3400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4f40d4650 0x7fd4f41555f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42224/0 (socket says 172.21.15.121:42224) 2024-04-20T10:21:08.097 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.095+0000 7fd4f3400640 1 -- 172.21.15.121:0/2337256919 learned_addr learned my addr 172.21.15.121:0/2337256919 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:08.097 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.096+0000 7fd4f2000640 1 --2- 172.21.15.121:0/2337256919 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd4f40d4270 0x7fd4f40cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:08.098 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.096+0000 7fd4f2a00640 1 --2- 172.21.15.121:0/2337256919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4f40b17d0 0x7fd4f40cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:08.098 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.096+0000 7fd4f3400640 1 -- 172.21.15.121:0/2337256919 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd4f40d4270 msgr2=0x7fd4f40cde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:08.098 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.096+0000 7fd4f3400640 1 --2- 172.21.15.121:0/2337256919 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd4f40d4270 0x7fd4f40cde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:08.098 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.096+0000 7fd4f3400640 1 -- 172.21.15.121:0/2337256919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4f40b17d0 msgr2=0x7fd4f40cb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:08.098 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.096+0000 7fd4f3400640 1 --2- 172.21.15.121:0/2337256919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4f40b17d0 0x7fd4f40cb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:08.098 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.096+0000 7fd4f3400640 1 -- 172.21.15.121:0/2337256919 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f406d5b0 con 0x7fd4f40d4650 2024-04-20T10:21:08.098 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.096+0000 7fd4f3400640 1 --2- 172.21.15.121:0/2337256919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4f40d4650 0x7fd4f41555f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec0090b0 tx=0x7fd4ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:08.098 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.096+0000 7fd4eaa00640 1 -- 172.21.15.121:0/2337256919 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec040070 con 0x7fd4f40d4650 2024-04-20T10:21:08.098 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.096+0000 7fd4eaa00640 1 -- 172.21.15.121:0/2337256919 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4ec03a420 con 0x7fd4f40d4650 2024-04-20T10:21:08.098 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.096+0000 7fd4eaa00640 1 -- 172.21.15.121:0/2337256919 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec035040 con 0x7fd4f40d4650 2024-04-20T10:21:08.098 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.097+0000 7fd4f9a00640 1 -- 172.21.15.121:0/2337256919 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4f406d890 con 0x7fd4f40d4650 2024-04-20T10:21:08.098 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.097+0000 7fd4f9a00640 1 -- 172.21.15.121:0/2337256919 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4f406de50 con 0x7fd4f40d4650 2024-04-20T10:21:08.099 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.098+0000 7fd4f9a00640 1 -- 172.21.15.121:0/2337256919 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4f4075c20 con 0x7fd4f40d4650 2024-04-20T10:21:08.099 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.099+0000 7fd4eaa00640 1 -- 172.21.15.121:0/2337256919 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd4ec04b060 con 0x7fd4f40d4650 2024-04-20T10:21:08.100 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.099+0000 7fd4eaa00640 1 --2- 172.21.15.121:0/2337256919 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fd4d006c8f0 0x7fd4d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:08.100 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.100+0000 7fd4eaa00640 1 -- 172.21.15.121:0/2337256919 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd4ec0add80 con 0x7fd4f40d4650 2024-04-20T10:21:08.100 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.100+0000 7fd4f2a00640 1 --2- 172.21.15.121:0/2337256919 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fd4d006c8f0 0x7fd4d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:08.100 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.100+0000 7fd4f2a00640 1 --2- 172.21.15.121:0/2337256919 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fd4d006c8f0 0x7fd4d006edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd4f40b2610 tx=0x7fd4e40094f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:08.104 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.104+0000 7fd4eaa00640 1 -- 172.21.15.121:0/2337256919 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd4f4075c20 con 0x7fd4f40d4650 2024-04-20T10:21:08.345 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.345+0000 7fd4f9a00640 1 -- 172.21.15.121:0/2337256919 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4f4068980 con 0x7fd4f40d4650 2024-04-20T10:21:08.347 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.346+0000 7fd4eaa00640 1 -- 172.21.15.121:0/2337256919 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5204 (secure 0 0 0) 0x7fd4ec07be60 con 0x7fd4f40d4650 2024-04-20T10:21:08.347 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:21:08.347 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1376743826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1376743826},{"type":"v1","addr":"172.21.15.123:6833","nonce":1376743826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/999167058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":999167058},{"type":"v1","addr":"172.21.15.121:6835","nonce":999167058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2160736935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2160736935},{"type":"v1","addr":"172.21.15.121:6837","nonce":2160736935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:21:02.048590+0000","modified":"2024-04-20T10:21:07.336873+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6835/3336775250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3336775250},{"type":"v1","addr":"172.21.15.123:6835","nonce":3336775250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1}]} 2024-04-20T10:21:08.347 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 6 2024-04-20T10:21:08.350 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.349+0000 7fd4f9a00640 1 -- 172.21.15.121:0/2337256919 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fd4d006c8f0 msgr2=0x7fd4d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:08.350 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.349+0000 7fd4f9a00640 1 --2- 172.21.15.121:0/2337256919 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fd4d006c8f0 0x7fd4d006edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd4f40b2610 tx=0x7fd4e40094f0 comp rx=0 tx=0).stop 2024-04-20T10:21:08.350 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.349+0000 7fd4f9a00640 1 -- 172.21.15.121:0/2337256919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4f40d4650 msgr2=0x7fd4f41555f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:08.350 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.349+0000 7fd4f9a00640 1 --2- 172.21.15.121:0/2337256919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4f40d4650 0x7fd4f41555f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec0090b0 tx=0x7fd4ec0046c0 comp rx=0 tx=0).stop 2024-04-20T10:21:08.350 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.349+0000 7fd4f9a00640 1 -- 172.21.15.121:0/2337256919 shutdown_connections 2024-04-20T10:21:08.350 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.349+0000 7fd4f9a00640 1 --2- 172.21.15.121:0/2337256919 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd4f40d4650 0x7fd4f41555f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:08.350 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.349+0000 7fd4f9a00640 1 --2- 172.21.15.121:0/2337256919 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd4f40d4270 0x7fd4f40cde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:08.350 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.349+0000 7fd4f9a00640 1 --2- 172.21.15.121:0/2337256919 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fd4d006c8f0 0x7fd4d006edb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:08.351 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.349+0000 7fd4f9a00640 1 --2- 172.21.15.121:0/2337256919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4f40b17d0 0x7fd4f40cb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:08.351 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.349+0000 7fd4f9a00640 1 -- 172.21.15.121:0/2337256919 >> 172.21.15.121:0/2337256919 conn(0x7fd4f40b99c0 msgr2=0x7fd4f40b6570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:08.351 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.349+0000 7fd4f9a00640 1 -- 172.21.15.121:0/2337256919 shutdown_connections 2024-04-20T10:21:08.351 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.349+0000 7fd4f9a00640 1 -- 172.21.15.121:0/2337256919 wait complete. 2024-04-20T10:21:08.369 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:21:08.561 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.560+0000 7f3119000640 1 Processor -- start 2024-04-20T10:21:08.561 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.560+0000 7f3119000640 1 -- start start 2024-04-20T10:21:08.563 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.561+0000 7f3119000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31140d4270 0x7f31140d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:08.563 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.561+0000 7f3119000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f31140d0380 0x7f31140d0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:08.563 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.562+0000 7f3119000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31140d0d50 con 0x7f31140d4650 2024-04-20T10:21:08.563 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.562+0000 7f3119000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f31140d40c0 con 0x7f31140d0380 2024-04-20T10:21:08.563 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.562+0000 7f3119000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f31140d3820 con 0x7f31140d4270 2024-04-20T10:21:08.563 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.562+0000 7f3112800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f31140d0380 0x7f31140d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:08.563 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.562+0000 7f3112800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f31140d0380 0x7f31140d0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42238/0 (socket says 172.21.15.121:42238) 2024-04-20T10:21:08.563 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.562+0000 7f3112800640 1 -- 172.21.15.121:0/1691902431 learned_addr learned my addr 172.21.15.121:0/1691902431 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:08.563 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.562+0000 7f3112800640 1 -- 172.21.15.121:0/1691902431 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31140d4270 msgr2=0x7f31140d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:08.563 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.562+0000 7f3112800640 1 --2- 172.21.15.121:0/1691902431 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31140d4270 0x7f31140d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:08.563 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.562+0000 7f3112800640 1 -- 172.21.15.121:0/1691902431 >> v1:172.21.15.123:6789/0 conn(0x7f31140d4650 legacy=0x7f31140b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:08.563 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.562+0000 7f3112800640 1 -- 172.21.15.121:0/1691902431 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31140d3b00 con 0x7f31140d0380 2024-04-20T10:21:08.565 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.563+0000 7f3112800640 1 --2- 172.21.15.121:0/1691902431 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31140d0380 0x7f31140d0810 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f310c009870 tx=0x7f310c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb934d1b190c2a0c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:08.565 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.564+0000 7f3110a00640 1 -- 172.21.15.121:0/1691902431 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310c035020 con 0x7f31140d0380 2024-04-20T10:21:08.565 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.564+0000 7f3110a00640 1 -- 172.21.15.121:0/1691902431 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f310c031640 con 0x7f31140d0380 2024-04-20T10:21:08.565 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.564+0000 7f3110a00640 1 -- 172.21.15.121:0/1691902431 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310c02d040 con 0x7f31140d0380 2024-04-20T10:21:08.565 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.564+0000 7f3119000640 1 -- 172.21.15.121:0/1691902431 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31140d0380 msgr2=0x7f31140d0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:08.565 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.564+0000 7f3119000640 1 --2- 172.21.15.121:0/1691902431 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31140d0380 0x7f31140d0810 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f310c009870 tx=0x7f310c02b8f0 comp rx=0 tx=0).stop 2024-04-20T10:21:08.565 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.564+0000 7f3119000640 1 -- 172.21.15.121:0/1691902431 shutdown_connections 2024-04-20T10:21:08.565 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.564+0000 7f3119000640 1 --2- 172.21.15.121:0/1691902431 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31140d0380 0x7f31140d0810 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:08.565 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.564+0000 7f3119000640 1 --2- 172.21.15.121:0/1691902431 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31140d4270 0x7f31140d1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:08.565 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.564+0000 7f3119000640 1 -- 172.21.15.121:0/1691902431 >> 172.21.15.121:0/1691902431 conn(0x7f31140b99c0 msgr2=0x7f31140b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:08.566 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.564+0000 7f3119000640 1 -- 172.21.15.121:0/1691902431 shutdown_connections 2024-04-20T10:21:08.566 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.564+0000 7f3119000640 1 -- 172.21.15.121:0/1691902431 wait complete. 2024-04-20T10:21:08.566 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.566+0000 7f3119000640 1 Processor -- start 2024-04-20T10:21:08.566 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.566+0000 7f3119000640 1 -- start start 2024-04-20T10:21:08.568 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.567+0000 7f3119000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31140d0380 0x7f311415c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:08.568 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.567+0000 7f3119000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31140d4270 0x7f31140cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:08.568 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.567+0000 7f3119000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31140d4650 0x7f31141f54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:08.568 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.567+0000 7f3119000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f31140b19a0 con 0x7f31140d4650 2024-04-20T10:21:08.568 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.567+0000 7f3119000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f31140cfe60 con 0x7f31140d4270 2024-04-20T10:21:08.568 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.567+0000 7f3119000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f31140d53b0 con 0x7f31140d0380 2024-04-20T10:21:08.569 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.567+0000 7f3112800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31140d4650 0x7f31141f54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:08.569 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.567+0000 7f3112800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31140d4650 0x7f31141f54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42248/0 (socket says 172.21.15.121:42248) 2024-04-20T10:21:08.569 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.567+0000 7f3112800640 1 -- 172.21.15.121:0/2119790419 learned_addr learned my addr 172.21.15.121:0/2119790419 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:08.569 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.568+0000 7f3111e00640 1 --2- 172.21.15.121:0/2119790419 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31140d0380 0x7f311415c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:08.569 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.568+0000 7f3112800640 1 -- 172.21.15.121:0/2119790419 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31140d0380 msgr2=0x7f311415c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:08.569 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.568+0000 7f3112800640 1 --2- 172.21.15.121:0/2119790419 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31140d0380 0x7f311415c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:08.569 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.568+0000 7f3112800640 1 -- 172.21.15.121:0/2119790419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31140d4270 msgr2=0x7f31140cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:08.569 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.568+0000 7f3112800640 1 --2- 172.21.15.121:0/2119790419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31140d4270 0x7f31140cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:08.569 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.568+0000 7f3112800640 1 -- 172.21.15.121:0/2119790419 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31140ce0f0 con 0x7f31140d4650 2024-04-20T10:21:08.569 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.568+0000 7f3111e00640 1 --2- 172.21.15.121:0/2119790419 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31140d0380 0x7f311415c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:08.569 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.568+0000 7f3112800640 1 --2- 172.21.15.121:0/2119790419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31140d4650 0x7f31141f54a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f310c02b930 tx=0x7f310c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:08.570 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.568+0000 7f310a000640 1 -- 172.21.15.121:0/2119790419 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310c03ac60 con 0x7f31140d4650 2024-04-20T10:21:08.570 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.568+0000 7f310a000640 1 -- 172.21.15.121:0/2119790419 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f310c03ae20 con 0x7f31140d4650 2024-04-20T10:21:08.570 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.568+0000 7f310a000640 1 -- 172.21.15.121:0/2119790419 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310c044630 con 0x7f31140d4650 2024-04-20T10:21:08.570 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.568+0000 7f3119000640 1 -- 172.21.15.121:0/2119790419 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f311415c9f0 con 0x7f31140d4650 2024-04-20T10:21:08.571 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.568+0000 7f3119000640 1 -- 172.21.15.121:0/2119790419 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f311415cfb0 con 0x7f31140d4650 2024-04-20T10:21:08.571 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.569+0000 7f3119000640 1 -- 172.21.15.121:0/2119790419 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3114069090 con 0x7f31140d4650 2024-04-20T10:21:08.571 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.570+0000 7f310a000640 1 -- 172.21.15.121:0/2119790419 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f310c04b060 con 0x7f31140d4650 2024-04-20T10:21:08.572 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.571+0000 7f310a000640 1 --2- 172.21.15.121:0/2119790419 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f30f006c8f0 0x7f30f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:08.572 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.571+0000 7f310a000640 1 -- 172.21.15.121:0/2119790419 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f310c0adc00 con 0x7f31140d4650 2024-04-20T10:21:08.572 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.571+0000 7f3111e00640 1 --2- 172.21.15.121:0/2119790419 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f30f006c8f0 0x7f30f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:08.572 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.572+0000 7f3111e00640 1 --2- 172.21.15.121:0/2119790419 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f30f006c8f0 0x7f30f006edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f311406ee50 tx=0x7f31040073e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:08.576 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.575+0000 7f310a000640 1 -- 172.21.15.121:0/2119790419 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f310c03f070 con 0x7f31140d4650 2024-04-20T10:21:08.780 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.780+0000 7f3119000640 1 -- 172.21.15.121:0/2119790419 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f31140b6ec0 con 0x7f31140d4650 2024-04-20T10:21:08.782 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.782+0000 7f310a000640 1 -- 172.21.15.121:0/2119790419 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7f310c02f300 con 0x7f31140d4650 2024-04-20T10:21:08.783 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:21:08.783 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":25,"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","created":"2024-04-20T10:20:38.076076+0000","modified":"2024-04-20T10:21:07.336542+0000","last_up_change":"2024-04-20T10:20:50.162700+0000","last_in_change":"2024-04-20T10:20:41.510382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:20:51.141365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:21:00.436937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:21:01.026957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:21:04.862693+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"629a7b8c-a40a-4a2a-a3cc-d9f357774310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6809","nonce":1044588835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6811","nonce":1044588835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6815","nonce":1044588835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6813","nonce":1044588835}]},"public_addr":"172.21.15.121:6809/1044588835","cluster_addr":"172.21.15.121:6811/1044588835","heartbeat_back_addr":"172.21.15.121:6815/1044588835","heartbeat_front_addr":"172.21.15.121:6813/1044588835","state":["exists","up"]},{"osd":1,"uuid":"f2570399-aa50-4ff3-bf5d-3d09207290db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6801","nonce":3505199624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6803","nonce":3505199624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6807","nonce":3505199624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6805","nonce":3505199624}]},"public_addr":"172.21.15.121:6801/3505199624","cluster_addr":"172.21.15.121:6803/3505199624","heartbeat_back_addr":"172.21.15.121:6807/3505199624","heartbeat_front_addr":"172.21.15.121:6805/3505199624","state":["exists","up"]},{"osd":2,"uuid":"50893807-dc30-49f4-b41f-739366626658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6817","nonce":3656724207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6819","nonce":3656724207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6827","nonce":3656724207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6822","nonce":3656724207}]},"public_addr":"172.21.15.121:6817/3656724207","cluster_addr":"172.21.15.121:6819/3656724207","heartbeat_back_addr":"172.21.15.121:6827/3656724207","heartbeat_front_addr":"172.21.15.121:6822/3656724207","state":["exists","up"]},{"osd":3,"uuid":"dadb01bc-5491-4ffc-8824-d48a057240b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6823","nonce":2232342832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6826","nonce":2232342832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6831","nonce":2232342832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6829","nonce":2232342832}]},"public_addr":"172.21.15.121:6823/2232342832","cluster_addr":"172.21.15.121:6826/2232342832","heartbeat_back_addr":"172.21.15.121:6831/2232342832","heartbeat_front_addr":"172.21.15.121:6829/2232342832","state":["exists","up"]},{"osd":4,"uuid":"b3f3b4a1-b375-4526-b69e-344ab24c2da1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6801","nonce":99353111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6803","nonce":99353111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6807","nonce":99353111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6805","nonce":99353111}]},"public_addr":"172.21.15.123:6801/99353111","cluster_addr":"172.21.15.123:6803/99353111","heartbeat_back_addr":"172.21.15.123:6807/99353111","heartbeat_front_addr":"172.21.15.123:6805/99353111","state":["exists","up"]},{"osd":5,"uuid":"2ee24a08-1624-4961-9642-d7b510282484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6818","nonce":2504374688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6822","nonce":2504374688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6830","nonce":2504374688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6826","nonce":2504374688}]},"public_addr":"172.21.15.123:6818/2504374688","cluster_addr":"172.21.15.123:6822/2504374688","heartbeat_back_addr":"172.21.15.123:6830/2504374688","heartbeat_front_addr":"172.21.15.123:6826/2504374688","state":["exists","up"]},{"osd":6,"uuid":"186118cc-6128-49de-acca-55260945c1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6819","nonce":4139299599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6823","nonce":4139299599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6829","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6831","nonce":4139299599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6827","nonce":4139299599}]},"public_addr":"172.21.15.123:6819/4139299599","cluster_addr":"172.21.15.123:6823/4139299599","heartbeat_back_addr":"172.21.15.123:6831/4139299599","heartbeat_front_addr":"172.21.15.123:6827/4139299599","state":["exists","up"]},{"osd":7,"uuid":"a00aac6e-4bd9-415a-a93d-ae85210865ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6809","nonce":4043981069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6811","nonce":4043981069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6815","nonce":4043981069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6813","nonce":4043981069}]},"public_addr":"172.21.15.123:6809/4043981069","cluster_addr":"172.21.15.123:6811/4043981069","heartbeat_back_addr":"172.21.15.123:6815/4043981069","heartbeat_front_addr":"172.21.15.123:6813/4043981069","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.904540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.958585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.616367+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.885244+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.103127+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.167248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.884348+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.862356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:21:08.786 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.785+0000 7f3119000640 1 -- 172.21.15.121:0/2119790419 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f30f006c8f0 msgr2=0x7f30f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:08.786 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.785+0000 7f3119000640 1 --2- 172.21.15.121:0/2119790419 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f30f006c8f0 0x7f30f006edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f311406ee50 tx=0x7f31040073e0 comp rx=0 tx=0).stop 2024-04-20T10:21:08.786 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.785+0000 7f3119000640 1 -- 172.21.15.121:0/2119790419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31140d4650 msgr2=0x7f31141f54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:08.786 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.785+0000 7f3119000640 1 --2- 172.21.15.121:0/2119790419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31140d4650 0x7f31141f54a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f310c02b930 tx=0x7f310c0046c0 comp rx=0 tx=0).stop 2024-04-20T10:21:08.786 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.785+0000 7f3119000640 1 -- 172.21.15.121:0/2119790419 shutdown_connections 2024-04-20T10:21:08.786 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.785+0000 7f3119000640 1 --2- 172.21.15.121:0/2119790419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31140d4650 0x7f31141f54a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:08.786 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.785+0000 7f3119000640 1 --2- 172.21.15.121:0/2119790419 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f30f006c8f0 0x7f30f006edb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:08.786 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.785+0000 7f3119000640 1 --2- 172.21.15.121:0/2119790419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31140d4270 0x7f31140cdbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:08.786 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.785+0000 7f3119000640 1 --2- 172.21.15.121:0/2119790419 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31140d0380 0x7f311415c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:08.786 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.785+0000 7f3119000640 1 -- 172.21.15.121:0/2119790419 >> 172.21.15.121:0/2119790419 conn(0x7f31140b99c0 msgr2=0x7f31140b65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:08.786 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.785+0000 7f3119000640 1 -- 172.21.15.121:0/2119790419 shutdown_connections 2024-04-20T10:21:08.787 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:08.785+0000 7f3119000640 1 -- 172.21.15.121:0/2119790419 wait complete. 2024-04-20T10:21:08.804 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:21:09.001 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.000+0000 7f9f00400640 1 Processor -- start 2024-04-20T10:21:09.001 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.000+0000 7f9f00400640 1 -- start start 2024-04-20T10:21:09.002 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.002+0000 7f9f00400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ef8072cc0 0x7f9ef80ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:09.002 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.002+0000 7f9f00400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9ef8071b50 0x7f9ef80adde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:09.003 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.002+0000 7f9f00400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ef806e840 con 0x7f9ef815abb0 2024-04-20T10:21:09.003 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.002+0000 7f9f00400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f9ef806eb00 con 0x7f9ef8071b50 2024-04-20T10:21:09.003 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.002+0000 7f9f00400640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f9ef815bf60 con 0x7f9ef8072cc0 2024-04-20T10:21:09.003 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.002+0000 7f9efda00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f9ef815abb0 0x7f9ef80725b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:35862/0 (socket says 172.21.15.121:35862) 2024-04-20T10:21:09.003 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.002+0000 7f9efda00640 1 -- 172.21.15.121:0/1928670432 learned_addr learned my addr 172.21.15.121:0/1928670432 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:09.005 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.003+0000 7f9efe400640 1 --2- 172.21.15.121:0/1928670432 >> v2:172.21.15.123:3300/0 conn(0x7f9ef8071b50 0x7f9ef80adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:09.005 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.003+0000 7f9efd000640 1 --2- 172.21.15.121:0/1928670432 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ef8072cc0 0x7f9ef80ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:09.005 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.003+0000 7f9ef7e00640 1 -- 172.21.15.121:0/1928670432 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3513006303 0 0) 0x7f9ef806e840 con 0x7f9ef815abb0 2024-04-20T10:21:09.005 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.003+0000 7f9ef7e00640 1 -- 172.21.15.121:0/1928670432 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ee0003520 con 0x7f9ef815abb0 2024-04-20T10:21:09.005 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.003+0000 7f9ef7e00640 1 -- 172.21.15.121:0/1928670432 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2745699165 0 0) 0x7f9ee0003520 con 0x7f9ef815abb0 2024-04-20T10:21:09.005 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.003+0000 7f9ef7e00640 1 -- 172.21.15.121:0/1928670432 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ef8072cc0 msgr2=0x7f9ef80ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:09.005 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.003+0000 7f9ef7e00640 1 --2- 172.21.15.121:0/1928670432 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ef8072cc0 0x7f9ef80ad8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:09.005 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.003+0000 7f9ef7e00640 1 -- 172.21.15.121:0/1928670432 >> v2:172.21.15.123:3300/0 conn(0x7f9ef8071b50 msgr2=0x7f9ef80adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:09.005 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.003+0000 7f9ef7e00640 1 --2- 172.21.15.121:0/1928670432 >> v2:172.21.15.123:3300/0 conn(0x7f9ef8071b50 0x7f9ef80adde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:09.005 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.003+0000 7f9ef7e00640 1 -- 172.21.15.121:0/1928670432 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef815b350 con 0x7f9ef815abb0 2024-04-20T10:21:09.005 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.003+0000 7f9ef7e00640 1 -- 172.21.15.121:0/1928670432 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f9ee8001670 con 0x7f9ef815abb0 2024-04-20T10:21:09.005 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.003+0000 7f9ef7e00640 1 -- 172.21.15.121:0/1928670432 >> v1:172.21.15.121:6789/0 conn(0x7f9ef815abb0 legacy=0x7f9ef80725b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:09.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.003+0000 7f9ef7e00640 1 --2- 172.21.15.121:0/1928670432 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ef8072cc0 0x7f9ee00039d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:09.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.003+0000 7f9ef7e00640 1 --2- 172.21.15.121:0/1928670432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ee0003f50 0x7f9ee0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:09.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.003+0000 7f9ef7e00640 1 --2- 172.21.15.121:0/1928670432 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ee0004d40 0x7f9ee00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:09.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.003+0000 7f9ef7e00640 1 -- 172.21.15.121:0/1928670432 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef815bf60 con 0x7f9ee0003f50 2024-04-20T10:21:09.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.003+0000 7f9ef7e00640 1 -- 172.21.15.121:0/1928670432 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee000dbd0 con 0x7f9ee0004d40 2024-04-20T10:21:09.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.003+0000 7f9ef7e00640 1 -- 172.21.15.121:0/1928670432 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f9ee000dd50 con 0x7f9ef8072cc0 2024-04-20T10:21:09.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.004+0000 7f9efd000640 1 --2- 172.21.15.121:0/1928670432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ee0003f50 0x7f9ee0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:09.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.004+0000 7f9efda00640 1 --2- 172.21.15.121:0/1928670432 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ef8072cc0 0x7f9ee00039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:09.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.004+0000 7f9efd000640 1 -- 172.21.15.121:0/1928670432 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ef8072cc0 msgr2=0x7f9ee00039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:09.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.004+0000 7f9efd000640 1 --2- 172.21.15.121:0/1928670432 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ef8072cc0 0x7f9ee00039d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:09.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.004+0000 7f9efd000640 1 -- 172.21.15.121:0/1928670432 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ee0004d40 msgr2=0x7f9ee00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:09.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.004+0000 7f9efd000640 1 --2- 172.21.15.121:0/1928670432 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ee0004d40 0x7f9ee00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:09.007 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.004+0000 7f9efd000640 1 -- 172.21.15.121:0/1928670432 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee0003520 con 0x7f9ee0003f50 2024-04-20T10:21:09.008 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.005+0000 7f9efd000640 1 --2- 172.21.15.121:0/1928670432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ee0003f50 0x7f9ee0004800 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9eec009af0 tx=0x7f9ee0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df1dfec1580c1dc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:09.008 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.005+0000 7f9ef7e00640 1 -- 172.21.15.121:0/1928670432 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eec039070 con 0x7f9ee0003f50 2024-04-20T10:21:09.008 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.005+0000 7f9ef7e00640 1 -- 172.21.15.121:0/1928670432 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9eec02c6b0 con 0x7f9ee0003f50 2024-04-20T10:21:09.008 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.005+0000 7f9ef7e00640 1 -- 172.21.15.121:0/1928670432 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eec009690 con 0x7f9ee0003f50 2024-04-20T10:21:09.008 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.006+0000 7f9f00400640 1 -- 172.21.15.121:0/1928670432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ee0003f50 msgr2=0x7f9ee0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:09.008 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.006+0000 7f9f00400640 1 --2- 172.21.15.121:0/1928670432 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ee0003f50 0x7f9ee0004800 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9eec009af0 tx=0x7f9ee0005830 comp rx=0 tx=0).stop 2024-04-20T10:21:09.008 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.006+0000 7f9efe400640 1 -- 172.21.15.121:0/1928670432 reap_dead start 2024-04-20T10:21:09.009 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.006+0000 7f9f00400640 1 -- 172.21.15.121:0/1928670432 shutdown_connections 2024-04-20T10:21:09.009 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.006+0000 7f9f00400640 1 -- 172.21.15.121:0/1928670432 >> 172.21.15.121:0/1928670432 conn(0x7f9ef80b3350 msgr2=0x7f9ef80b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:09.009 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.006+0000 7f9f00400640 1 -- 172.21.15.121:0/1928670432 shutdown_connections 2024-04-20T10:21:09.009 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.006+0000 7f9f00400640 1 -- 172.21.15.121:0/1928670432 wait complete. 2024-04-20T10:21:09.009 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.008+0000 7f9f00400640 1 Processor -- start 2024-04-20T10:21:09.009 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.008+0000 7f9f00400640 1 -- start start 2024-04-20T10:21:09.011 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.009+0000 7f9f00400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ee0003f50 0x7f9ef8158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:09.011 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.009+0000 7f9f00400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ef8153300 0x7f9ef81f3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:09.011 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.009+0000 7f9f00400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ef80b0840 0x7f9ef80b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:09.011 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.009+0000 7f9f00400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef815bbf0 con 0x7f9ee0003f50 2024-04-20T10:21:09.011 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.009+0000 7f9f00400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef815b910 con 0x7f9ef80b0840 2024-04-20T10:21:09.011 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.009+0000 7f9f00400640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f9ef815af90 con 0x7f9ef8153300 2024-04-20T10:21:09.011 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.009+0000 7f9efda00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ee0003f50 0x7f9ef8158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:09.011 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.009+0000 7f9efda00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ee0003f50 0x7f9ef8158bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42260/0 (socket says 172.21.15.121:42260) 2024-04-20T10:21:09.011 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.009+0000 7f9efda00640 1 -- 172.21.15.121:0/4010931190 learned_addr learned my addr 172.21.15.121:0/4010931190 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:09.011 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.009+0000 7f9efd000640 1 --2- 172.21.15.121:0/4010931190 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ef8153300 0x7f9ef81f3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:09.012 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.009+0000 7f9efda00640 1 -- 172.21.15.121:0/4010931190 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ef8153300 msgr2=0x7f9ef81f3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:09.012 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.009+0000 7f9efe400640 1 --2- 172.21.15.121:0/4010931190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ef80b0840 0x7f9ef80b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:09.012 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.009+0000 7f9efda00640 1 --2- 172.21.15.121:0/4010931190 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ef8153300 0x7f9ef81f3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:09.012 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.009+0000 7f9efda00640 1 -- 172.21.15.121:0/4010931190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ef80b0840 msgr2=0x7f9ef80b0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:09.012 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.009+0000 7f9efda00640 1 --2- 172.21.15.121:0/4010931190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ef80b0840 0x7f9ef80b0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:09.012 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.009+0000 7f9efda00640 1 -- 172.21.15.121:0/4010931190 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef8153780 con 0x7f9ee0003f50 2024-04-20T10:21:09.012 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.010+0000 7f9efda00640 1 --2- 172.21.15.121:0/4010931190 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ee0003f50 0x7f9ef8158bb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9ee800d7d0 tx=0x7f9ee800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:09.012 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.010+0000 7f9ef5600640 1 -- 172.21.15.121:0/4010931190 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee8019070 con 0x7f9ee0003f50 2024-04-20T10:21:09.012 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.010+0000 7f9ef5600640 1 -- 172.21.15.121:0/4010931190 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ee80074e0 con 0x7f9ee0003f50 2024-04-20T10:21:09.012 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.010+0000 7f9ef5600640 1 -- 172.21.15.121:0/4010931190 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee8004a20 con 0x7f9ee0003f50 2024-04-20T10:21:09.012 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.010+0000 7f9f00400640 1 -- 172.21.15.121:0/4010931190 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ef81592b0 con 0x7f9ee0003f50 2024-04-20T10:21:09.012 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.010+0000 7f9f00400640 1 -- 172.21.15.121:0/4010931190 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ef8159870 con 0x7f9ee0003f50 2024-04-20T10:21:09.013 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.011+0000 7f9f00400640 1 -- 172.21.15.121:0/4010931190 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ef8068980 con 0x7f9ee0003f50 2024-04-20T10:21:09.013 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.012+0000 7f9ef5600640 1 -- 172.21.15.121:0/4010931190 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9ee801c070 con 0x7f9ee0003f50 2024-04-20T10:21:09.013 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.013+0000 7f9ef5600640 1 --2- 172.21.15.121:0/4010931190 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9edc06c920 0x7f9edc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:09.014 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.013+0000 7f9ef5600640 1 -- 172.21.15.121:0/4010931190 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9ee808b860 con 0x7f9ee0003f50 2024-04-20T10:21:09.014 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.013+0000 7f9efd000640 1 --2- 172.21.15.121:0/4010931190 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9edc06c920 0x7f9edc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:09.014 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.014+0000 7f9efd000640 1 --2- 172.21.15.121:0/4010931190 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9edc06c920 0x7f9edc06ede0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9eec036000 tx=0x7f9eec030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:09.017 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.017+0000 7f9ef5600640 1 -- 172.21.15.121:0/4010931190 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9ee8059a10 con 0x7f9ee0003f50 2024-04-20T10:21:09.220 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.220+0000 7f9f00400640 1 -- 172.21.15.121:0/4010931190 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9ef80cfe90 con 0x7f9ee0003f50 2024-04-20T10:21:09.223 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.223+0000 7f9ef5600640 1 -- 172.21.15.121:0/4010931190 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7f9ee8059310 con 0x7f9ee0003f50 2024-04-20T10:21:09.223 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:21:09.224 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":25,"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","created":"2024-04-20T10:20:38.076076+0000","modified":"2024-04-20T10:21:07.336542+0000","last_up_change":"2024-04-20T10:20:50.162700+0000","last_in_change":"2024-04-20T10:20:41.510382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:20:51.141365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:21:00.436937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:21:01.026957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:21:04.862693+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"629a7b8c-a40a-4a2a-a3cc-d9f357774310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6809","nonce":1044588835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6811","nonce":1044588835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6815","nonce":1044588835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6813","nonce":1044588835}]},"public_addr":"172.21.15.121:6809/1044588835","cluster_addr":"172.21.15.121:6811/1044588835","heartbeat_back_addr":"172.21.15.121:6815/1044588835","heartbeat_front_addr":"172.21.15.121:6813/1044588835","state":["exists","up"]},{"osd":1,"uuid":"f2570399-aa50-4ff3-bf5d-3d09207290db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6801","nonce":3505199624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6803","nonce":3505199624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6807","nonce":3505199624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6805","nonce":3505199624}]},"public_addr":"172.21.15.121:6801/3505199624","cluster_addr":"172.21.15.121:6803/3505199624","heartbeat_back_addr":"172.21.15.121:6807/3505199624","heartbeat_front_addr":"172.21.15.121:6805/3505199624","state":["exists","up"]},{"osd":2,"uuid":"50893807-dc30-49f4-b41f-739366626658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6817","nonce":3656724207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6819","nonce":3656724207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6827","nonce":3656724207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6822","nonce":3656724207}]},"public_addr":"172.21.15.121:6817/3656724207","cluster_addr":"172.21.15.121:6819/3656724207","heartbeat_back_addr":"172.21.15.121:6827/3656724207","heartbeat_front_addr":"172.21.15.121:6822/3656724207","state":["exists","up"]},{"osd":3,"uuid":"dadb01bc-5491-4ffc-8824-d48a057240b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6823","nonce":2232342832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6826","nonce":2232342832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6831","nonce":2232342832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6829","nonce":2232342832}]},"public_addr":"172.21.15.121:6823/2232342832","cluster_addr":"172.21.15.121:6826/2232342832","heartbeat_back_addr":"172.21.15.121:6831/2232342832","heartbeat_front_addr":"172.21.15.121:6829/2232342832","state":["exists","up"]},{"osd":4,"uuid":"b3f3b4a1-b375-4526-b69e-344ab24c2da1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6801","nonce":99353111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6803","nonce":99353111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6807","nonce":99353111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6805","nonce":99353111}]},"public_addr":"172.21.15.123:6801/99353111","cluster_addr":"172.21.15.123:6803/99353111","heartbeat_back_addr":"172.21.15.123:6807/99353111","heartbeat_front_addr":"172.21.15.123:6805/99353111","state":["exists","up"]},{"osd":5,"uuid":"2ee24a08-1624-4961-9642-d7b510282484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6818","nonce":2504374688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6822","nonce":2504374688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6830","nonce":2504374688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6826","nonce":2504374688}]},"public_addr":"172.21.15.123:6818/2504374688","cluster_addr":"172.21.15.123:6822/2504374688","heartbeat_back_addr":"172.21.15.123:6830/2504374688","heartbeat_front_addr":"172.21.15.123:6826/2504374688","state":["exists","up"]},{"osd":6,"uuid":"186118cc-6128-49de-acca-55260945c1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6819","nonce":4139299599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6823","nonce":4139299599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6829","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6831","nonce":4139299599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6827","nonce":4139299599}]},"public_addr":"172.21.15.123:6819/4139299599","cluster_addr":"172.21.15.123:6823/4139299599","heartbeat_back_addr":"172.21.15.123:6831/4139299599","heartbeat_front_addr":"172.21.15.123:6827/4139299599","state":["exists","up"]},{"osd":7,"uuid":"a00aac6e-4bd9-415a-a93d-ae85210865ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6809","nonce":4043981069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6811","nonce":4043981069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6815","nonce":4043981069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6813","nonce":4043981069}]},"public_addr":"172.21.15.123:6809/4043981069","cluster_addr":"172.21.15.123:6811/4043981069","heartbeat_back_addr":"172.21.15.123:6815/4043981069","heartbeat_front_addr":"172.21.15.123:6813/4043981069","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.904540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.958585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.616367+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.885244+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.103127+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.167248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.884348+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.862356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:21:09.226 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.225+0000 7f9f00400640 1 -- 172.21.15.121:0/4010931190 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9edc06c920 msgr2=0x7f9edc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:09.227 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.225+0000 7f9f00400640 1 --2- 172.21.15.121:0/4010931190 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9edc06c920 0x7f9edc06ede0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9eec036000 tx=0x7f9eec030000 comp rx=0 tx=0).stop 2024-04-20T10:21:09.227 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.226+0000 7f9f00400640 1 -- 172.21.15.121:0/4010931190 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ee0003f50 msgr2=0x7f9ef8158bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:09.227 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.226+0000 7f9f00400640 1 --2- 172.21.15.121:0/4010931190 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ee0003f50 0x7f9ef8158bb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9ee800d7d0 tx=0x7f9ee800dc90 comp rx=0 tx=0).stop 2024-04-20T10:21:09.227 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.226+0000 7f9f00400640 1 -- 172.21.15.121:0/4010931190 shutdown_connections 2024-04-20T10:21:09.227 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.226+0000 7f9f00400640 1 --2- 172.21.15.121:0/4010931190 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9edc06c920 0x7f9edc06ede0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:09.227 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.226+0000 7f9f00400640 1 --2- 172.21.15.121:0/4010931190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ef80b0840 0x7f9ef80b0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:09.227 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.226+0000 7f9f00400640 1 --2- 172.21.15.121:0/4010931190 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ef8153300 0x7f9ef81f3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:09.227 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.226+0000 7f9f00400640 1 --2- 172.21.15.121:0/4010931190 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ee0003f50 0x7f9ef8158bb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:09.227 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.226+0000 7f9f00400640 1 -- 172.21.15.121:0/4010931190 >> 172.21.15.121:0/4010931190 conn(0x7f9ef80b3350 msgr2=0x7f9ef806f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:09.227 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.226+0000 7f9f00400640 1 -- 172.21.15.121:0/4010931190 shutdown_connections 2024-04-20T10:21:09.227 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.226+0000 7f9f00400640 1 -- 172.21.15.121:0/4010931190 wait complete. 2024-04-20T10:21:09.259 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:21:09.457 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.456+0000 7f6461a00640 1 Processor -- start 2024-04-20T10:21:09.457 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.456+0000 7f6461a00640 1 -- start start 2024-04-20T10:21:09.458 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.458+0000 7f6461a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f645c0d0a70 0x7f645c06e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:09.458 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.458+0000 7f6461a00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f645c06dbb0 0x7f645c0ad680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:09.459 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.458+0000 7f6461a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f645c06f060 con 0x7f645c15e8a0 2024-04-20T10:21:09.459 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.458+0000 7f6461a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f645c06e7c0 con 0x7f645c0d0a70 2024-04-20T10:21:09.459 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.458+0000 7f6461a00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f645c06eaa0 con 0x7f645c06dbb0 2024-04-20T10:21:09.459 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.458+0000 7f645aa00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f645c15e8a0 0x7f645c0d0360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:35872/0 (socket says 172.21.15.121:35872) 2024-04-20T10:21:09.459 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.458+0000 7f645aa00640 1 -- 172.21.15.121:0/1844163176 learned_addr learned my addr 172.21.15.121:0/1844163176 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:09.461 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.458+0000 7f6453e00640 1 --2- 172.21.15.121:0/1844163176 >> v2:172.21.15.123:3300/0 conn(0x7f645c0d0a70 0x7f645c06e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:09.461 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.459+0000 7f645b400640 1 --2- 172.21.15.121:0/1844163176 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f645c06dbb0 0x7f645c0ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:09.461 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.459+0000 7f645a000640 1 -- 172.21.15.121:0/1844163176 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2368526880 0 0) 0x7f645c06f060 con 0x7f645c15e8a0 2024-04-20T10:21:09.461 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.459+0000 7f645a000640 1 -- 172.21.15.121:0/1844163176 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6440003520 con 0x7f645c15e8a0 2024-04-20T10:21:09.461 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.459+0000 7f645a000640 1 -- 172.21.15.121:0/1844163176 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1005088342 0 0) 0x7f6440003520 con 0x7f645c15e8a0 2024-04-20T10:21:09.461 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.459+0000 7f645a000640 1 -- 172.21.15.121:0/1844163176 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f645c06dbb0 msgr2=0x7f645c0ad680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:09.461 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.459+0000 7f645a000640 1 --2- 172.21.15.121:0/1844163176 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f645c06dbb0 0x7f645c0ad680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:09.461 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.459+0000 7f645a000640 1 -- 172.21.15.121:0/1844163176 >> v2:172.21.15.123:3300/0 conn(0x7f645c0d0a70 msgr2=0x7f645c06e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:09.461 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.459+0000 7f645a000640 1 --2- 172.21.15.121:0/1844163176 >> v2:172.21.15.123:3300/0 conn(0x7f645c0d0a70 0x7f645c06e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:09.462 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.459+0000 7f645a000640 1 -- 172.21.15.121:0/1844163176 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645c06ed80 con 0x7f645c15e8a0 2024-04-20T10:21:09.462 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.459+0000 7f645a000640 1 -- 172.21.15.121:0/1844163176 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f644c001670 con 0x7f645c15e8a0 2024-04-20T10:21:09.462 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.459+0000 7f645a000640 1 -- 172.21.15.121:0/1844163176 >> v1:172.21.15.121:6789/0 conn(0x7f645c15e8a0 legacy=0x7f645c0d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:09.462 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.459+0000 7f645a000640 1 --2- 172.21.15.121:0/1844163176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64400039b0 0x7f6440003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:09.462 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.459+0000 7f645a000640 1 --2- 172.21.15.121:0/1844163176 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f645c06dbb0 0x7f6440004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:09.462 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.459+0000 7f645a000640 1 --2- 172.21.15.121:0/1844163176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6440004d40 0x7f64400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:09.462 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.459+0000 7f645a000640 1 -- 172.21.15.121:0/1844163176 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f645c06e7c0 con 0x7f6440004d40 2024-04-20T10:21:09.462 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.459+0000 7f645a000640 1 -- 172.21.15.121:0/1844163176 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f645c06eaa0 con 0x7f64400039b0 2024-04-20T10:21:09.462 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.459+0000 7f645a000640 1 -- 172.21.15.121:0/1844163176 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f644000b9c0 con 0x7f645c06dbb0 2024-04-20T10:21:09.462 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.460+0000 7f645b400640 1 --2- 172.21.15.121:0/1844163176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6440004d40 0x7f64400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:09.462 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.460+0000 7f6453e00640 1 --2- 172.21.15.121:0/1844163176 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f645c06dbb0 0x7f6440004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:09.462 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.460+0000 7f645aa00640 1 --2- 172.21.15.121:0/1844163176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64400039b0 0x7f6440003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:09.463 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.460+0000 7f645b400640 1 -- 172.21.15.121:0/1844163176 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f645c06dbb0 msgr2=0x7f6440004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:09.463 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.460+0000 7f645b400640 1 --2- 172.21.15.121:0/1844163176 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f645c06dbb0 0x7f6440004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:09.463 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.460+0000 7f645b400640 1 -- 172.21.15.121:0/1844163176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64400039b0 msgr2=0x7f6440003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:09.463 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.460+0000 7f645b400640 1 --2- 172.21.15.121:0/1844163176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64400039b0 0x7f6440003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:09.463 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.460+0000 7f645b400640 1 -- 172.21.15.121:0/1844163176 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6440003520 con 0x7f6440004d40 2024-04-20T10:21:09.464 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.461+0000 7f645b400640 1 --2- 172.21.15.121:0/1844163176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6440004d40 0x7f64400051c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6454009fc0 tx=0x7f6440005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d219237f98841e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:09.464 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.461+0000 7f645a000640 1 -- 172.21.15.121:0/1844163176 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6454039070 con 0x7f6440004d40 2024-04-20T10:21:09.464 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.461+0000 7f645a000640 1 -- 172.21.15.121:0/1844163176 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f645402cc20 con 0x7f6440004d40 2024-04-20T10:21:09.464 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.461+0000 7f645a000640 1 -- 172.21.15.121:0/1844163176 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6454009c10 con 0x7f6440004d40 2024-04-20T10:21:09.464 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.462+0000 7f6461a00640 1 -- 172.21.15.121:0/1844163176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6440004d40 msgr2=0x7f64400051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:09.464 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.462+0000 7f6461a00640 1 --2- 172.21.15.121:0/1844163176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6440004d40 0x7f64400051c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6454009fc0 tx=0x7f6440005830 comp rx=0 tx=0).stop 2024-04-20T10:21:09.465 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.462+0000 7f645b400640 1 -- 172.21.15.121:0/1844163176 reap_dead start 2024-04-20T10:21:09.465 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.462+0000 7f6461a00640 1 -- 172.21.15.121:0/1844163176 shutdown_connections 2024-04-20T10:21:09.465 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.462+0000 7f6461a00640 1 -- 172.21.15.121:0/1844163176 >> 172.21.15.121:0/1844163176 conn(0x7f645c0b3350 msgr2=0x7f645c0b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:09.465 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.462+0000 7f6461a00640 1 -- 172.21.15.121:0/1844163176 shutdown_connections 2024-04-20T10:21:09.465 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.462+0000 7f6461a00640 1 -- 172.21.15.121:0/1844163176 wait complete. 2024-04-20T10:21:09.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.464+0000 7f6461a00640 1 Processor -- start 2024-04-20T10:21:09.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.464+0000 7f6461a00640 1 -- start start 2024-04-20T10:21:09.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.465+0000 7f6461a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6440004d40 0x7f645c147190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:09.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.465+0000 7f6461a00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f645c1476d0 0x7f645c14d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:09.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.465+0000 7f6461a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f645c14d580 0x7f645c14bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:09.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.465+0000 7f6461a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f645c06fef0 con 0x7f645c14d580 2024-04-20T10:21:09.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.465+0000 7f6461a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f645c06fd70 con 0x7f6440004d40 2024-04-20T10:21:09.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.465+0000 7f6461a00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f645c0ad680 con 0x7f645c1476d0 2024-04-20T10:21:09.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.466+0000 7f645b400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f645c14d580 0x7f645c14bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:09.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.466+0000 7f645b400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f645c14d580 0x7f645c14bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42278/0 (socket says 172.21.15.121:42278) 2024-04-20T10:21:09.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.466+0000 7f645b400640 1 -- 172.21.15.121:0/3205030265 learned_addr learned my addr 172.21.15.121:0/3205030265 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:09.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.466+0000 7f6453e00640 1 --2- 172.21.15.121:0/3205030265 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f645c1476d0 0x7f645c14d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:09.468 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.466+0000 7f645aa00640 1 --2- 172.21.15.121:0/3205030265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6440004d40 0x7f645c147190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:09.468 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.466+0000 7f645b400640 1 -- 172.21.15.121:0/3205030265 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f645c1476d0 msgr2=0x7f645c14d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:09.468 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.466+0000 7f645b400640 1 --2- 172.21.15.121:0/3205030265 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f645c1476d0 0x7f645c14d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:09.468 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.466+0000 7f645b400640 1 -- 172.21.15.121:0/3205030265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6440004d40 msgr2=0x7f645c147190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:09.468 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.466+0000 7f645b400640 1 --2- 172.21.15.121:0/3205030265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6440004d40 0x7f645c147190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:09.468 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.466+0000 7f645b400640 1 -- 172.21.15.121:0/3205030265 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645c14da90 con 0x7f645c14d580 2024-04-20T10:21:09.468 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.466+0000 7f645b400640 1 --2- 172.21.15.121:0/3205030265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f645c14d580 0x7f645c14bb10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6454002230 tx=0x7f6454009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:09.469 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.467+0000 7f6452a00640 1 -- 172.21.15.121:0/3205030265 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6454039030 con 0x7f645c14d580 2024-04-20T10:21:09.469 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.467+0000 7f6452a00640 1 -- 172.21.15.121:0/3205030265 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6454043070 con 0x7f645c14d580 2024-04-20T10:21:09.469 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.467+0000 7f6452a00640 1 -- 172.21.15.121:0/3205030265 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645403d7f0 con 0x7f645c14d580 2024-04-20T10:21:09.469 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.467+0000 7f6461a00640 1 -- 172.21.15.121:0/3205030265 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f645c14dd70 con 0x7f645c14d580 2024-04-20T10:21:09.469 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.467+0000 7f6461a00640 1 -- 172.21.15.121:0/3205030265 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f645c14b5f0 con 0x7f645c14d580 2024-04-20T10:21:09.469 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.468+0000 7f6461a00640 1 -- 172.21.15.121:0/3205030265 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f645c068980 con 0x7f645c14d580 2024-04-20T10:21:09.470 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.469+0000 7f6452a00640 1 -- 172.21.15.121:0/3205030265 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6454005d10 con 0x7f645c14d580 2024-04-20T10:21:09.470 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.470+0000 7f6452a00640 1 --2- 172.21.15.121:0/3205030265 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f643806c920 0x7f643806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:09.470 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.470+0000 7f6452a00640 1 -- 172.21.15.121:0/3205030265 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f64540ae2a0 con 0x7f645c14d580 2024-04-20T10:21:09.471 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.470+0000 7f645aa00640 1 --2- 172.21.15.121:0/3205030265 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f643806c920 0x7f643806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:09.471 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.471+0000 7f645aa00640 1 --2- 172.21.15.121:0/3205030265 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f643806c920 0x7f643806ede0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f644c001eb0 tx=0x7f644c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:09.475 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.474+0000 7f6452a00640 1 -- 172.21.15.121:0/3205030265 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f645404d020 con 0x7f645c14d580 2024-04-20T10:21:09.712 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.711+0000 7f6461a00640 1 -- 172.21.15.121:0/3205030265 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f645c15cc00 con 0x7f645c14d580 2024-04-20T10:21:09.713 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.713+0000 7f6452a00640 1 -- 172.21.15.121:0/3205030265 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5204 (secure 0 0 0) 0x7f645407bd40 con 0x7f645c14d580 2024-04-20T10:21:09.713 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:21:09.714 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1376743826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1376743826},{"type":"v1","addr":"172.21.15.123:6833","nonce":1376743826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/999167058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":999167058},{"type":"v1","addr":"172.21.15.121:6835","nonce":999167058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2160736935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2160736935},{"type":"v1","addr":"172.21.15.121:6837","nonce":2160736935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:21:02.048590+0000","modified":"2024-04-20T10:21:07.336873+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6835/3336775250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3336775250},{"type":"v1","addr":"172.21.15.123:6835","nonce":3336775250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1}]} 2024-04-20T10:21:09.714 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 6 2024-04-20T10:21:09.716 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.715+0000 7f6461a00640 1 -- 172.21.15.121:0/3205030265 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f643806c920 msgr2=0x7f643806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:09.716 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.715+0000 7f6461a00640 1 --2- 172.21.15.121:0/3205030265 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f643806c920 0x7f643806ede0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f644c001eb0 tx=0x7f644c009040 comp rx=0 tx=0).stop 2024-04-20T10:21:09.716 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.715+0000 7f6461a00640 1 -- 172.21.15.121:0/3205030265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f645c14d580 msgr2=0x7f645c14bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:09.716 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.715+0000 7f6461a00640 1 --2- 172.21.15.121:0/3205030265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f645c14d580 0x7f645c14bb10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6454002230 tx=0x7f6454009750 comp rx=0 tx=0).stop 2024-04-20T10:21:09.716 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.716+0000 7f6461a00640 1 -- 172.21.15.121:0/3205030265 shutdown_connections 2024-04-20T10:21:09.716 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.716+0000 7f6461a00640 1 --2- 172.21.15.121:0/3205030265 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f645c14d580 0x7f645c14bb10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:09.717 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.716+0000 7f6461a00640 1 --2- 172.21.15.121:0/3205030265 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f645c1476d0 0x7f645c14d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:09.717 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.716+0000 7f6461a00640 1 --2- 172.21.15.121:0/3205030265 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f643806c920 0x7f643806ede0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:09.717 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.716+0000 7f6461a00640 1 --2- 172.21.15.121:0/3205030265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6440004d40 0x7f645c147190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:09.717 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.716+0000 7f6461a00640 1 -- 172.21.15.121:0/3205030265 >> 172.21.15.121:0/3205030265 conn(0x7f645c0b3350 msgr2=0x7f645c072300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:09.717 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.716+0000 7f6461a00640 1 -- 172.21.15.121:0/3205030265 shutdown_connections 2024-04-20T10:21:09.717 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.716+0000 7f6461a00640 1 -- 172.21.15.121:0/3205030265 wait complete. 2024-04-20T10:21:09.733 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:21:09.921 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.921+0000 7f09e9400640 1 Processor -- start 2024-04-20T10:21:09.922 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.921+0000 7f09e9400640 1 -- start start 2024-04-20T10:21:09.923 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.922+0000 7f09e9400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f09e406dbb0 0x7f09e406e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:09.923 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.922+0000 7f09e9400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09e406f060 con 0x7f09e415e8a0 2024-04-20T10:21:09.923 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.922+0000 7f09e9400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09e406e7c0 con 0x7f09e40d0a70 2024-04-20T10:21:09.923 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.922+0000 7f09e9400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f09e406eaa0 con 0x7f09e406dbb0 2024-04-20T10:21:09.923 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.923+0000 7f09e2800640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f09e415e8a0 0x7f09e40d0360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:35884/0 (socket says 172.21.15.121:35884) 2024-04-20T10:21:09.923 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.923+0000 7f09e2800640 1 -- 172.21.15.121:0/1847040726 learned_addr learned my addr 172.21.15.121:0/1847040726 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:09.926 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.923+0000 7f09e1400640 1 -- 172.21.15.121:0/1847040726 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1881022684 0 0) 0x7f09e406f060 con 0x7f09e415e8a0 2024-04-20T10:21:09.926 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.923+0000 7f09e1400640 1 -- 172.21.15.121:0/1847040726 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f09c8003520 con 0x7f09e415e8a0 2024-04-20T10:21:09.926 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.923+0000 7f09e3200640 1 --2- 172.21.15.121:0/1847040726 >> v2:172.21.15.123:3300/0 conn(0x7f09e406dbb0 0x7f09e406e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:09.926 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.923+0000 7f09e1400640 1 -- 172.21.15.121:0/1847040726 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3464502618 0 0) 0x7f09c8003520 con 0x7f09e415e8a0 2024-04-20T10:21:09.927 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.923+0000 7f09e1400640 1 -- 172.21.15.121:0/1847040726 >> v2:172.21.15.123:3300/0 conn(0x7f09e406dbb0 msgr2=0x7f09e406e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:09.927 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.923+0000 7f09e1400640 1 --2- 172.21.15.121:0/1847040726 >> v2:172.21.15.123:3300/0 conn(0x7f09e406dbb0 0x7f09e406e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:09.927 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.923+0000 7f09e1400640 1 -- 172.21.15.121:0/1847040726 >> v1:172.21.15.123:6789/0 conn(0x7f09e40d0a70 legacy=0x7f09e40ad680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:09.927 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.923+0000 7f09e1400640 1 -- 172.21.15.121:0/1847040726 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e406ed80 con 0x7f09e415e8a0 2024-04-20T10:21:09.927 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.923+0000 7f09e1400640 1 -- 172.21.15.121:0/1847040726 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f09d4001670 con 0x7f09e415e8a0 2024-04-20T10:21:09.927 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.924+0000 7f09e1400640 1 -- 172.21.15.121:0/1847040726 >> v1:172.21.15.121:6789/0 conn(0x7f09e415e8a0 legacy=0x7f09e40d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:09.927 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.924+0000 7f09e1400640 1 --2- 172.21.15.121:0/1847040726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09c8003a00 0x7f09c8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:09.927 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.924+0000 7f09e1400640 1 --2- 172.21.15.121:0/1847040726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09c8004800 0x7f09c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:09.927 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.924+0000 7f09e1400640 1 --2- 172.21.15.121:0/1847040726 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09c8005210 0x7f09c800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:09.927 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.924+0000 7f09e1400640 1 -- 172.21.15.121:0/1847040726 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f09e406eaa0 con 0x7f09c8004800 2024-04-20T10:21:09.927 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.924+0000 7f09e1400640 1 -- 172.21.15.121:0/1847040726 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f09c800c160 con 0x7f09c8003a00 2024-04-20T10:21:09.927 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.924+0000 7f09e1400640 1 -- 172.21.15.121:0/1847040726 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f09c800c360 con 0x7f09c8005210 2024-04-20T10:21:09.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.924+0000 7f09e1e00640 1 --2- 172.21.15.121:0/1847040726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09c8004800 0x7f09c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:09.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.924+0000 7f09e2800640 1 --2- 172.21.15.121:0/1847040726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09c8003a00 0x7f09c8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:09.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.924+0000 7f09e1e00640 1 -- 172.21.15.121:0/1847040726 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09c8005210 msgr2=0x7f09c800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:09.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.924+0000 7f09e1e00640 1 --2- 172.21.15.121:0/1847040726 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09c8005210 0x7f09c800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:09.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.924+0000 7f09e1e00640 1 -- 172.21.15.121:0/1847040726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09c8003a00 msgr2=0x7f09c8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:09.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.924+0000 7f09e1e00640 1 --2- 172.21.15.121:0/1847040726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09c8003a00 0x7f09c8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:09.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.924+0000 7f09e1e00640 1 -- 172.21.15.121:0/1847040726 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c8003520 con 0x7f09c8004800 2024-04-20T10:21:09.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.924+0000 7f09e3200640 1 -- 172.21.15.121:0/1847040726 reap_dead start 2024-04-20T10:21:09.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.926+0000 7f09e1e00640 1 --2- 172.21.15.121:0/1847040726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09c8004800 0x7f09c8004c80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f09d0001770 tx=0x7f09c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90b1013be6ce6f4e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:09.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.926+0000 7f09e1400640 1 -- 172.21.15.121:0/1847040726 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d0001390 con 0x7f09c8004800 2024-04-20T10:21:09.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.926+0000 7f09e1400640 1 -- 172.21.15.121:0/1847040726 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09d00052d0 con 0x7f09c8004800 2024-04-20T10:21:09.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.926+0000 7f09e1400640 1 -- 172.21.15.121:0/1847040726 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d0003830 con 0x7f09c8004800 2024-04-20T10:21:09.930 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.926+0000 7f09e9400640 1 -- 172.21.15.121:0/1847040726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09c8004800 msgr2=0x7f09c8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:09.931 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.926+0000 7f09e9400640 1 --2- 172.21.15.121:0/1847040726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09c8004800 0x7f09c8004c80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f09d0001770 tx=0x7f09c8005830 comp rx=0 tx=0).stop 2024-04-20T10:21:09.931 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.926+0000 7f09e9400640 1 -- 172.21.15.121:0/1847040726 shutdown_connections 2024-04-20T10:21:09.931 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.926+0000 7f09e9400640 1 --2- 172.21.15.121:0/1847040726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09c8004800 0x7f09c8004c80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:09.931 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.926+0000 7f09e9400640 1 -- 172.21.15.121:0/1847040726 >> 172.21.15.121:0/1847040726 conn(0x7f09e40b3350 msgr2=0x7f09e40b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:09.931 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.927+0000 7f09e9400640 1 -- 172.21.15.121:0/1847040726 shutdown_connections 2024-04-20T10:21:09.931 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.927+0000 7f09e9400640 1 -- 172.21.15.121:0/1847040726 wait complete. 2024-04-20T10:21:09.931 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.928+0000 7f09e9400640 1 Processor -- start 2024-04-20T10:21:09.931 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.928+0000 7f09e9400640 1 -- start start 2024-04-20T10:21:09.931 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.929+0000 7f09e9400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09c8004800 0x7f09e409da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:09.931 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.929+0000 7f09e9400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09e4150b90 0x7f09e4151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:09.931 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.929+0000 7f09e9400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09e40af010 0x7f09e40af490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:09.931 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.929+0000 7f09e9400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f09e406fef0 con 0x7f09c8004800 2024-04-20T10:21:09.931 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.929+0000 7f09e9400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f09e406fd70 con 0x7f09e4150b90 2024-04-20T10:21:09.931 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.929+0000 7f09e9400640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f09e406e1e0 con 0x7f09e40af010 2024-04-20T10:21:09.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.930+0000 7f09e2800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09c8004800 0x7f09e409da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:09.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.930+0000 7f09e2800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09c8004800 0x7f09e409da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42302/0 (socket says 172.21.15.121:42302) 2024-04-20T10:21:09.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.930+0000 7f09e2800640 1 -- 172.21.15.121:0/3408788477 learned_addr learned my addr 172.21.15.121:0/3408788477 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:09.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.930+0000 7f09e3200640 1 --2- 172.21.15.121:0/3408788477 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09e40af010 0x7f09e40af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:09.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.930+0000 7f09e1e00640 1 --2- 172.21.15.121:0/3408788477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09e4150b90 0x7f09e4151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:09.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.930+0000 7f09e2800640 1 -- 172.21.15.121:0/3408788477 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09e40af010 msgr2=0x7f09e40af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:09.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.930+0000 7f09e2800640 1 --2- 172.21.15.121:0/3408788477 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09e40af010 0x7f09e40af490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:09.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.930+0000 7f09e2800640 1 -- 172.21.15.121:0/3408788477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09e4150b90 msgr2=0x7f09e4151010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:09.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.930+0000 7f09e2800640 1 --2- 172.21.15.121:0/3408788477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09e4150b90 0x7f09e4151010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:09.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.930+0000 7f09e2800640 1 -- 172.21.15.121:0/3408788477 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e409dfc0 con 0x7f09c8004800 2024-04-20T10:21:09.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.930+0000 7f09e2800640 1 --2- 172.21.15.121:0/3408788477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09c8004800 0x7f09e409da80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f09d400e800 tx=0x7f09d400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:09.933 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.930+0000 7f09daa00640 1 -- 172.21.15.121:0/3408788477 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d400c660 con 0x7f09c8004800 2024-04-20T10:21:09.933 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.931+0000 7f09daa00640 1 -- 172.21.15.121:0/3408788477 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09d4018070 con 0x7f09c8004800 2024-04-20T10:21:09.933 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.931+0000 7f09daa00640 1 -- 172.21.15.121:0/3408788477 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d400ca40 con 0x7f09c8004800 2024-04-20T10:21:09.933 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.931+0000 7f09e9400640 1 -- 172.21.15.121:0/3408788477 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e4152ce0 con 0x7f09c8004800 2024-04-20T10:21:09.933 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.931+0000 7f09e9400640 1 -- 172.21.15.121:0/3408788477 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e41519e0 con 0x7f09c8004800 2024-04-20T10:21:09.933 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.932+0000 7f09e9400640 1 -- 172.21.15.121:0/3408788477 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09e40aecd0 con 0x7f09c8004800 2024-04-20T10:21:09.933 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.933+0000 7f09daa00640 1 -- 172.21.15.121:0/3408788477 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f09d400cbe0 con 0x7f09c8004800 2024-04-20T10:21:09.934 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.934+0000 7f09daa00640 1 --2- 172.21.15.121:0/3408788477 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f09c406c920 0x7f09c406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:09.934 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.934+0000 7f09daa00640 1 -- 172.21.15.121:0/3408788477 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f09d408b930 con 0x7f09c8004800 2024-04-20T10:21:09.934 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.934+0000 7f09e1e00640 1 --2- 172.21.15.121:0/3408788477 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f09c406c920 0x7f09c406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:09.935 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.934+0000 7f09e1e00640 1 --2- 172.21.15.121:0/3408788477 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f09c406c920 0x7f09c406ede0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f09d0033000 tx=0x7f09d0009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:09.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:09.938+0000 7f09daa00640 1 -- 172.21.15.121:0/3408788477 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f09e409dfc0 con 0x7f09c8004800 2024-04-20T10:21:10.125 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.125+0000 7f09e9400640 1 -- 172.21.15.121:0/3408788477 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f09e4068980 con 0x7f09c8004800 2024-04-20T10:21:10.128 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.128+0000 7f09daa00640 1 -- 172.21.15.121:0/3408788477 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7f09d4059980 con 0x7f09c8004800 2024-04-20T10:21:10.128 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:21:10.129 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":25,"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","created":"2024-04-20T10:20:38.076076+0000","modified":"2024-04-20T10:21:07.336542+0000","last_up_change":"2024-04-20T10:20:50.162700+0000","last_in_change":"2024-04-20T10:20:41.510382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:20:51.141365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:21:00.436937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:21:01.026957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:21:04.862693+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"629a7b8c-a40a-4a2a-a3cc-d9f357774310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6809","nonce":1044588835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6811","nonce":1044588835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6815","nonce":1044588835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6813","nonce":1044588835}]},"public_addr":"172.21.15.121:6809/1044588835","cluster_addr":"172.21.15.121:6811/1044588835","heartbeat_back_addr":"172.21.15.121:6815/1044588835","heartbeat_front_addr":"172.21.15.121:6813/1044588835","state":["exists","up"]},{"osd":1,"uuid":"f2570399-aa50-4ff3-bf5d-3d09207290db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6801","nonce":3505199624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6803","nonce":3505199624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6807","nonce":3505199624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6805","nonce":3505199624}]},"public_addr":"172.21.15.121:6801/3505199624","cluster_addr":"172.21.15.121:6803/3505199624","heartbeat_back_addr":"172.21.15.121:6807/3505199624","heartbeat_front_addr":"172.21.15.121:6805/3505199624","state":["exists","up"]},{"osd":2,"uuid":"50893807-dc30-49f4-b41f-739366626658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6817","nonce":3656724207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6819","nonce":3656724207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6827","nonce":3656724207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6822","nonce":3656724207}]},"public_addr":"172.21.15.121:6817/3656724207","cluster_addr":"172.21.15.121:6819/3656724207","heartbeat_back_addr":"172.21.15.121:6827/3656724207","heartbeat_front_addr":"172.21.15.121:6822/3656724207","state":["exists","up"]},{"osd":3,"uuid":"dadb01bc-5491-4ffc-8824-d48a057240b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6823","nonce":2232342832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6826","nonce":2232342832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6831","nonce":2232342832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6829","nonce":2232342832}]},"public_addr":"172.21.15.121:6823/2232342832","cluster_addr":"172.21.15.121:6826/2232342832","heartbeat_back_addr":"172.21.15.121:6831/2232342832","heartbeat_front_addr":"172.21.15.121:6829/2232342832","state":["exists","up"]},{"osd":4,"uuid":"b3f3b4a1-b375-4526-b69e-344ab24c2da1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6801","nonce":99353111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6803","nonce":99353111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6807","nonce":99353111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6805","nonce":99353111}]},"public_addr":"172.21.15.123:6801/99353111","cluster_addr":"172.21.15.123:6803/99353111","heartbeat_back_addr":"172.21.15.123:6807/99353111","heartbeat_front_addr":"172.21.15.123:6805/99353111","state":["exists","up"]},{"osd":5,"uuid":"2ee24a08-1624-4961-9642-d7b510282484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6818","nonce":2504374688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6822","nonce":2504374688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6830","nonce":2504374688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6826","nonce":2504374688}]},"public_addr":"172.21.15.123:6818/2504374688","cluster_addr":"172.21.15.123:6822/2504374688","heartbeat_back_addr":"172.21.15.123:6830/2504374688","heartbeat_front_addr":"172.21.15.123:6826/2504374688","state":["exists","up"]},{"osd":6,"uuid":"186118cc-6128-49de-acca-55260945c1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6819","nonce":4139299599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6823","nonce":4139299599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6829","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6831","nonce":4139299599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6827","nonce":4139299599}]},"public_addr":"172.21.15.123:6819/4139299599","cluster_addr":"172.21.15.123:6823/4139299599","heartbeat_back_addr":"172.21.15.123:6831/4139299599","heartbeat_front_addr":"172.21.15.123:6827/4139299599","state":["exists","up"]},{"osd":7,"uuid":"a00aac6e-4bd9-415a-a93d-ae85210865ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6809","nonce":4043981069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6811","nonce":4043981069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6815","nonce":4043981069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6813","nonce":4043981069}]},"public_addr":"172.21.15.123:6809/4043981069","cluster_addr":"172.21.15.123:6811/4043981069","heartbeat_back_addr":"172.21.15.123:6815/4043981069","heartbeat_front_addr":"172.21.15.123:6813/4043981069","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.904540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.958585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.616367+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.885244+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.103127+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.167248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.884348+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.862356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:21:10.131 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.130+0000 7f09e9400640 1 -- 172.21.15.121:0/3408788477 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f09c406c920 msgr2=0x7f09c406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:10.131 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.130+0000 7f09e9400640 1 --2- 172.21.15.121:0/3408788477 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f09c406c920 0x7f09c406ede0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f09d0033000 tx=0x7f09d0009030 comp rx=0 tx=0).stop 2024-04-20T10:21:10.131 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.130+0000 7f09e9400640 1 -- 172.21.15.121:0/3408788477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09c8004800 msgr2=0x7f09e409da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:10.131 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.130+0000 7f09e9400640 1 --2- 172.21.15.121:0/3408788477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09c8004800 0x7f09e409da80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f09d400e800 tx=0x7f09d400ecc0 comp rx=0 tx=0).stop 2024-04-20T10:21:10.131 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.130+0000 7f09e9400640 1 -- 172.21.15.121:0/3408788477 shutdown_connections 2024-04-20T10:21:10.131 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.130+0000 7f09e9400640 1 --2- 172.21.15.121:0/3408788477 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09e40af010 0x7f09e40af490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:10.131 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.130+0000 7f09e9400640 1 --2- 172.21.15.121:0/3408788477 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f09c406c920 0x7f09c406ede0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:10.132 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.130+0000 7f09e9400640 1 --2- 172.21.15.121:0/3408788477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09e4150b90 0x7f09e4151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:10.132 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.130+0000 7f09e9400640 1 --2- 172.21.15.121:0/3408788477 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09c8004800 0x7f09e409da80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:10.132 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.130+0000 7f09e9400640 1 -- 172.21.15.121:0/3408788477 >> 172.21.15.121:0/3408788477 conn(0x7f09e40b3350 msgr2=0x7f09e415ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:10.132 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.131+0000 7f09e9400640 1 -- 172.21.15.121:0/3408788477 shutdown_connections 2024-04-20T10:21:10.132 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.131+0000 7f09e9400640 1 -- 172.21.15.121:0/3408788477 wait complete. 2024-04-20T10:21:10.551 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T10:21:02.048590+0000', 'modified': '2024-04-20T10:21:07.336873+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4281}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4281': {'gid': 4281, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.123:6835/3336775250', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6834', 'nonce': 3336775250}, {'type': 'v1', 'addr': '172.21.15.123:6835', 'nonce': 3336775250}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4281, 'qdb_cluster': [4281]} 2024-04-20T10:21:10.551 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T10:21:10.551 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-20T10:21:10.762 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.762+0000 7f50d8e00640 1 Processor -- start 2024-04-20T10:21:10.763 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.762+0000 7f50d8e00640 1 -- start start 2024-04-20T10:21:10.764 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.764+0000 7f50d8e00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f50d415e740 0x7f50d40cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:10.764 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.764+0000 7f50d8e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50d406dd00 con 0x7f50d406c010 2024-04-20T10:21:10.764 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.764+0000 7f50d8e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50d40cfac0 con 0x7f50d40709c0 2024-04-20T10:21:10.764 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.764+0000 7f50d8e00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f50d40b4240 con 0x7f50d415e740 2024-04-20T10:21:10.764 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.764+0000 7f50d2800640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f50d406c010 0x7f50d40b46d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:35894/0 (socket says 172.21.15.121:35894) 2024-04-20T10:21:10.764 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.764+0000 7f50d2800640 1 -- 172.21.15.121:0/312074217 learned_addr learned my addr 172.21.15.121:0/312074217 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:10.766 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.764+0000 7f50d1e00640 1 --2- 172.21.15.121:0/312074217 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f50d415e740 0x7f50d40cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:10.766 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.764+0000 7f50d0a00640 1 -- 172.21.15.121:0/312074217 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2889848148 0 0) 0x7f50d406dd00 con 0x7f50d406c010 2024-04-20T10:21:10.766 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.765+0000 7f50d0a00640 1 -- 172.21.15.121:0/312074217 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50b8003520 con 0x7f50d406c010 2024-04-20T10:21:10.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.765+0000 7f50d0a00640 1 -- 172.21.15.121:0/312074217 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3447249033 0 0) 0x7f50b8003520 con 0x7f50d406c010 2024-04-20T10:21:10.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.765+0000 7f50d0a00640 1 -- 172.21.15.121:0/312074217 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f50d415e740 msgr2=0x7f50d40cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:10.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.765+0000 7f50d0a00640 1 --2- 172.21.15.121:0/312074217 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f50d415e740 0x7f50d40cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:10.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.765+0000 7f50d0a00640 1 -- 172.21.15.121:0/312074217 >> v1:172.21.15.123:6789/0 conn(0x7f50d40709c0 legacy=0x7f50d406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:10.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.765+0000 7f50d0a00640 1 -- 172.21.15.121:0/312074217 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d40cfda0 con 0x7f50d406c010 2024-04-20T10:21:10.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.765+0000 7f50d0a00640 1 -- 172.21.15.121:0/312074217 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f50cc0019f0 con 0x7f50d406c010 2024-04-20T10:21:10.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.765+0000 7f50d0a00640 1 -- 172.21.15.121:0/312074217 >> v1:172.21.15.121:6789/0 conn(0x7f50d406c010 legacy=0x7f50d40b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:10.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.765+0000 7f50d0a00640 1 --2- 172.21.15.121:0/312074217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50b80039e0 0x7f50b8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:10.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.765+0000 7f50d0a00640 1 --2- 172.21.15.121:0/312074217 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f50d415e740 0x7f50b8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:10.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.765+0000 7f50d0a00640 1 --2- 172.21.15.121:0/312074217 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50b8004d40 0x7f50b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:10.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.765+0000 7f50d0a00640 1 -- 172.21.15.121:0/312074217 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f50d40b4240 con 0x7f50b8004d40 2024-04-20T10:21:10.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.765+0000 7f50d0a00640 1 -- 172.21.15.121:0/312074217 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f50b800bba0 con 0x7f50b80039e0 2024-04-20T10:21:10.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.765+0000 7f50d0a00640 1 -- 172.21.15.121:0/312074217 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f50b800bda0 con 0x7f50d415e740 2024-04-20T10:21:10.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.765+0000 7f50d2800640 1 --2- 172.21.15.121:0/312074217 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50b8004d40 0x7f50b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:10.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.765+0000 7f50d1e00640 1 --2- 172.21.15.121:0/312074217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50b80039e0 0x7f50b8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:10.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.765+0000 7f50d1400640 1 --2- 172.21.15.121:0/312074217 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f50d415e740 0x7f50b8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:10.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.766+0000 7f50d2800640 1 -- 172.21.15.121:0/312074217 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f50d415e740 msgr2=0x7f50b8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:10.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.766+0000 7f50d2800640 1 --2- 172.21.15.121:0/312074217 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f50d415e740 0x7f50b8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:10.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.766+0000 7f50d2800640 1 -- 172.21.15.121:0/312074217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50b80039e0 msgr2=0x7f50b8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:10.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.766+0000 7f50d2800640 1 --2- 172.21.15.121:0/312074217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50b80039e0 0x7f50b8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:10.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.766+0000 7f50d2800640 1 -- 172.21.15.121:0/312074217 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50b8003520 con 0x7f50b8004d40 2024-04-20T10:21:10.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.766+0000 7f50d1400640 1 --2- 172.21.15.121:0/312074217 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f50d415e740 0x7f50b8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:10.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.767+0000 7f50d2800640 1 --2- 172.21.15.121:0/312074217 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50b8004d40 0x7f50b80051c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f50cc00a830 tx=0x7f50b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf713826c67901d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:10.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.767+0000 7f50d0a00640 1 -- 172.21.15.121:0/312074217 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50cc00a480 con 0x7f50b8004d40 2024-04-20T10:21:10.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.767+0000 7f50d0a00640 1 -- 172.21.15.121:0/312074217 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50cc034630 con 0x7f50b8004d40 2024-04-20T10:21:10.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.767+0000 7f50d0a00640 1 -- 172.21.15.121:0/312074217 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50cc034a40 con 0x7f50b8004d40 2024-04-20T10:21:10.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.768+0000 7f50d8e00640 1 -- 172.21.15.121:0/312074217 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50b8004d40 msgr2=0x7f50b80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:10.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.768+0000 7f50d8e00640 1 --2- 172.21.15.121:0/312074217 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50b8004d40 0x7f50b80051c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f50cc00a830 tx=0x7f50b8005830 comp rx=0 tx=0).stop 2024-04-20T10:21:10.770 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.768+0000 7f50d2800640 1 -- 172.21.15.121:0/312074217 reap_dead start 2024-04-20T10:21:10.770 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.768+0000 7f50d8e00640 1 -- 172.21.15.121:0/312074217 shutdown_connections 2024-04-20T10:21:10.770 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.768+0000 7f50d8e00640 1 -- 172.21.15.121:0/312074217 >> 172.21.15.121:0/312074217 conn(0x7f50d40cd530 msgr2=0x7f50d40cd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:10.770 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.768+0000 7f50d8e00640 1 -- 172.21.15.121:0/312074217 shutdown_connections 2024-04-20T10:21:10.770 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.768+0000 7f50d8e00640 1 -- 172.21.15.121:0/312074217 wait complete. 2024-04-20T10:21:10.770 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.770+0000 7f50d8e00640 1 Processor -- start 2024-04-20T10:21:10.771 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.770+0000 7f50d8e00640 1 -- start start 2024-04-20T10:21:10.773 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.771+0000 7f50d8e00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f50b8004d40 0x7f50d414a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:10.773 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.771+0000 7f50d8e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50d415ee30 0x7f50d40aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:10.773 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.771+0000 7f50d8e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50d415f290 0x7f50d40af230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:10.773 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.771+0000 7f50d8e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f50d40cf3c0 con 0x7f50d415f290 2024-04-20T10:21:10.773 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.771+0000 7f50d8e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f50d415eb20 con 0x7f50d415ee30 2024-04-20T10:21:10.773 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.771+0000 7f50d8e00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f50d40cf540 con 0x7f50b8004d40 2024-04-20T10:21:10.773 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.771+0000 7f50d2800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50d415f290 0x7f50d40af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:10.773 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.771+0000 7f50d2800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50d415f290 0x7f50d40af230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42312/0 (socket says 172.21.15.121:42312) 2024-04-20T10:21:10.773 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.771+0000 7f50d2800640 1 -- 172.21.15.121:0/3664838053 learned_addr learned my addr 172.21.15.121:0/3664838053 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:10.774 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.771+0000 7f50d2800640 1 -- 172.21.15.121:0/3664838053 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f50b8004d40 msgr2=0x7f50d414a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:10.774 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.771+0000 7f50d2800640 1 --2- 172.21.15.121:0/3664838053 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f50b8004d40 0x7f50d414a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:10.774 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.771+0000 7f50d2800640 1 -- 172.21.15.121:0/3664838053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50d415ee30 msgr2=0x7f50d40aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:10.774 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.771+0000 7f50d2800640 1 --2- 172.21.15.121:0/3664838053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50d415ee30 0x7f50d40aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:10.774 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.771+0000 7f50d2800640 1 -- 172.21.15.121:0/3664838053 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d41e8550 con 0x7f50d415f290 2024-04-20T10:21:10.774 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.772+0000 7f50d2800640 1 --2- 172.21.15.121:0/3664838053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50d415f290 0x7f50d40af230 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f50cc00a950 tx=0x7f50d40cff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:10.774 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.772+0000 7f50ca000640 1 -- 172.21.15.121:0/3664838053 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50cc045070 con 0x7f50d415f290 2024-04-20T10:21:10.774 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.772+0000 7f50ca000640 1 -- 172.21.15.121:0/3664838053 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50cc03a040 con 0x7f50d415f290 2024-04-20T10:21:10.774 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.772+0000 7f50d8e00640 1 -- 172.21.15.121:0/3664838053 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50d41e8830 con 0x7f50d415f290 2024-04-20T10:21:10.774 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.772+0000 7f50ca000640 1 -- 172.21.15.121:0/3664838053 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50cc00a480 con 0x7f50d415f290 2024-04-20T10:21:10.774 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.772+0000 7f50d8e00640 1 -- 172.21.15.121:0/3664838053 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50d41e8df0 con 0x7f50d415f290 2024-04-20T10:21:10.775 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.774+0000 7f50ca000640 1 -- 172.21.15.121:0/3664838053 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f50cc048070 con 0x7f50d415f290 2024-04-20T10:21:10.775 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.775+0000 7f50ca000640 1 --2- 172.21.15.121:0/3664838053 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f50b406c920 0x7f50b406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:10.775 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.775+0000 7f50ca000640 1 -- 172.21.15.121:0/3664838053 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f50cc0ad900 con 0x7f50d415f290 2024-04-20T10:21:10.775 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.775+0000 7f50d1e00640 1 --2- 172.21.15.121:0/3664838053 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f50b406c920 0x7f50b406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:10.776 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.776+0000 7f50dba5a740 1 Processor -- start 2024-04-20T10:21:10.776 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.776+0000 7f50dba5a740 1 -- start start 2024-04-20T10:21:10.776 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.776+0000 7f50d1e00640 1 --2- 172.21.15.121:0/3664838053 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f50b406c920 0x7f50b406ede0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f50c4006fd0 tx=0x7f50c400b2e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:10.777 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.777+0000 7f50dba5a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55594a377460 0x55594a32ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:10.777 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.777+0000 7f50dba5a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55594a32d4e0 0x55594a32efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:10.777 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.777+0000 7f50dba5a740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55594a329930 0x55594a359770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:10.778 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.777+0000 7f50dba5a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55594a2959d0 con 0x55594a32d4e0 2024-04-20T10:21:10.778 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.777+0000 7f50dba5a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55594a23c270 con 0x55594a377460 2024-04-20T10:21:10.778 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.777+0000 7f50dba5a740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x55594a291990 con 0x55594a329930 2024-04-20T10:21:10.778 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.777+0000 7f50d1400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55594a32d4e0 0x55594a32efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:10.778 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.777+0000 7f50d1400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55594a32d4e0 0x55594a32efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42322/0 (socket says 172.21.15.121:42322) 2024-04-20T10:21:10.778 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.777+0000 7f50d1400640 1 -- 172.21.15.121:0/3396159393 learned_addr learned my addr 172.21.15.121:0/3396159393 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:10.779 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.777+0000 7f50d1400640 1 -- 172.21.15.121:0/3396159393 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55594a329930 msgr2=0x55594a359770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:10.779 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.777+0000 7f50d1400640 1 --2- 172.21.15.121:0/3396159393 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55594a329930 0x55594a359770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:10.780 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.777+0000 7f50d1e00640 1 --2- 172.21.15.121:0/3396159393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55594a377460 0x55594a32ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:10.780 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.777+0000 7f50d1400640 1 -- 172.21.15.121:0/3396159393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55594a377460 msgr2=0x55594a32ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:10.780 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.778+0000 7f50d1400640 1 --2- 172.21.15.121:0/3396159393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55594a377460 0x55594a32ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:10.780 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.778+0000 7f50d1400640 1 -- 172.21.15.121:0/3396159393 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555949fa82d0 con 0x55594a32d4e0 2024-04-20T10:21:10.780 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.778+0000 7f50d1400640 1 --2- 172.21.15.121:0/3396159393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55594a32d4e0 0x55594a32efd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f50c00017a0 tx=0x7f50c0001c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44446004b8e75b6c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:10.780 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.778+0000 7f50bea00640 1 -- 172.21.15.121:0/3396159393 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c0016020 con 0x55594a32d4e0 2024-04-20T10:21:10.780 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.778+0000 7f50bea00640 1 -- 172.21.15.121:0/3396159393 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50c000db60 con 0x55594a32d4e0 2024-04-20T10:21:10.780 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.778+0000 7f50bea00640 1 -- 172.21.15.121:0/3396159393 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c00123f0 con 0x55594a32d4e0 2024-04-20T10:21:10.780 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.779+0000 7f50dba5a740 1 -- 172.21.15.121:0/3396159393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55594a32d4e0 msgr2=0x55594a32efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:10.780 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.779+0000 7f50dba5a740 1 --2- 172.21.15.121:0/3396159393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55594a32d4e0 0x55594a32efd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f50c00017a0 tx=0x7f50c0001c60 comp rx=0 tx=0).stop 2024-04-20T10:21:10.780 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.779+0000 7f50dba5a740 1 -- 172.21.15.121:0/3396159393 shutdown_connections 2024-04-20T10:21:10.780 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.779+0000 7f50dba5a740 1 --2- 172.21.15.121:0/3396159393 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55594a329930 0x55594a359770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:10.781 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.779+0000 7f50dba5a740 1 --2- 172.21.15.121:0/3396159393 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55594a32d4e0 0x55594a32efd0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:10.781 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.779+0000 7f50dba5a740 1 --2- 172.21.15.121:0/3396159393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55594a377460 0x55594a32ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:10.781 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.779+0000 7f50dba5a740 1 -- 172.21.15.121:0/3396159393 >> 172.21.15.121:0/3396159393 conn(0x55594a32bbf0 msgr2=0x55594a374770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:10.781 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.779+0000 7f50dba5a740 1 -- 172.21.15.121:0/3396159393 shutdown_connections 2024-04-20T10:21:10.781 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.779+0000 7f50dba5a740 1 -- 172.21.15.121:0/3396159393 wait complete. 2024-04-20T10:21:10.784 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.780+0000 7f50dba5a740 1 -- start start 2024-04-20T10:21:10.784 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.780+0000 7f50dba5a740 1 Processor -- start 2024-04-20T10:21:10.784 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.781+0000 7f50dba5a740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55594a32d4e0 0x55594a3892c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:10.784 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.781+0000 7f50dba5a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55594a377460 0x55594a356200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:10.784 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.781+0000 7f50dba5a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55594a3502f0 0x55594a356740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:10.784 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.781+0000 7f50dba5a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55594a32d9f0 con 0x55594a3502f0 2024-04-20T10:21:10.784 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.781+0000 7f50dba5a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55594a24f5e0 con 0x55594a377460 2024-04-20T10:21:10.784 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.781+0000 7f50dba5a740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x55594a32dcd0 con 0x55594a32d4e0 2024-04-20T10:21:10.785 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.781+0000 7f50dba5a740 10 client.0 _ll_register_callbacks cb 0x555949fa7ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:21:10.785 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.781+0000 7f50d2800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55594a3502f0 0x55594a356740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:10.785 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.781+0000 7f50d2800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55594a3502f0 0x55594a356740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42330/0 (socket says 172.21.15.121:42330) 2024-04-20T10:21:10.785 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.781+0000 7f50d2800640 1 -- 172.21.15.121:0/2805020834 learned_addr learned my addr 172.21.15.121:0/2805020834 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:10.785 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.781+0000 7f50d1e00640 1 --2- 172.21.15.121:0/2805020834 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55594a32d4e0 0x55594a3892c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:10.785 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.781+0000 7f50d2800640 1 -- 172.21.15.121:0/2805020834 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55594a32d4e0 msgr2=0x55594a3892c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:10.785 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.781+0000 7f50d2800640 1 --2- 172.21.15.121:0/2805020834 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55594a32d4e0 0x55594a3892c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:10.785 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.782+0000 7f50d2800640 1 -- 172.21.15.121:0/2805020834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55594a377460 msgr2=0x55594a356200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:10.785 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.782+0000 7f50d2800640 1 --2- 172.21.15.121:0/2805020834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55594a377460 0x55594a356200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:10.785 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.782+0000 7f50d2800640 1 -- 172.21.15.121:0/2805020834 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55594a0726b0 con 0x55594a3502f0 2024-04-20T10:21:10.785 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.782+0000 7f50d2800640 1 --2- 172.21.15.121:0/2805020834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55594a3502f0 0x55594a356740 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f50cc07b320 tx=0x7f50cc07b450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c91b407113ea219 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:10.785 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.782+0000 7f50b3400640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-04-20T10:21:10.785 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.782+0000 7f50b3400640 1 -- 172.21.15.121:0/2805020834 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50cc079690 con 0x55594a3502f0 2024-04-20T10:21:10.786 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.782+0000 7f50b3400640 1 -- 172.21.15.121:0/2805020834 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f50cc077a00 con 0x55594a3502f0 2024-04-20T10:21:10.786 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.782+0000 7f50dba5a740 1 -- 172.21.15.121:0/2805020834 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50c000db60 con 0x55594a3502f0 2024-04-20T10:21:10.786 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.782+0000 7f50b3400640 1 -- 172.21.15.121:0/2805020834 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7f50cc07a7e0 con 0x55594a3502f0 2024-04-20T10:21:10.786 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.782+0000 7f50dba5a740 10 client.4401 fetch_fsmap learned FSMap version 6 2024-04-20T10:21:10.786 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.782+0000 7f50dba5a740 1 -- 172.21.15.121:0/2805020834 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x55594a37a620 con 0x55594a3502f0 2024-04-20T10:21:10.786 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.782+0000 7f50b3400640 1 -- 172.21.15.121:0/2805020834 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50c000db60 con 0x55594a3502f0 2024-04-20T10:21:10.786 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.783+0000 7f50b3400640 1 -- 172.21.15.121:0/2805020834 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 6) ==== 3503+0+0 (secure 0 0 0) 0x7f50cc07a980 con 0x55594a3502f0 2024-04-20T10:21:10.786 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.783+0000 7f50dba5a740 10 client.4401 fetch_fsmap finished waiting for FSMap version 6 2024-04-20T10:21:10.786 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.783+0000 7f50dba5a740 10 client.4401 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-20T10:21:10.786 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.783+0000 7f50dba5a740 20 client.4401 populate_metadata read hostname 'smithi121' 2024-04-20T10:21:10.786 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.783+0000 7f50dba5a740 1 --2- 172.21.15.121:0/2805020834 >> [v2:172.21.15.123:6834/3336775250,v1:172.21.15.123:6835/3336775250] conn(0x55594a37bb90 0x55594a37bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:10.786 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.783+0000 7f50dba5a740 4 client.4401 mds_command: new command op to 4281 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:21:10.786 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.783+0000 7f50dba5a740 1 -- 172.21.15.121:0/2805020834 --> [v2:172.21.15.123:6834/3336775250,v1:172.21.15.123:6835/3336775250] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55594a2a33f0 con 0x55594a37bb90 2024-04-20T10:21:10.786 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.783+0000 7f50d1e00640 1 --2- 172.21.15.121:0/2805020834 >> [v2:172.21.15.123:6834/3336775250,v1:172.21.15.123:6835/3336775250] conn(0x55594a37bb90 0x55594a37bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:10.788 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.784+0000 7f50d1e00640 1 --2- 172.21.15.121:0/2805020834 >> [v2:172.21.15.123:6834/3336775250,v1:172.21.15.123:6835/3336775250] conn(0x55594a37bb90 0x55594a37bf70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a507fdd7cb375d43 server_cookie=b3ad19c0043976fe in_seq=0 out_seq=0 2024-04-20T10:21:10.788 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.784+0000 7f50b3400640 10 client.4401 ms_handle_connect on v2:172.21.15.123:6834/3336775250 2024-04-20T10:21:10.788 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.787+0000 7f50b3400640 1 -- 172.21.15.121:0/2805020834 <== mds.0 v2:172.21.15.123:6834/3336775250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55594a2a33f0 con 0x55594a37bb90 2024-04-20T10:21:10.788 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.787+0000 7f50b3400640 10 client.4401 handle_command_reply: tid=0 2024-04-20T10:21:10.788 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.787+0000 7f50dba5a740 1 client.4401 shutdown 2024-04-20T10:21:10.788 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.787+0000 7f50dba5a740 1 -- 172.21.15.121:0/2805020834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55594a3502f0 msgr2=0x55594a356740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:10.788 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.787+0000 7f50dba5a740 1 --2- 172.21.15.121:0/2805020834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55594a3502f0 0x55594a356740 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f50cc07b320 tx=0x7f50cc07b450 comp rx=0 tx=0).stop 2024-04-20T10:21:10.788 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.787+0000 7f50dba5a740 1 -- 172.21.15.121:0/2805020834 shutdown_connections 2024-04-20T10:21:10.788 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.787+0000 7f50dba5a740 1 --2- 172.21.15.121:0/2805020834 >> [v2:172.21.15.123:6834/3336775250,v1:172.21.15.123:6835/3336775250] conn(0x55594a37bb90 0x55594a37bf70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:10.788 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.787+0000 7f50dba5a740 1 --2- 172.21.15.121:0/2805020834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55594a3502f0 0x55594a356740 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:10.788 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.787+0000 7f50dba5a740 1 --2- 172.21.15.121:0/2805020834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55594a377460 0x55594a356200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:10.788 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.787+0000 7f50dba5a740 1 --2- 172.21.15.121:0/2805020834 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55594a32d4e0 0x55594a3892c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:10.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.787+0000 7f50b3400640 0 client.4401 ms_handle_reset on v2:172.21.15.123:6834/3336775250 2024-04-20T10:21:10.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.787+0000 7f50dba5a740 1 -- 172.21.15.121:0/2805020834 >> 172.21.15.121:0/2805020834 conn(0x55594a329930 msgr2=0x55594a32f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:10.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.787+0000 7f50dba5a740 1 -- 172.21.15.121:0/2805020834 shutdown_connections 2024-04-20T10:21:10.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.787+0000 7f50dba5a740 1 -- 172.21.15.121:0/2805020834 wait complete. 2024-04-20T10:21:10.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.788+0000 7f50dba5a740 20 client.4401 trim_cache size 0 max 16384 2024-04-20T10:21:10.807 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.806+0000 7f50dba5a740 1 Processor -- start 2024-04-20T10:21:10.807 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.807+0000 7f50dba5a740 1 -- start start 2024-04-20T10:21:10.808 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.807+0000 7f50dba5a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55594a329930 0x55594a32a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:10.808 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.807+0000 7f50dba5a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55594a32d4e0 0x55594a3518a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:10.808 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.807+0000 7f50dba5a740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55594a3502f0 0x55594a351de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:10.808 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.807+0000 7f50dba5a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f50d42018b0 con 0x55594a32d4e0 2024-04-20T10:21:10.808 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.807+0000 7f50dba5a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f50d4200c10 con 0x55594a329930 2024-04-20T10:21:10.808 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.807+0000 7f50dba5a740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f50d4201260 con 0x55594a3502f0 2024-04-20T10:21:10.809 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.808+0000 7f50d1400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55594a32d4e0 0x55594a3518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:10.809 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.808+0000 7f50d1400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55594a32d4e0 0x55594a3518a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42338/0 (socket says 172.21.15.121:42338) 2024-04-20T10:21:10.809 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.808+0000 7f50d1400640 1 -- 172.21.15.121:0/1512410421 learned_addr learned my addr 172.21.15.121:0/1512410421 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:10.810 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.808+0000 7f50d2800640 1 --2- 172.21.15.121:0/1512410421 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55594a3502f0 0x55594a351de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:10.810 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.808+0000 7f50d1e00640 1 --2- 172.21.15.121:0/1512410421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55594a329930 0x55594a32a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:10.810 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.808+0000 7f50d1400640 1 -- 172.21.15.121:0/1512410421 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55594a3502f0 msgr2=0x55594a351de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:10.810 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.808+0000 7f50d1400640 1 --2- 172.21.15.121:0/1512410421 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55594a3502f0 0x55594a351de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:10.810 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.808+0000 7f50d1400640 1 -- 172.21.15.121:0/1512410421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55594a329930 msgr2=0x55594a32a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:10.810 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.808+0000 7f50d1400640 1 --2- 172.21.15.121:0/1512410421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55594a329930 0x55594a32a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:10.810 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.808+0000 7f50d1400640 1 -- 172.21.15.121:0/1512410421 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55594a37c990 con 0x55594a32d4e0 2024-04-20T10:21:10.810 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.809+0000 7f50d1400640 1 --2- 172.21.15.121:0/1512410421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55594a32d4e0 0x55594a3518a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f50c0014000 tx=0x7f50c0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6660191cc4a288fe server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:10.810 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.809+0000 7f50be000640 1 -- 172.21.15.121:0/1512410421 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c000db40 con 0x55594a32d4e0 2024-04-20T10:21:10.810 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.809+0000 7f50be000640 1 -- 172.21.15.121:0/1512410421 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50c000dd00 con 0x55594a32d4e0 2024-04-20T10:21:10.810 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.809+0000 7f50be000640 1 -- 172.21.15.121:0/1512410421 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c0007280 con 0x55594a32d4e0 2024-04-20T10:21:10.811 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.810+0000 7f50dba5a740 1 -- 172.21.15.121:0/1512410421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55594a32d4e0 msgr2=0x55594a3518a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:10.811 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.810+0000 7f50dba5a740 1 --2- 172.21.15.121:0/1512410421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55594a32d4e0 0x55594a3518a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f50c0014000 tx=0x7f50c0010000 comp rx=0 tx=0).stop 2024-04-20T10:21:10.811 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.810+0000 7f50dba5a740 1 -- 172.21.15.121:0/1512410421 shutdown_connections 2024-04-20T10:21:10.811 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.810+0000 7f50dba5a740 1 --2- 172.21.15.121:0/1512410421 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55594a3502f0 0x55594a351de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:10.812 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.810+0000 7f50dba5a740 1 --2- 172.21.15.121:0/1512410421 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55594a32d4e0 0x55594a3518a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:10.812 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.810+0000 7f50dba5a740 1 --2- 172.21.15.121:0/1512410421 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55594a329930 0x55594a32a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:10.812 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.810+0000 7f50dba5a740 1 -- 172.21.15.121:0/1512410421 >> 172.21.15.121:0/1512410421 conn(0x55594a388c00 msgr2=0x55594a389000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:10.812 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.810+0000 7f50dba5a740 1 -- 172.21.15.121:0/1512410421 shutdown_connections 2024-04-20T10:21:10.812 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.810+0000 7f50dba5a740 1 -- 172.21.15.121:0/1512410421 wait complete. 2024-04-20T10:21:10.812 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.810+0000 7f50dba5a740 1 -- start start 2024-04-20T10:21:10.812 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.811+0000 7f50dba5a740 1 Processor -- start 2024-04-20T10:21:10.815 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.812+0000 7f50dba5a740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55594a329930 0x55594a372140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:10.815 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.812+0000 7f50dba5a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55594a32d4e0 0x55594a372680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:10.815 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.812+0000 7f50dba5a740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55594a3502f0 0x55594a3efd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:10.815 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.812+0000 7f50dba5a740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55594a32d9f0 con 0x55594a3502f0 2024-04-20T10:21:10.815 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.812+0000 7f50dba5a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55594a24f5e0 con 0x55594a32d4e0 2024-04-20T10:21:10.815 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.812+0000 7f50dba5a740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x55594a37abe0 con 0x55594a329930 2024-04-20T10:21:10.815 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.812+0000 7f50dba5a740 10 client.0 _ll_register_callbacks cb 0x7f50cc00a620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:21:10.815 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.812+0000 7f50d2800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55594a3502f0 0x55594a3efd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:10.815 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.812+0000 7f50d2800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55594a3502f0 0x55594a3efd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42340/0 (socket says 172.21.15.121:42340) 2024-04-20T10:21:10.815 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.812+0000 7f50d2800640 1 -- 172.21.15.121:0/215124487 learned_addr learned my addr 172.21.15.121:0/215124487 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:10.815 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.812+0000 7f50d1e00640 1 --2- 172.21.15.121:0/215124487 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55594a329930 0x55594a372140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:10.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.812+0000 7f50d1400640 1 --2- 172.21.15.121:0/215124487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55594a32d4e0 0x55594a372680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:10.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.812+0000 7f50d2800640 1 -- 172.21.15.121:0/215124487 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55594a329930 msgr2=0x55594a372140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:10.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.812+0000 7f50d2800640 1 --2- 172.21.15.121:0/215124487 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55594a329930 0x55594a372140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:10.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.812+0000 7f50d2800640 1 -- 172.21.15.121:0/215124487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55594a32d4e0 msgr2=0x55594a372680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:10.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.812+0000 7f50d2800640 1 --2- 172.21.15.121:0/215124487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55594a32d4e0 0x55594a372680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:10.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.812+0000 7f50d2800640 1 -- 172.21.15.121:0/215124487 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55594a373e10 con 0x55594a3502f0 2024-04-20T10:21:10.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.812+0000 7f50d1400640 1 --2- 172.21.15.121:0/215124487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55594a32d4e0 0x55594a372680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:10.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.813+0000 7f50d2800640 1 --2- 172.21.15.121:0/215124487 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55594a3502f0 0x55594a3efd20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f50cc07f850 tx=0x7f50cc07d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=189a01606b392248 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:10.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.813+0000 7f50b3400640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-04-20T10:21:10.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.813+0000 7f50b3400640 1 -- 172.21.15.121:0/215124487 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50cc07aed0 con 0x55594a3502f0 2024-04-20T10:21:10.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.813+0000 7f50b3400640 1 -- 172.21.15.121:0/215124487 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f50cc077130 con 0x55594a3502f0 2024-04-20T10:21:10.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.813+0000 7f50dba5a740 1 -- 172.21.15.121:0/215124487 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50c000dd00 con 0x55594a3502f0 2024-04-20T10:21:10.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.813+0000 7f50b3400640 1 -- 172.21.15.121:0/215124487 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7f50cc079760 con 0x55594a3502f0 2024-04-20T10:21:10.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.813+0000 7f50dba5a740 10 client.4407 fetch_fsmap learned FSMap version 6 2024-04-20T10:21:10.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.813+0000 7f50dba5a740 1 -- 172.21.15.121:0/215124487 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x55594a373e10 con 0x55594a3502f0 2024-04-20T10:21:10.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.813+0000 7f50b3400640 1 -- 172.21.15.121:0/215124487 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50c000dd00 con 0x55594a3502f0 2024-04-20T10:21:10.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.813+0000 7f50b3400640 1 -- 172.21.15.121:0/215124487 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 6) ==== 3503+0+0 (secure 0 0 0) 0x7f50cc093ac0 con 0x55594a3502f0 2024-04-20T10:21:10.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.813+0000 7f50dba5a740 10 client.4407 fetch_fsmap finished waiting for FSMap version 6 2024-04-20T10:21:10.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.813+0000 7f50dba5a740 10 client.4407 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-20T10:21:10.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.813+0000 7f50dba5a740 20 client.4407 populate_metadata read hostname 'smithi121' 2024-04-20T10:21:10.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.813+0000 7f50dba5a740 1 --2- 172.21.15.121:0/215124487 >> [v2:172.21.15.123:6834/3336775250,v1:172.21.15.123:6835/3336775250] conn(0x55594a377460 0x55594a3f0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:10.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.813+0000 7f50dba5a740 4 client.4407 mds_command: new command op to 4281 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T10:21:10.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.813+0000 7f50dba5a740 1 -- 172.21.15.121:0/215124487 --> [v2:172.21.15.123:6834/3336775250,v1:172.21.15.123:6835/3336775250] -- command(tid 0: {"prefix": "status"}) -- 0x55594a29d260 con 0x55594a377460 2024-04-20T10:21:10.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.814+0000 7f50d1e00640 1 --2- 172.21.15.121:0/215124487 >> [v2:172.21.15.123:6834/3336775250,v1:172.21.15.123:6835/3336775250] conn(0x55594a377460 0x55594a3f0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:10.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.815+0000 7f50d1e00640 1 --2- 172.21.15.121:0/215124487 >> [v2:172.21.15.123:6834/3336775250,v1:172.21.15.123:6835/3336775250] conn(0x55594a377460 0x55594a3f0530 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6f0f7b6506d1929 server_cookie=198ff1140bf947a7 in_seq=0 out_seq=0 2024-04-20T10:21:10.819 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.815+0000 7f50b3400640 10 client.4407 ms_handle_connect on v2:172.21.15.123:6834/3336775250 2024-04-20T10:21:10.819 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.815+0000 7f50b3400640 1 -- 172.21.15.121:0/215124487 <== mds.0 v2:172.21.15.123:6834/3336775250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55594a29d260 con 0x55594a377460 2024-04-20T10:21:10.819 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.815+0000 7f50b3400640 10 client.4407 handle_command_reply: tid=0 2024-04-20T10:21:10.819 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.815+0000 7f50dba5a740 1 client.4407 shutdown 2024-04-20T10:21:10.819 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.816+0000 7f50dba5a740 1 -- 172.21.15.121:0/215124487 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55594a3502f0 msgr2=0x55594a3efd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:10.819 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.816+0000 7f50dba5a740 1 --2- 172.21.15.121:0/215124487 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55594a3502f0 0x55594a3efd20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f50cc07f850 tx=0x7f50cc07d000 comp rx=0 tx=0).stop 2024-04-20T10:21:10.819 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.816+0000 7f50dba5a740 1 -- 172.21.15.121:0/215124487 shutdown_connections 2024-04-20T10:21:10.819 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.816+0000 7f50dba5a740 1 --2- 172.21.15.121:0/215124487 >> [v2:172.21.15.123:6834/3336775250,v1:172.21.15.123:6835/3336775250] conn(0x55594a377460 0x55594a3f0530 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:10.819 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.816+0000 7f50dba5a740 1 --2- 172.21.15.121:0/215124487 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55594a3502f0 0x55594a3efd20 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:10.819 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.816+0000 7f50b3400640 0 client.4407 ms_handle_reset on v2:172.21.15.123:6834/3336775250 2024-04-20T10:21:10.819 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.816+0000 7f50dba5a740 1 --2- 172.21.15.121:0/215124487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55594a32d4e0 0x55594a372680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:10.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.816+0000 7f50dba5a740 1 --2- 172.21.15.121:0/215124487 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55594a329930 0x55594a372140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:10.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.816+0000 7f50dba5a740 1 -- 172.21.15.121:0/215124487 >> 172.21.15.121:0/215124487 conn(0x55594a388c00 msgr2=0x55594a32f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:10.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.816+0000 7f50dba5a740 1 -- 172.21.15.121:0/215124487 shutdown_connections 2024-04-20T10:21:10.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.816+0000 7f50dba5a740 1 -- 172.21.15.121:0/215124487 wait complete. 2024-04-20T10:21:10.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.816+0000 7f50dba5a740 20 client.4407 trim_cache size 0 max 16384 2024-04-20T10:21:10.820 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T10:21:10.820 INFO:teuthology.orchestra.run.smithi121.stdout: "cluster_fsid": "ceaac016-4ab2-472d-a36e-7bfe9497c762", 2024-04-20T10:21:10.820 INFO:teuthology.orchestra.run.smithi121.stdout: "whoami": 0, 2024-04-20T10:21:10.821 INFO:teuthology.orchestra.run.smithi121.stdout: "id": 4281, 2024-04-20T10:21:10.821 INFO:teuthology.orchestra.run.smithi121.stdout: "want_state": "up:active", 2024-04-20T10:21:10.821 INFO:teuthology.orchestra.run.smithi121.stdout: "state": "up:active", 2024-04-20T10:21:10.821 INFO:teuthology.orchestra.run.smithi121.stdout: "fs_name": "cephfs", 2024-04-20T10:21:10.821 INFO:teuthology.orchestra.run.smithi121.stdout: "rank_uptime": 8.2149349669999996, 2024-04-20T10:21:10.821 INFO:teuthology.orchestra.run.smithi121.stdout: "mdsmap_epoch": 6, 2024-04-20T10:21:10.821 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch": 25, 2024-04-20T10:21:10.821 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch_barrier": 25, 2024-04-20T10:21:10.821 INFO:teuthology.orchestra.run.smithi121.stdout: "uptime": 13.256181509999999 2024-04-20T10:21:10.821 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T10:21:10.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.817+0000 7f50d8e00640 1 -- 172.21.15.121:0/3664838053 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f50b406c920 msgr2=0x7f50b406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:10.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.817+0000 7f50d8e00640 1 --2- 172.21.15.121:0/3664838053 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f50b406c920 0x7f50b406ede0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f50c4006fd0 tx=0x7f50c400b2e0 comp rx=0 tx=0).stop 2024-04-20T10:21:10.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.817+0000 7f50d8e00640 1 -- 172.21.15.121:0/3664838053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50d415f290 msgr2=0x7f50d40af230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:10.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.817+0000 7f50d8e00640 1 --2- 172.21.15.121:0/3664838053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50d415f290 0x7f50d40af230 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f50cc00a950 tx=0x7f50d40cff70 comp rx=0 tx=0).stop 2024-04-20T10:21:10.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.817+0000 7f50d8e00640 1 -- 172.21.15.121:0/3664838053 shutdown_connections 2024-04-20T10:21:10.823 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.817+0000 7f50d8e00640 1 --2- 172.21.15.121:0/3664838053 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f50d415f290 0x7f50d40af230 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:10.823 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.817+0000 7f50d8e00640 1 --2- 172.21.15.121:0/3664838053 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f50b406c920 0x7f50b406ede0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:10.823 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.817+0000 7f50d8e00640 1 --2- 172.21.15.121:0/3664838053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50d415ee30 0x7f50d40aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:10.823 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.817+0000 7f50d8e00640 1 --2- 172.21.15.121:0/3664838053 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f50b8004d40 0x7f50d414a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:10.823 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.817+0000 7f50d8e00640 1 -- 172.21.15.121:0/3664838053 >> 172.21.15.121:0/3664838053 conn(0x7f50d40cd530 msgr2=0x7f50d406f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:10.823 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.817+0000 7f50d8e00640 1 -- 172.21.15.121:0/3664838053 shutdown_connections 2024-04-20T10:21:10.823 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:10.817+0000 7f50d8e00640 1 -- 172.21.15.121:0/3664838053 wait complete. 2024-04-20T10:21:10.833 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-20T10:21:10.833 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-20T10:21:10.833 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T10:21:11.013 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.013+0000 7ff3d1600640 1 Processor -- start 2024-04-20T10:21:11.013 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.013+0000 7ff3d1600640 1 -- start start 2024-04-20T10:21:11.016 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.014+0000 7ff3d1600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff3cc06cde0 0x7ff3cc0d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:11.016 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.014+0000 7ff3d1600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3cc06c190 con 0x7ff3cc161d50 2024-04-20T10:21:11.016 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.014+0000 7ff3d1600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3cc06c470 con 0x7ff3cc0b3330 2024-04-20T10:21:11.016 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.014+0000 7ff3d1600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7ff3cc162130 con 0x7ff3cc06cde0 2024-04-20T10:21:11.016 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.015+0000 7ff3cbe00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7ff3cc161d50 0x7ff3cc06c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:35896/0 (socket says 172.21.15.121:35896) 2024-04-20T10:21:11.016 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.015+0000 7ff3cbe00640 1 -- 172.21.15.121:0/2714970010 learned_addr learned my addr 172.21.15.121:0/2714970010 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:11.016 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.015+0000 7ff3c3400640 1 --2- 172.21.15.121:0/2714970010 >> v2:172.21.15.121:3300/0 conn(0x7ff3cc06cde0 0x7ff3cc0d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:11.016 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.015+0000 7ff3cb400640 1 -- 172.21.15.121:0/2714970010 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1971587291 0 0) 0x7ff3cc06c190 con 0x7ff3cc161d50 2024-04-20T10:21:11.016 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.015+0000 7ff3cb400640 1 -- 172.21.15.121:0/2714970010 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff3b0003520 con 0x7ff3cc161d50 2024-04-20T10:21:11.016 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.015+0000 7ff3c3400640 1 -- 172.21.15.121:0/2714970010 >> v1:172.21.15.123:6789/0 conn(0x7ff3cc0b3330 legacy=0x7ff3cc0b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:11.016 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.015+0000 7ff3c3400640 1 -- 172.21.15.121:0/2714970010 >> v1:172.21.15.121:6789/0 conn(0x7ff3cc161d50 legacy=0x7ff3cc06c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:11.016 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.015+0000 7ff3c3400640 1 -- 172.21.15.121:0/2714970010 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3cc0d2e00 con 0x7ff3cc06cde0 2024-04-20T10:21:11.018 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.016+0000 7ff3c3400640 1 --2- 172.21.15.121:0/2714970010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3cc06cde0 0x7ff3cc0d38e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff3b40097a0 tx=0x7ff3b402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8433bf68b223922 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:11.018 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.016+0000 7ff3cb400640 1 -- 172.21.15.121:0/2714970010 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4034070 con 0x7ff3cc06cde0 2024-04-20T10:21:11.018 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.017+0000 7ff3cb400640 1 -- 172.21.15.121:0/2714970010 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3b4030610 con 0x7ff3cc06cde0 2024-04-20T10:21:11.018 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.017+0000 7ff3cb400640 1 -- 172.21.15.121:0/2714970010 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b40309c0 con 0x7ff3cc06cde0 2024-04-20T10:21:11.018 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.017+0000 7ff3d1600640 1 -- 172.21.15.121:0/2714970010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3cc06cde0 msgr2=0x7ff3cc0d38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:11.018 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.017+0000 7ff3d1600640 1 --2- 172.21.15.121:0/2714970010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3cc06cde0 0x7ff3cc0d38e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff3b40097a0 tx=0x7ff3b402b960 comp rx=0 tx=0).stop 2024-04-20T10:21:11.018 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.017+0000 7ff3d1600640 1 -- 172.21.15.121:0/2714970010 shutdown_connections 2024-04-20T10:21:11.018 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.017+0000 7ff3d1600640 1 --2- 172.21.15.121:0/2714970010 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3cc06cde0 0x7ff3cc0d38e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:11.018 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.017+0000 7ff3d1600640 1 -- 172.21.15.121:0/2714970010 >> 172.21.15.121:0/2714970010 conn(0x7ff3cc0d4450 msgr2=0x7ff3cc0d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:11.018 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.017+0000 7ff3d1600640 1 -- 172.21.15.121:0/2714970010 shutdown_connections 2024-04-20T10:21:11.018 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.017+0000 7ff3d1600640 1 -- 172.21.15.121:0/2714970010 wait complete. 2024-04-20T10:21:11.019 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.019+0000 7ff3d1600640 1 Processor -- start 2024-04-20T10:21:11.019 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.019+0000 7ff3d1600640 1 -- start start 2024-04-20T10:21:11.021 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.020+0000 7ff3d1600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3cc06cde0 0x7ff3cc149920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:11.022 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.020+0000 7ff3d1600640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff3cc0b3330 0x7ff3cc15fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:11.022 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.020+0000 7ff3d1600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3cc161d50 0x7ff3cc0c76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:11.022 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.020+0000 7ff3d1600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc0b3880 con 0x7ff3cc06cde0 2024-04-20T10:21:11.022 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.020+0000 7ff3d1600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc0d33c0 con 0x7ff3cc161d50 2024-04-20T10:21:11.022 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.020+0000 7ff3d1600640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ff3cc0b3a00 con 0x7ff3cc0b3330 2024-04-20T10:21:11.022 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.020+0000 7ff3cbe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3cc06cde0 0x7ff3cc149920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:11.022 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.020+0000 7ff3cbe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3cc06cde0 0x7ff3cc149920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42362/0 (socket says 172.21.15.121:42362) 2024-04-20T10:21:11.022 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.020+0000 7ff3cbe00640 1 -- 172.21.15.121:0/1336316051 learned_addr learned my addr 172.21.15.121:0/1336316051 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:11.022 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.020+0000 7ff3cbe00640 1 -- 172.21.15.121:0/1336316051 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff3cc0b3330 msgr2=0x7ff3cc15fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:11.022 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.020+0000 7ff3cbe00640 1 --2- 172.21.15.121:0/1336316051 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff3cc0b3330 0x7ff3cc15fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:11.022 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.020+0000 7ff3cbe00640 1 -- 172.21.15.121:0/1336316051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3cc161d50 msgr2=0x7ff3cc0c76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:11.022 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.020+0000 7ff3cbe00640 1 --2- 172.21.15.121:0/1336316051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3cc161d50 0x7ff3cc0c76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:11.022 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.021+0000 7ff3cbe00640 1 -- 172.21.15.121:0/1336316051 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3cc147970 con 0x7ff3cc06cde0 2024-04-20T10:21:11.023 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.021+0000 7ff3cbe00640 1 --2- 172.21.15.121:0/1336316051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3cc06cde0 0x7ff3cc149920 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc00c410 tx=0x7ff3bc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:11.023 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.021+0000 7ff3c8c00640 1 -- 172.21.15.121:0/1336316051 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc017070 con 0x7ff3cc06cde0 2024-04-20T10:21:11.023 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.021+0000 7ff3c8c00640 1 -- 172.21.15.121:0/1336316051 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3bc005ce0 con 0x7ff3cc06cde0 2024-04-20T10:21:11.023 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.021+0000 7ff3d1600640 1 -- 172.21.15.121:0/1336316051 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3cc149fa0 con 0x7ff3cc06cde0 2024-04-20T10:21:11.023 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.021+0000 7ff3c8c00640 1 -- 172.21.15.121:0/1336316051 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc0072f0 con 0x7ff3cc06cde0 2024-04-20T10:21:11.023 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.021+0000 7ff3d1600640 1 -- 172.21.15.121:0/1336316051 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3cc1600e0 con 0x7ff3cc06cde0 2024-04-20T10:21:11.023 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.022+0000 7ff3d1600640 1 -- 172.21.15.121:0/1336316051 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3cc075b60 con 0x7ff3cc06cde0 2024-04-20T10:21:11.024 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.023+0000 7ff3c8c00640 1 -- 172.21.15.121:0/1336316051 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff3bc01a070 con 0x7ff3cc06cde0 2024-04-20T10:21:11.025 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.024+0000 7ff3c8c00640 1 --2- 172.21.15.121:0/1336316051 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff3a806c8f0 0x7ff3a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:11.025 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.024+0000 7ff3c8c00640 1 -- 172.21.15.121:0/1336316051 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff3bc08b320 con 0x7ff3cc06cde0 2024-04-20T10:21:11.025 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.025+0000 7ff3c3400640 1 --2- 172.21.15.121:0/1336316051 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff3a806c8f0 0x7ff3a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:11.025 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.025+0000 7ff3c3400640 1 --2- 172.21.15.121:0/1336316051 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff3a806c8f0 0x7ff3a806edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4036000 tx=0x7ff3b4030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:11.029 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.029+0000 7ff3c8c00640 1 -- 172.21.15.121:0/1336316051 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff3cc149fa0 con 0x7ff3cc06cde0 2024-04-20T10:21:11.273 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.273+0000 7ff3d1600640 1 -- 172.21.15.121:0/1336316051 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff3cc0d23f0 con 0x7ff3cc06cde0 2024-04-20T10:21:11.274 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.274+0000 7ff3c8c00640 1 -- 172.21.15.121:0/1336316051 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+88 (secure 0 0 0) 0x7ff3bc0594c0 con 0x7ff3cc06cde0 2024-04-20T10:21:11.274 INFO:teuthology.orchestra.run.smithi121.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-20T10:21:11.277 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.276+0000 7ff3d1600640 1 -- 172.21.15.121:0/1336316051 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff3a806c8f0 msgr2=0x7ff3a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:11.277 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.276+0000 7ff3d1600640 1 --2- 172.21.15.121:0/1336316051 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff3a806c8f0 0x7ff3a806edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4036000 tx=0x7ff3b4030000 comp rx=0 tx=0).stop 2024-04-20T10:21:11.277 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.276+0000 7ff3d1600640 1 -- 172.21.15.121:0/1336316051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3cc06cde0 msgr2=0x7ff3cc149920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:11.277 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.276+0000 7ff3d1600640 1 --2- 172.21.15.121:0/1336316051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3cc06cde0 0x7ff3cc149920 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc00c410 tx=0x7ff3bc00c8d0 comp rx=0 tx=0).stop 2024-04-20T10:21:11.277 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.276+0000 7ff3d1600640 1 -- 172.21.15.121:0/1336316051 shutdown_connections 2024-04-20T10:21:11.277 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.276+0000 7ff3d1600640 1 --2- 172.21.15.121:0/1336316051 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff3a806c8f0 0x7ff3a806edb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:11.277 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.276+0000 7ff3d1600640 1 --2- 172.21.15.121:0/1336316051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3cc161d50 0x7ff3cc0c76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:11.277 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.276+0000 7ff3d1600640 1 --2- 172.21.15.121:0/1336316051 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff3cc0b3330 0x7ff3cc15fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:11.277 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.276+0000 7ff3d1600640 1 --2- 172.21.15.121:0/1336316051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3cc06cde0 0x7ff3cc149920 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:11.277 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.276+0000 7ff3d1600640 1 -- 172.21.15.121:0/1336316051 >> 172.21.15.121:0/1336316051 conn(0x7ff3cc0d4450 msgr2=0x7ff3cc0cf5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:11.277 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.276+0000 7ff3d1600640 1 -- 172.21.15.121:0/1336316051 shutdown_connections 2024-04-20T10:21:11.277 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11.276+0000 7ff3d1600640 1 -- 172.21.15.121:0/1336316051 wait complete. 2024-04-20T10:21:11.294 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-20T10:21:11.295 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T10:21:11.295 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2024-04-20T10:21:11.295 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-20T10:21:11.295 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-20T10:21:11.295 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-20T10:21:11.295 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T10:21:11.295 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T10:21:11.295 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-20T10:21:11.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:11.295 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2024-04-20T10:21:11.346 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T10:21:11.347 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T10:21:11.347 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T10:21:11.347 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:21:11.347 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2024-04-20T10:21:11.347 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:21:11.347 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:21:11.347 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:21:11.347 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2024-04-20T10:21:11.347 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:21:11.347 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:21:11.347 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2024-04-20T10:21:11.347 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:21:11.348 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:21:11.348 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T10:21:11.348 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:21:11.348 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T10:21:11.348 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 42637sec preferred_lft 42637sec 2024-04-20T10:21:11.348 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ec4:7aff:fe8f:cba3/64 scope link 2024-04-20T10:21:11.348 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:21:11.349 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-20T10:21:11.349 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:21:11.349 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-20T10:21:11.349 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link add name ceph-brx type bridge 2024-04-20T10:21:11.350 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip addr flush dev ceph-brx 2024-04-20T10:21:11.350 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set ceph-brx up 2024-04-20T10:21:11.350 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-20T10:21:11.350 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-20T10:21:11.454 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:21:11.823 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:21:11.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:11.833 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T10:21:11.902 INFO:teuthology.orchestra.run.smithi121.stdout:1 2024-04-20T10:21:11.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:11.906 DEBUG:teuthology.orchestra.run.smithi121:> route 2024-04-20T10:21:11.964 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2024-04-20T10:21:11.964 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T10:21:11.964 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T10:21:11.965 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T10:21:11.965 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T10:21:11.965 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:21:11.965 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-20T10:21:11.965 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T10:21:11.965 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T10:21:11.965 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T10:21:11.965 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-20T10:21:12.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:21:12.706 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:21:12.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:12.717 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2024-04-20T10:21:12.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:12.770 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list-id 2024-04-20T10:21:12.819 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:21:12.819 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-20T10:21:12.819 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-20T10:21:12.819 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-20T10:21:12.819 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-20T10:21:12.919 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:21:12.989 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:21:12.999 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-20T10:21:12.999 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:21:12.999 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-20T10:21:12.999 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-20T10:21:12.999 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-20T10:21:13.000 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-20T10:21:13.000 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-20T10:21:13.000 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-20T10:21:13.000 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-20T10:21:13.099 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:21:13.280 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:21:13.289 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:21:13.289 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-20T10:21:13.289 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.0 up 2024-04-20T10:21:13.289 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set dev brx.0 master ceph-brx 2024-04-20T10:21:13.289 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-20T10:21:13.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:21:13.474 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:21:13.483 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-20T10:21:13.483 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T10:21:13.483 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-20T10:21:13.527 INFO:teuthology.orchestra.run.smithi121.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-20T10:21:13.527 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T10:21:13.527 DEBUG:teuthology.orchestra.run.smithi121:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-20T10:21:13.620 DEBUG:teuthology.orchestra.run.smithi121:> sudo modprobe fuse 2024-04-20T10:21:13.701 DEBUG:teuthology.orchestra.run.smithi121:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T10:21:13.752 INFO:teuthology.orchestra.run.smithi121.stdout:/proc 2024-04-20T10:21:13.752 INFO:teuthology.orchestra.run.smithi121.stdout:/sys 2024-04-20T10:21:13.752 INFO:teuthology.orchestra.run.smithi121.stdout:/dev 2024-04-20T10:21:13.752 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/security 2024-04-20T10:21:13.752 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/shm 2024-04-20T10:21:13.752 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/pts 2024-04-20T10:21:13.752 INFO:teuthology.orchestra.run.smithi121.stdout:/run 2024-04-20T10:21:13.753 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/cgroup 2024-04-20T10:21:13.753 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/pstore 2024-04-20T10:21:13.753 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/bpf 2024-04-20T10:21:13.753 INFO:teuthology.orchestra.run.smithi121.stdout:/ 2024-04-20T10:21:13.753 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/selinux 2024-04-20T10:21:13.753 INFO:teuthology.orchestra.run.smithi121.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T10:21:13.753 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/hugepages 2024-04-20T10:21:13.753 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/mqueue 2024-04-20T10:21:13.753 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/debug 2024-04-20T10:21:13.753 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/tracing 2024-04-20T10:21:13.753 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/fuse/connections 2024-04-20T10:21:13.753 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/config 2024-04-20T10:21:13.753 INFO:teuthology.orchestra.run.smithi121.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T10:21:13.753 INFO:teuthology.orchestra.run.smithi121.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T10:21:13.754 INFO:teuthology.orchestra.run.smithi121.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T10:21:13.754 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T10:21:13.754 INFO:teuthology.orchestra.run.smithi121.stdout:/run/user/1000 2024-04-20T10:21:13.754 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph 2024-04-20T10:21:13.754 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T10:21:13.754 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T10:21:13.754 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T10:21:13.754 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T10:21:13.754 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns 2024-04-20T10:21:13.754 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-20T10:21:13.754 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-20T10:21:13.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:13.755 DEBUG:teuthology.orchestra.run.smithi121:> ls /sys/fs/fuse/connections 2024-04-20T10:21:13.801 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T10:21:13.801 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-20T10:21:13.848 DEBUG:teuthology.orchestra.run.smithi121:> sudo modprobe fuse 2024-04-20T10:21:13.918 DEBUG:teuthology.orchestra.run.smithi121:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T10:21:13.961 INFO:teuthology.orchestra.run.smithi121.stdout:/proc 2024-04-20T10:21:13.961 INFO:teuthology.orchestra.run.smithi121.stdout:/sys 2024-04-20T10:21:13.961 INFO:teuthology.orchestra.run.smithi121.stdout:/dev 2024-04-20T10:21:13.961 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/security 2024-04-20T10:21:13.961 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/shm 2024-04-20T10:21:13.961 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/pts 2024-04-20T10:21:13.961 INFO:teuthology.orchestra.run.smithi121.stdout:/run 2024-04-20T10:21:13.961 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/cgroup 2024-04-20T10:21:13.962 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/pstore 2024-04-20T10:21:13.962 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/bpf 2024-04-20T10:21:13.962 INFO:teuthology.orchestra.run.smithi121.stdout:/ 2024-04-20T10:21:13.962 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/selinux 2024-04-20T10:21:13.962 INFO:teuthology.orchestra.run.smithi121.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T10:21:13.962 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/hugepages 2024-04-20T10:21:13.962 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/mqueue 2024-04-20T10:21:13.962 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/debug 2024-04-20T10:21:13.962 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/tracing 2024-04-20T10:21:13.962 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/fuse/connections 2024-04-20T10:21:13.962 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/config 2024-04-20T10:21:13.962 INFO:teuthology.orchestra.run.smithi121.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T10:21:13.962 INFO:teuthology.orchestra.run.smithi121.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T10:21:13.963 INFO:teuthology.orchestra.run.smithi121.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T10:21:13.963 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T10:21:13.963 INFO:teuthology.orchestra.run.smithi121.stdout:/run/user/1000 2024-04-20T10:21:13.963 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph 2024-04-20T10:21:13.963 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T10:21:13.963 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T10:21:13.963 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T10:21:13.963 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T10:21:13.963 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns 2024-04-20T10:21:13.963 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-20T10:21:13.963 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-20T10:21:13.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:13.964 DEBUG:teuthology.orchestra.run.smithi121:> ls /sys/fs/fuse/connections 2024-04-20T10:21:14.035 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi121.stderr:2024-04-20T10:21:14.035+0000 7f7316757480 -1 init, newargv = 0x55b8cc7eade0 newargc=15 2024-04-20T10:21:14.036 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi121.stderr:ceph-fuse[29669]: starting ceph client 2024-04-20T10:21:14.049 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi121.stderr:ceph-fuse[29669]: starting fuse 2024-04-20T10:21:15.147 DEBUG:teuthology.orchestra.run.smithi121:> sudo modprobe fuse 2024-04-20T10:21:15.237 DEBUG:teuthology.orchestra.run.smithi121:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T10:21:15.293 INFO:teuthology.orchestra.run.smithi121.stdout:/proc 2024-04-20T10:21:15.294 INFO:teuthology.orchestra.run.smithi121.stdout:/sys 2024-04-20T10:21:15.294 INFO:teuthology.orchestra.run.smithi121.stdout:/dev 2024-04-20T10:21:15.294 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/security 2024-04-20T10:21:15.294 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/shm 2024-04-20T10:21:15.294 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/pts 2024-04-20T10:21:15.294 INFO:teuthology.orchestra.run.smithi121.stdout:/run 2024-04-20T10:21:15.294 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/cgroup 2024-04-20T10:21:15.294 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/pstore 2024-04-20T10:21:15.294 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/bpf 2024-04-20T10:21:15.294 INFO:teuthology.orchestra.run.smithi121.stdout:/ 2024-04-20T10:21:15.294 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/selinux 2024-04-20T10:21:15.294 INFO:teuthology.orchestra.run.smithi121.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T10:21:15.295 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/hugepages 2024-04-20T10:21:15.295 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/mqueue 2024-04-20T10:21:15.295 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/debug 2024-04-20T10:21:15.295 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/tracing 2024-04-20T10:21:15.295 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/fuse/connections 2024-04-20T10:21:15.295 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/config 2024-04-20T10:21:15.295 INFO:teuthology.orchestra.run.smithi121.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T10:21:15.295 INFO:teuthology.orchestra.run.smithi121.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T10:21:15.295 INFO:teuthology.orchestra.run.smithi121.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T10:21:15.295 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T10:21:15.295 INFO:teuthology.orchestra.run.smithi121.stdout:/run/user/1000 2024-04-20T10:21:15.295 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph 2024-04-20T10:21:15.295 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T10:21:15.295 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T10:21:15.295 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T10:21:15.295 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T10:21:15.296 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns 2024-04-20T10:21:15.296 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-20T10:21:15.296 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-20T10:21:15.296 INFO:teuthology.orchestra.run.smithi121.stdout:/home/ubuntu/cephtest/mnt.admin 2024-04-20T10:21:15.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:15.297 DEBUG:teuthology.orchestra.run.smithi121:> ls /sys/fs/fuse/connections 2024-04-20T10:21:15.346 INFO:teuthology.orchestra.run.smithi121.stdout:41 2024-04-20T10:21:15.347 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-20T10:21:15.347 DEBUG:teuthology.orchestra.run.smithi121:> sudo stdin-killer -- python3 -c ' 2024-04-20T10:21:15.347 DEBUG:teuthology.orchestra.run.smithi121:> import glob 2024-04-20T10:21:15.347 DEBUG:teuthology.orchestra.run.smithi121:> import re 2024-04-20T10:21:15.347 DEBUG:teuthology.orchestra.run.smithi121:> import os 2024-04-20T10:21:15.347 DEBUG:teuthology.orchestra.run.smithi121:> import subprocess 2024-04-20T10:21:15.347 DEBUG:teuthology.orchestra.run.smithi121:> 2024-04-20T10:21:15.347 DEBUG:teuthology.orchestra.run.smithi121:> def _find_admin_socket(client_name): 2024-04-20T10:21:15.347 DEBUG:teuthology.orchestra.run.smithi121:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-20T10:21:15.347 DEBUG:teuthology.orchestra.run.smithi121:> files = glob.glob(asok_path) 2024-04-20T10:21:15.347 DEBUG:teuthology.orchestra.run.smithi121:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-20T10:21:15.348 DEBUG:teuthology.orchestra.run.smithi121:> 2024-04-20T10:21:15.348 DEBUG:teuthology.orchestra.run.smithi121:> # Given a non-glob path, it better be there 2024-04-20T10:21:15.348 DEBUG:teuthology.orchestra.run.smithi121:> if "*" not in asok_path: 2024-04-20T10:21:15.348 DEBUG:teuthology.orchestra.run.smithi121:> assert(len(files) == 1) 2024-04-20T10:21:15.348 DEBUG:teuthology.orchestra.run.smithi121:> return files[0] 2024-04-20T10:21:15.348 DEBUG:teuthology.orchestra.run.smithi121:> 2024-04-20T10:21:15.348 DEBUG:teuthology.orchestra.run.smithi121:> for f in files: 2024-04-20T10:21:15.348 DEBUG:teuthology.orchestra.run.smithi121:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T10:21:15.348 DEBUG:teuthology.orchestra.run.smithi121:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T10:21:15.348 DEBUG:teuthology.orchestra.run.smithi121:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T10:21:15.348 DEBUG:teuthology.orchestra.run.smithi121:> contents = proc_f.read() 2024-04-20T10:21:15.348 DEBUG:teuthology.orchestra.run.smithi121:> if mountpoint in contents: 2024-04-20T10:21:15.348 DEBUG:teuthology.orchestra.run.smithi121:> return f 2024-04-20T10:21:15.348 DEBUG:teuthology.orchestra.run.smithi121:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T10:21:15.348 DEBUG:teuthology.orchestra.run.smithi121:> 2024-04-20T10:21:15.348 DEBUG:teuthology.orchestra.run.smithi121:> print(_find_admin_socket("client.admin")) 2024-04-20T10:21:15.349 DEBUG:teuthology.orchestra.run.smithi121:> ' 2024-04-20T10:21:15.508 INFO:teuthology.orchestra.run.smithi121.stdout:/var/run/ceph/ceph-client.admin.29669.asok 2024-04-20T10:21:15.513 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:21:15.526 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.29669.asok 2024-04-20T10:21:15.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:15.526 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.29669.asok status 2024-04-20T10:21:15.750 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T10:21:15.751 INFO:teuthology.orchestra.run.smithi121.stdout: "metadata": { 2024-04-20T10:21:15.751 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T10:21:15.751 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T10:21:15.751 INFO:teuthology.orchestra.run.smithi121.stdout: "entity_id": "admin", 2024-04-20T10:21:15.751 INFO:teuthology.orchestra.run.smithi121.stdout: "hostname": "smithi121", 2024-04-20T10:21:15.751 INFO:teuthology.orchestra.run.smithi121.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-20T10:21:15.751 INFO:teuthology.orchestra.run.smithi121.stdout: "pid": "29669", 2024-04-20T10:21:15.751 INFO:teuthology.orchestra.run.smithi121.stdout: "root": "/" 2024-04-20T10:21:15.751 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T10:21:15.751 INFO:teuthology.orchestra.run.smithi121.stdout: "dentry_count": 0, 2024-04-20T10:21:15.751 INFO:teuthology.orchestra.run.smithi121.stdout: "dentry_pinned_count": 0, 2024-04-20T10:21:15.752 INFO:teuthology.orchestra.run.smithi121.stdout: "id": 4425, 2024-04-20T10:21:15.752 INFO:teuthology.orchestra.run.smithi121.stdout: "inst": { 2024-04-20T10:21:15.752 INFO:teuthology.orchestra.run.smithi121.stdout: "name": { 2024-04-20T10:21:15.752 INFO:teuthology.orchestra.run.smithi121.stdout: "type": "client", 2024-04-20T10:21:15.752 INFO:teuthology.orchestra.run.smithi121.stdout: "num": 4425 2024-04-20T10:21:15.752 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T10:21:15.752 INFO:teuthology.orchestra.run.smithi121.stdout: "addr": { 2024-04-20T10:21:15.752 INFO:teuthology.orchestra.run.smithi121.stdout: "type": "v1", 2024-04-20T10:21:15.752 INFO:teuthology.orchestra.run.smithi121.stdout: "addr": "192.168.0.1:0", 2024-04-20T10:21:15.752 INFO:teuthology.orchestra.run.smithi121.stdout: "nonce": 577255525 2024-04-20T10:21:15.752 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T10:21:15.752 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T10:21:15.752 INFO:teuthology.orchestra.run.smithi121.stdout: "addr": { 2024-04-20T10:21:15.752 INFO:teuthology.orchestra.run.smithi121.stdout: "type": "v1", 2024-04-20T10:21:15.752 INFO:teuthology.orchestra.run.smithi121.stdout: "addr": "192.168.0.1:0", 2024-04-20T10:21:15.752 INFO:teuthology.orchestra.run.smithi121.stdout: "nonce": 577255525 2024-04-20T10:21:15.753 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T10:21:15.753 INFO:teuthology.orchestra.run.smithi121.stdout: "inst_str": "client.4425 192.168.0.1:0/577255525", 2024-04-20T10:21:15.753 INFO:teuthology.orchestra.run.smithi121.stdout: "addr_str": "192.168.0.1:0/577255525", 2024-04-20T10:21:15.753 INFO:teuthology.orchestra.run.smithi121.stdout: "inode_count": 1, 2024-04-20T10:21:15.753 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_epoch": 6, 2024-04-20T10:21:15.753 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_epoch": 25, 2024-04-20T10:21:15.753 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_epoch_barrier": 0, 2024-04-20T10:21:15.753 INFO:teuthology.orchestra.run.smithi121.stdout: "blocklisted": false, 2024-04-20T10:21:15.753 INFO:teuthology.orchestra.run.smithi121.stdout: "fs_name": "cephfs" 2024-04-20T10:21:15.753 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T10:21:15.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:15.765 DEBUG:teuthology.orchestra.run.smithi121:> stat --file-system '--printf=%T 2024-04-20T10:21:15.765 DEBUG:teuthology.orchestra.run.smithi121:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-20T10:21:15.819 INFO:teuthology.orchestra.run.smithi121.stdout:fuseblk 2024-04-20T10:21:15.819 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-20T10:21:15.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:15.819 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-20T10:21:15.905 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-20T10:21:16.018 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-20T10:21:16.057 INFO:teuthology.orchestra.run.smithi121.stdout:# file: . 2024-04-20T10:21:16.058 INFO:teuthology.orchestra.run.smithi121.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-20T10:21:16.058 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:21:16.058 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:21:16.068 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T10:21:16.068 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:21:16.120 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:21:16.369 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.368+0000 7f0f8ae00640 1 Processor -- start 2024-04-20T10:21:16.369 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.368+0000 7f0f8ae00640 1 -- start start 2024-04-20T10:21:16.370 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.369+0000 7f0f8ae00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0f8415e8a0 0x7f0f8406e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:16.370 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.369+0000 7f0f8ae00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0f840d0a70 0x7f0f840ad680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:16.370 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.369+0000 7f0f8ae00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f8406f060 con 0x7f0f8406dbb0 2024-04-20T10:21:16.370 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.369+0000 7f0f8ae00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f0f8406e7c0 con 0x7f0f840d0a70 2024-04-20T10:21:16.370 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.369+0000 7f0f8ae00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f0f8406eaa0 con 0x7f0f8415e8a0 2024-04-20T10:21:16.371 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.370+0000 7f0f88e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0f840d0a70 0x7f0f840ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:16.371 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.370+0000 7f0f88e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0f840d0a70 0x7f0f840ad680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.121:41506/0 (socket says 172.21.15.121:41506) 2024-04-20T10:21:16.371 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.370+0000 7f0f88e00640 1 -- 172.21.15.121:0/1963351106 learned_addr learned my addr 172.21.15.121:0/1963351106 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:16.371 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.370+0000 7f0f83e00640 1 --2- 172.21.15.121:0/1963351106 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0f8415e8a0 0x7f0f8406e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:16.372 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.371+0000 7f0f88e00640 1 -- 172.21.15.121:0/1963351106 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0f8415e8a0 msgr2=0x7f0f8406e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:16.372 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.371+0000 7f0f88e00640 1 --2- 172.21.15.121:0/1963351106 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0f8415e8a0 0x7f0f8406e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:16.372 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.371+0000 7f0f88e00640 1 -- 172.21.15.121:0/1963351106 >> v1:172.21.15.123:6789/0 conn(0x7f0f8406dbb0 legacy=0x7f0f840d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:16.372 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.371+0000 7f0f88e00640 1 -- 172.21.15.121:0/1963351106 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f8406ed80 con 0x7f0f840d0a70 2024-04-20T10:21:16.372 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.372+0000 7f0f88e00640 1 --2- 172.21.15.121:0/1963351106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f840d0a70 0x7f0f840ad680 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c009e50 tx=0x7f0f7c02bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d9b969ced0da978 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:16.373 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.373+0000 7f0f82a00640 1 -- 172.21.15.121:0/1963351106 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c02cb00 con 0x7f0f840d0a70 2024-04-20T10:21:16.373 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.373+0000 7f0f82a00640 1 -- 172.21.15.121:0/1963351106 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f7c02cca0 con 0x7f0f840d0a70 2024-04-20T10:21:16.373 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.373+0000 7f0f82a00640 1 -- 172.21.15.121:0/1963351106 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c034610 con 0x7f0f840d0a70 2024-04-20T10:21:16.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.373+0000 7f0f8ae00640 1 -- 172.21.15.121:0/1963351106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f840d0a70 msgr2=0x7f0f840ad680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:16.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.373+0000 7f0f8ae00640 1 --2- 172.21.15.121:0/1963351106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f840d0a70 0x7f0f840ad680 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c009e50 tx=0x7f0f7c02bf60 comp rx=0 tx=0).stop 2024-04-20T10:21:16.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.373+0000 7f0f8ae00640 1 -- 172.21.15.121:0/1963351106 shutdown_connections 2024-04-20T10:21:16.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.373+0000 7f0f8ae00640 1 --2- 172.21.15.121:0/1963351106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f840d0a70 0x7f0f840ad680 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:16.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.373+0000 7f0f8ae00640 1 --2- 172.21.15.121:0/1963351106 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0f8415e8a0 0x7f0f8406e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:16.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.373+0000 7f0f8ae00640 1 -- 172.21.15.121:0/1963351106 >> 172.21.15.121:0/1963351106 conn(0x7f0f840b3350 msgr2=0x7f0f840b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:16.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.373+0000 7f0f8ae00640 1 -- 172.21.15.121:0/1963351106 shutdown_connections 2024-04-20T10:21:16.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.374+0000 7f0f8ae00640 1 -- 172.21.15.121:0/1963351106 wait complete. 2024-04-20T10:21:16.375 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.375+0000 7f0f8ae00640 1 Processor -- start 2024-04-20T10:21:16.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.375+0000 7f0f8ae00640 1 -- start start 2024-04-20T10:21:16.377 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.376+0000 7f0f8ae00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f8406dbb0 0x7f0f8414d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:16.377 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.376+0000 7f0f8ae00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0f840d0a70 0x7f0f84160070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:16.377 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.376+0000 7f0f8ae00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f8415e8a0 0x7f0f841610c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:16.377 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.376+0000 7f0f8ae00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8406fef0 con 0x7f0f8415e8a0 2024-04-20T10:21:16.377 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.376+0000 7f0f8ae00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0f8406fd70 con 0x7f0f8406dbb0 2024-04-20T10:21:16.377 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.376+0000 7f0f8ae00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f0f840adbc0 con 0x7f0f840d0a70 2024-04-20T10:21:16.377 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.377+0000 7f0f88e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f8415e8a0 0x7f0f841610c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:16.377 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.377+0000 7f0f88e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f8415e8a0 0x7f0f841610c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37510/0 (socket says 172.21.15.121:37510) 2024-04-20T10:21:16.378 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.377+0000 7f0f88e00640 1 -- 172.21.15.121:0/3215798574 learned_addr learned my addr 172.21.15.121:0/3215798574 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:16.378 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.377+0000 7f0f83e00640 1 --2- 172.21.15.121:0/3215798574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f8406dbb0 0x7f0f8414d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:16.378 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.377+0000 7f0f83400640 1 --2- 172.21.15.121:0/3215798574 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0f840d0a70 0x7f0f84160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:16.379 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.377+0000 7f0f88e00640 1 -- 172.21.15.121:0/3215798574 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0f840d0a70 msgr2=0x7f0f84160070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:16.379 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.377+0000 7f0f88e00640 1 --2- 172.21.15.121:0/3215798574 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0f840d0a70 0x7f0f84160070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:16.379 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.377+0000 7f0f88e00640 1 -- 172.21.15.121:0/3215798574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f8406dbb0 msgr2=0x7f0f8414d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:16.379 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.377+0000 7f0f88e00640 1 --2- 172.21.15.121:0/3215798574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f8406dbb0 0x7f0f8414d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:16.379 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.377+0000 7f0f88e00640 1 -- 172.21.15.121:0/3215798574 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f8415efd0 con 0x7f0f8415e8a0 2024-04-20T10:21:16.379 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.378+0000 7f0f88e00640 1 --2- 172.21.15.121:0/3215798574 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f8415e8a0 0x7f0f841610c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c009e50 tx=0x7f0f7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:16.379 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.378+0000 7f0f7be00640 1 -- 172.21.15.121:0/3215798574 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c043020 con 0x7f0f8415e8a0 2024-04-20T10:21:16.379 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.378+0000 7f0f7be00640 1 -- 172.21.15.121:0/3215798574 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f7c02ce50 con 0x7f0f8415e8a0 2024-04-20T10:21:16.380 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.378+0000 7f0f8ae00640 1 -- 172.21.15.121:0/3215798574 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f8415f2b0 con 0x7f0f8415e8a0 2024-04-20T10:21:16.380 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.378+0000 7f0f7be00640 1 -- 172.21.15.121:0/3215798574 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c009510 con 0x7f0f8415e8a0 2024-04-20T10:21:16.380 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.378+0000 7f0f8ae00640 1 -- 172.21.15.121:0/3215798574 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f8415f870 con 0x7f0f8415e8a0 2024-04-20T10:21:16.380 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.379+0000 7f0f8ae00640 1 -- 172.21.15.121:0/3215798574 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f84068980 con 0x7f0f8415e8a0 2024-04-20T10:21:16.380 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.380+0000 7f0f7be00640 1 -- 172.21.15.121:0/3215798574 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0f7c046020 con 0x7f0f8415e8a0 2024-04-20T10:21:16.381 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.381+0000 7f0f7be00640 1 --2- 172.21.15.121:0/3215798574 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f0f6006c8f0 0x7f0f6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:16.381 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.381+0000 7f0f7be00640 1 -- 172.21.15.121:0/3215798574 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0f7c0ae3d0 con 0x7f0f8415e8a0 2024-04-20T10:21:16.381 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.381+0000 7f0f83e00640 1 --2- 172.21.15.121:0/3215798574 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f0f6006c8f0 0x7f0f6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:16.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.382+0000 7f0f83e00640 1 --2- 172.21.15.121:0/3215798574 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f0f6006c8f0 0x7f0f6006edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0f84158c40 tx=0x7f0f74008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:16.386 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.385+0000 7f0f7be00640 1 -- 172.21.15.121:0/3215798574 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0f7c04d030 con 0x7f0f8415e8a0 2024-04-20T10:21:16.585 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.585+0000 7f0f8ae00640 1 -- 172.21.15.121:0/3215798574 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f840d0430 con 0x7f0f8415e8a0 2024-04-20T10:21:16.587 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.587+0000 7f0f7be00640 1 -- 172.21.15.121:0/3215798574 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7f0f7c07c4b0 con 0x7f0f8415e8a0 2024-04-20T10:21:16.588 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:21:16.588 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":25,"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","created":"2024-04-20T10:20:38.076076+0000","modified":"2024-04-20T10:21:07.336542+0000","last_up_change":"2024-04-20T10:20:50.162700+0000","last_in_change":"2024-04-20T10:20:41.510382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:20:51.141365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:21:00.436937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:21:01.026957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:21:04.862693+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"629a7b8c-a40a-4a2a-a3cc-d9f357774310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6809","nonce":1044588835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6811","nonce":1044588835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6815","nonce":1044588835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6813","nonce":1044588835}]},"public_addr":"172.21.15.121:6809/1044588835","cluster_addr":"172.21.15.121:6811/1044588835","heartbeat_back_addr":"172.21.15.121:6815/1044588835","heartbeat_front_addr":"172.21.15.121:6813/1044588835","state":["exists","up"]},{"osd":1,"uuid":"f2570399-aa50-4ff3-bf5d-3d09207290db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6801","nonce":3505199624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6803","nonce":3505199624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6807","nonce":3505199624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6805","nonce":3505199624}]},"public_addr":"172.21.15.121:6801/3505199624","cluster_addr":"172.21.15.121:6803/3505199624","heartbeat_back_addr":"172.21.15.121:6807/3505199624","heartbeat_front_addr":"172.21.15.121:6805/3505199624","state":["exists","up"]},{"osd":2,"uuid":"50893807-dc30-49f4-b41f-739366626658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6817","nonce":3656724207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6819","nonce":3656724207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6827","nonce":3656724207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6822","nonce":3656724207}]},"public_addr":"172.21.15.121:6817/3656724207","cluster_addr":"172.21.15.121:6819/3656724207","heartbeat_back_addr":"172.21.15.121:6827/3656724207","heartbeat_front_addr":"172.21.15.121:6822/3656724207","state":["exists","up"]},{"osd":3,"uuid":"dadb01bc-5491-4ffc-8824-d48a057240b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6823","nonce":2232342832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6826","nonce":2232342832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6831","nonce":2232342832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6829","nonce":2232342832}]},"public_addr":"172.21.15.121:6823/2232342832","cluster_addr":"172.21.15.121:6826/2232342832","heartbeat_back_addr":"172.21.15.121:6831/2232342832","heartbeat_front_addr":"172.21.15.121:6829/2232342832","state":["exists","up"]},{"osd":4,"uuid":"b3f3b4a1-b375-4526-b69e-344ab24c2da1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6801","nonce":99353111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6803","nonce":99353111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6807","nonce":99353111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6805","nonce":99353111}]},"public_addr":"172.21.15.123:6801/99353111","cluster_addr":"172.21.15.123:6803/99353111","heartbeat_back_addr":"172.21.15.123:6807/99353111","heartbeat_front_addr":"172.21.15.123:6805/99353111","state":["exists","up"]},{"osd":5,"uuid":"2ee24a08-1624-4961-9642-d7b510282484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6818","nonce":2504374688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6822","nonce":2504374688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6830","nonce":2504374688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6826","nonce":2504374688}]},"public_addr":"172.21.15.123:6818/2504374688","cluster_addr":"172.21.15.123:6822/2504374688","heartbeat_back_addr":"172.21.15.123:6830/2504374688","heartbeat_front_addr":"172.21.15.123:6826/2504374688","state":["exists","up"]},{"osd":6,"uuid":"186118cc-6128-49de-acca-55260945c1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6819","nonce":4139299599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6823","nonce":4139299599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6829","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6831","nonce":4139299599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6827","nonce":4139299599}]},"public_addr":"172.21.15.123:6819/4139299599","cluster_addr":"172.21.15.123:6823/4139299599","heartbeat_back_addr":"172.21.15.123:6831/4139299599","heartbeat_front_addr":"172.21.15.123:6827/4139299599","state":["exists","up"]},{"osd":7,"uuid":"a00aac6e-4bd9-415a-a93d-ae85210865ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6809","nonce":4043981069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6811","nonce":4043981069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6815","nonce":4043981069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6813","nonce":4043981069}]},"public_addr":"172.21.15.123:6809/4043981069","cluster_addr":"172.21.15.123:6811/4043981069","heartbeat_back_addr":"172.21.15.123:6815/4043981069","heartbeat_front_addr":"172.21.15.123:6813/4043981069","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.904540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.958585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.616367+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.885244+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.103127+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.167248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.884348+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.862356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:21:16.591 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.590+0000 7f0f8ae00640 1 -- 172.21.15.121:0/3215798574 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f0f6006c8f0 msgr2=0x7f0f6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:16.591 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.590+0000 7f0f8ae00640 1 --2- 172.21.15.121:0/3215798574 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f0f6006c8f0 0x7f0f6006edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0f84158c40 tx=0x7f0f74008040 comp rx=0 tx=0).stop 2024-04-20T10:21:16.591 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.590+0000 7f0f8ae00640 1 -- 172.21.15.121:0/3215798574 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f8415e8a0 msgr2=0x7f0f841610c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:16.591 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.590+0000 7f0f8ae00640 1 --2- 172.21.15.121:0/3215798574 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f8415e8a0 0x7f0f841610c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c009e50 tx=0x7f0f7c004a40 comp rx=0 tx=0).stop 2024-04-20T10:21:16.592 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.590+0000 7f0f8ae00640 1 -- 172.21.15.121:0/3215798574 shutdown_connections 2024-04-20T10:21:16.592 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.590+0000 7f0f8ae00640 1 --2- 172.21.15.121:0/3215798574 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0f8415e8a0 0x7f0f841610c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:16.592 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.590+0000 7f0f8ae00640 1 --2- 172.21.15.121:0/3215798574 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0f840d0a70 0x7f0f84160070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:16.592 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.590+0000 7f0f8ae00640 1 --2- 172.21.15.121:0/3215798574 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f0f6006c8f0 0x7f0f6006edb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:16.592 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.590+0000 7f0f8ae00640 1 --2- 172.21.15.121:0/3215798574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f8406dbb0 0x7f0f8414d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:16.592 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.590+0000 7f0f8ae00640 1 -- 172.21.15.121:0/3215798574 >> 172.21.15.121:0/3215798574 conn(0x7f0f840b3350 msgr2=0x7f0f840b25e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:16.592 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.591+0000 7f0f8ae00640 1 -- 172.21.15.121:0/3215798574 shutdown_connections 2024-04-20T10:21:16.592 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.591+0000 7f0f8ae00640 1 -- 172.21.15.121:0/3215798574 wait complete. 2024-04-20T10:21:16.609 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:20:51.141365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:21:00.436937+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:21:01.026957+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T10:21:04.862693+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T10:21:16.609 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:21:16.813 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.812+0000 7f19b8c00640 1 Processor -- start 2024-04-20T10:21:16.813 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.813+0000 7f19b8c00640 1 -- start start 2024-04-20T10:21:16.815 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.814+0000 7f19b8c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f19b40d4270 0x7f19b40d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:16.815 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.814+0000 7f19b8c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f19b40d4650 0x7f19b40b1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:16.815 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.814+0000 7f19b8c00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19b40b17d0 0x7f19b40d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:16.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.814+0000 7f19b8c00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f19b40d40c0 con 0x7f19b40d4650 2024-04-20T10:21:16.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.814+0000 7f19b8c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f19b40d3820 con 0x7f19b40d4270 2024-04-20T10:21:16.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.814+0000 7f19b8c00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f19b40d3b00 con 0x7f19b40b17d0 2024-04-20T10:21:16.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.814+0000 7f19b1400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f19b40d4650 0x7f19b40b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:16.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.814+0000 7f19b1400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f19b40d4650 0x7f19b40b1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37522/0 (socket says 172.21.15.121:37522) 2024-04-20T10:21:16.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.814+0000 7f19b1400640 1 -- 172.21.15.121:0/288987472 learned_addr learned my addr 172.21.15.121:0/288987472 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:16.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.815+0000 7f19b1e00640 1 --2- 172.21.15.121:0/288987472 >> v2:172.21.15.123:3300/0 conn(0x7f19b40d4270 0x7f19b40d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:16.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.815+0000 7f19b2800640 1 --2- 172.21.15.121:0/288987472 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19b40b17d0 0x7f19b40d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:16.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.815+0000 7f19b1400640 1 -- 172.21.15.121:0/288987472 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19b40b17d0 msgr2=0x7f19b40d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:16.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.815+0000 7f19b1400640 1 --2- 172.21.15.121:0/288987472 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19b40b17d0 0x7f19b40d0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:16.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.815+0000 7f19b1400640 1 -- 172.21.15.121:0/288987472 >> v2:172.21.15.123:3300/0 conn(0x7f19b40d4270 msgr2=0x7f19b40d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:16.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.815+0000 7f19b1400640 1 --2- 172.21.15.121:0/288987472 >> v2:172.21.15.123:3300/0 conn(0x7f19b40d4270 0x7f19b40d1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:16.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.815+0000 7f19b1400640 1 -- 172.21.15.121:0/288987472 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19b40d3de0 con 0x7f19b40d4650 2024-04-20T10:21:16.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.815+0000 7f19b2800640 1 --2- 172.21.15.121:0/288987472 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19b40b17d0 0x7f19b40d0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:16.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.816+0000 7f19b1400640 1 --2- 172.21.15.121:0/288987472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19b40d4650 0x7f19b40b1290 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f19b408e820 tx=0x7f19a002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d99311e9a6b15b4f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:16.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.816+0000 7f19b0a00640 1 -- 172.21.15.121:0/288987472 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a0035020 con 0x7f19b40d4650 2024-04-20T10:21:16.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.816+0000 7f19b0a00640 1 -- 172.21.15.121:0/288987472 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19a002d040 con 0x7f19b40d4650 2024-04-20T10:21:16.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.816+0000 7f19b0a00640 1 -- 172.21.15.121:0/288987472 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a0031a80 con 0x7f19b40d4650 2024-04-20T10:21:16.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.817+0000 7f19b8c00640 1 -- 172.21.15.121:0/288987472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19b40d4650 msgr2=0x7f19b40b1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:16.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.817+0000 7f19b8c00640 1 --2- 172.21.15.121:0/288987472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19b40d4650 0x7f19b40b1290 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f19b408e820 tx=0x7f19a002b900 comp rx=0 tx=0).stop 2024-04-20T10:21:16.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.817+0000 7f19b8c00640 1 -- 172.21.15.121:0/288987472 shutdown_connections 2024-04-20T10:21:16.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.817+0000 7f19b8c00640 1 --2- 172.21.15.121:0/288987472 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19b40b17d0 0x7f19b40d0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:16.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.817+0000 7f19b8c00640 1 --2- 172.21.15.121:0/288987472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19b40d4650 0x7f19b40b1290 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:16.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.817+0000 7f19b8c00640 1 --2- 172.21.15.121:0/288987472 >> v2:172.21.15.123:3300/0 conn(0x7f19b40d4270 0x7f19b40d1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:16.819 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.817+0000 7f19b8c00640 1 -- 172.21.15.121:0/288987472 >> 172.21.15.121:0/288987472 conn(0x7f19b40b99c0 msgr2=0x7f19b40b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:16.819 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.817+0000 7f19b8c00640 1 -- 172.21.15.121:0/288987472 shutdown_connections 2024-04-20T10:21:16.819 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.817+0000 7f19b8c00640 1 -- 172.21.15.121:0/288987472 wait complete. 2024-04-20T10:21:16.819 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.819+0000 7f19b8c00640 1 Processor -- start 2024-04-20T10:21:16.819 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.819+0000 7f19b8c00640 1 -- start start 2024-04-20T10:21:16.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.820+0000 7f19b8c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19b40b17d0 0x7f19b40caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:16.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.820+0000 7f19b8c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19b40d4270 0x7f19b4155b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:16.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.820+0000 7f19b8c00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19b40d4650 0x7f19b406e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:16.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.820+0000 7f19b8c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f19b40d4ae0 con 0x7f19b40d4270 2024-04-20T10:21:16.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.820+0000 7f19b8c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f19b40d53b0 con 0x7f19b40b17d0 2024-04-20T10:21:16.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.820+0000 7f19b8c00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f19b40d4c60 con 0x7f19b40d4650 2024-04-20T10:21:16.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.820+0000 7f19b1400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19b40d4270 0x7f19b4155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:16.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.820+0000 7f19b1400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19b40d4270 0x7f19b4155b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37534/0 (socket says 172.21.15.121:37534) 2024-04-20T10:21:16.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.820+0000 7f19b1400640 1 -- 172.21.15.121:0/2438001299 learned_addr learned my addr 172.21.15.121:0/2438001299 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:16.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.821+0000 7f19b1e00640 1 --2- 172.21.15.121:0/2438001299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19b40b17d0 0x7f19b40caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:16.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.821+0000 7f19b2800640 1 --2- 172.21.15.121:0/2438001299 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19b40d4650 0x7f19b406e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:16.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.821+0000 7f19b1400640 1 -- 172.21.15.121:0/2438001299 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19b40d4650 msgr2=0x7f19b406e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:16.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.821+0000 7f19b1400640 1 --2- 172.21.15.121:0/2438001299 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19b40d4650 0x7f19b406e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:16.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.821+0000 7f19b1400640 1 -- 172.21.15.121:0/2438001299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19b40b17d0 msgr2=0x7f19b40caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:16.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.821+0000 7f19b1400640 1 --2- 172.21.15.121:0/2438001299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19b40b17d0 0x7f19b40caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:16.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.821+0000 7f19b1400640 1 -- 172.21.15.121:0/2438001299 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19b40cc4a0 con 0x7f19b40d4270 2024-04-20T10:21:16.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.821+0000 7f19b1400640 1 --2- 172.21.15.121:0/2438001299 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19b40d4270 0x7f19b4155b00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f19a002b940 tx=0x7f19a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:16.823 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.821+0000 7f19aa000640 1 -- 172.21.15.121:0/2438001299 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a0041070 con 0x7f19b40d4270 2024-04-20T10:21:16.823 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.821+0000 7f19aa000640 1 -- 172.21.15.121:0/2438001299 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19a003aba0 con 0x7f19b40d4270 2024-04-20T10:21:16.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.821+0000 7f19b8c00640 1 -- 172.21.15.121:0/2438001299 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19b40cc780 con 0x7f19b40d4270 2024-04-20T10:21:16.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.821+0000 7f19aa000640 1 -- 172.21.15.121:0/2438001299 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19a0035040 con 0x7f19b40d4270 2024-04-20T10:21:16.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.822+0000 7f19b8c00640 1 -- 172.21.15.121:0/2438001299 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19b40ccd40 con 0x7f19b40d4270 2024-04-20T10:21:16.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.823+0000 7f19b8c00640 1 -- 172.21.15.121:0/2438001299 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19b4068980 con 0x7f19b40d4270 2024-04-20T10:21:16.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.824+0000 7f19aa000640 1 -- 172.21.15.121:0/2438001299 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f19a004c060 con 0x7f19b40d4270 2024-04-20T10:21:16.825 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.825+0000 7f19aa000640 1 --2- 172.21.15.121:0/2438001299 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f199406c8f0 0x7f199406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:16.825 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.825+0000 7f19aa000640 1 -- 172.21.15.121:0/2438001299 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f19a00adbe0 con 0x7f19b40d4270 2024-04-20T10:21:16.825 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.825+0000 7f19b1e00640 1 --2- 172.21.15.121:0/2438001299 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f199406c8f0 0x7f199406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:16.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.826+0000 7f19b1e00640 1 --2- 172.21.15.121:0/2438001299 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f199406c8f0 0x7f199406edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f19a4006fd0 tx=0x7f19a4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:16.829 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:16.829+0000 7f19aa000640 1 -- 172.21.15.121:0/2438001299 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f19a007bba0 con 0x7f19b40d4270 2024-04-20T10:21:17.028 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.028+0000 7f19b8c00640 1 -- 172.21.15.121:0/2438001299 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f19b40b2f40 con 0x7f19b40d4270 2024-04-20T10:21:17.029 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.029+0000 7f19aa000640 1 -- 172.21.15.121:0/2438001299 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f19a007b4a0 con 0x7f19b40d4270 2024-04-20T10:21:17.029 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2024-04-20T10:21:17.032 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.031+0000 7f19b8c00640 1 -- 172.21.15.121:0/2438001299 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f199406c8f0 msgr2=0x7f199406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:17.032 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.031+0000 7f19b8c00640 1 --2- 172.21.15.121:0/2438001299 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f199406c8f0 0x7f199406edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f19a4006fd0 tx=0x7f19a4008040 comp rx=0 tx=0).stop 2024-04-20T10:21:17.032 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.031+0000 7f19b8c00640 1 -- 172.21.15.121:0/2438001299 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19b40d4270 msgr2=0x7f19b4155b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:17.032 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.031+0000 7f19b8c00640 1 --2- 172.21.15.121:0/2438001299 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19b40d4270 0x7f19b4155b00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f19a002b940 tx=0x7f19a00046c0 comp rx=0 tx=0).stop 2024-04-20T10:21:17.033 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.031+0000 7f19b8c00640 1 -- 172.21.15.121:0/2438001299 shutdown_connections 2024-04-20T10:21:17.033 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.031+0000 7f19b8c00640 1 --2- 172.21.15.121:0/2438001299 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f19b40d4650 0x7f19b406e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:17.033 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.031+0000 7f19b8c00640 1 --2- 172.21.15.121:0/2438001299 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f19b40d4270 0x7f19b4155b00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:17.033 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.031+0000 7f19b8c00640 1 --2- 172.21.15.121:0/2438001299 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f199406c8f0 0x7f199406edb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:17.033 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.031+0000 7f19b8c00640 1 --2- 172.21.15.121:0/2438001299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19b40b17d0 0x7f19b40caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:17.033 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.031+0000 7f19b8c00640 1 -- 172.21.15.121:0/2438001299 >> 172.21.15.121:0/2438001299 conn(0x7f19b40b99c0 msgr2=0x7f19b40d01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:17.033 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.032+0000 7f19b8c00640 1 -- 172.21.15.121:0/2438001299 shutdown_connections 2024-04-20T10:21:17.033 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.032+0000 7f19b8c00640 1 -- 172.21.15.121:0/2438001299 wait complete. 2024-04-20T10:21:17.050 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:21:17.253 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.252+0000 7f10f3200640 1 Processor -- start 2024-04-20T10:21:17.253 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.252+0000 7f10f3200640 1 -- start start 2024-04-20T10:21:17.254 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.254+0000 7f10f3200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f10ec15b1f0 0x7f10ec15f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:17.254 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.254+0000 7f10f3200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f10ec06ca10 0x7f10ec06ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:17.254 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.254+0000 7f10f3200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10ec15b5d0 con 0x7f10ec0d0010 2024-04-20T10:21:17.254 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.254+0000 7f10f3200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f10ec06d3a0 con 0x7f10ec06ca10 2024-04-20T10:21:17.254 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.254+0000 7f10f3200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f10ec06e730 con 0x7f10ec15b1f0 2024-04-20T10:21:17.255 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.254+0000 7f10f2800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f10ec06ca10 0x7f10ec06ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:17.255 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.254+0000 7f10f1400640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f10ec0d0010 0x7f10ec0adc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:51794/0 (socket says 172.21.15.121:51794) 2024-04-20T10:21:17.255 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.254+0000 7f10f1400640 1 -- 172.21.15.121:0/2346288458 learned_addr learned my addr 172.21.15.121:0/2346288458 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:17.255 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.254+0000 7f10f1e00640 1 --2- 172.21.15.121:0/2346288458 >> v2:172.21.15.123:3300/0 conn(0x7f10ec15b1f0 0x7f10ec15f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:17.255 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.254+0000 7f10f0a00640 1 -- 172.21.15.121:0/2346288458 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 57209456 0 0) 0x7f10ec15b5d0 con 0x7f10ec0d0010 2024-04-20T10:21:17.256 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.255+0000 7f10f2800640 1 -- 172.21.15.121:0/2346288458 >> v2:172.21.15.123:3300/0 conn(0x7f10ec15b1f0 msgr2=0x7f10ec15f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:17.256 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.255+0000 7f10f2800640 1 --2- 172.21.15.121:0/2346288458 >> v2:172.21.15.123:3300/0 conn(0x7f10ec15b1f0 0x7f10ec15f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:17.256 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.255+0000 7f10f2800640 1 -- 172.21.15.121:0/2346288458 >> v1:172.21.15.121:6789/0 conn(0x7f10ec0d0010 legacy=0x7f10ec0adc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:17.256 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.255+0000 7f10f2800640 1 -- 172.21.15.121:0/2346288458 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10ec15fc70 con 0x7f10ec06ca10 2024-04-20T10:21:17.256 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.256+0000 7f10f2800640 1 --2- 172.21.15.121:0/2346288458 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10ec06ca10 0x7f10ec06ce60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f10e8009d90 tx=0x7f10e802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abde89f08eecae81 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:17.257 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.256+0000 7f10f0a00640 1 -- 172.21.15.121:0/2346288458 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e8039070 con 0x7f10ec06ca10 2024-04-20T10:21:17.257 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.256+0000 7f10f0a00640 1 -- 172.21.15.121:0/2346288458 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10e802cbd0 con 0x7f10ec06ca10 2024-04-20T10:21:17.257 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.256+0000 7f10f0a00640 1 -- 172.21.15.121:0/2346288458 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e8034440 con 0x7f10ec06ca10 2024-04-20T10:21:17.257 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.257+0000 7f10f3200640 1 -- 172.21.15.121:0/2346288458 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10ec06ca10 msgr2=0x7f10ec06ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:17.257 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.257+0000 7f10f3200640 1 --2- 172.21.15.121:0/2346288458 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10ec06ca10 0x7f10ec06ce60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f10e8009d90 tx=0x7f10e802be20 comp rx=0 tx=0).stop 2024-04-20T10:21:17.258 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.257+0000 7f10f3200640 1 -- 172.21.15.121:0/2346288458 shutdown_connections 2024-04-20T10:21:17.258 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.257+0000 7f10f3200640 1 --2- 172.21.15.121:0/2346288458 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10ec06ca10 0x7f10ec06ce60 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:17.258 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.257+0000 7f10f3200640 1 --2- 172.21.15.121:0/2346288458 >> v2:172.21.15.123:3300/0 conn(0x7f10ec15b1f0 0x7f10ec15f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:17.258 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.257+0000 7f10f3200640 1 -- 172.21.15.121:0/2346288458 >> 172.21.15.121:0/2346288458 conn(0x7f10ec0b5270 msgr2=0x7f10ec0b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:17.258 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.257+0000 7f10f3200640 1 -- 172.21.15.121:0/2346288458 shutdown_connections 2024-04-20T10:21:17.258 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.257+0000 7f10f3200640 1 -- 172.21.15.121:0/2346288458 wait complete. 2024-04-20T10:21:17.259 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.259+0000 7f10f3200640 1 Processor -- start 2024-04-20T10:21:17.259 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.259+0000 7f10f3200640 1 -- start start 2024-04-20T10:21:17.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.260+0000 7f10f3200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10ec06ca10 0x7f10ec1f1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:17.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.260+0000 7f10f3200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10ec0d0010 0x7f10ec1f4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:17.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.260+0000 7f10f3200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10ec15b1f0 0x7f10ec153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:17.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.260+0000 7f10f3200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f10ec0d08b0 con 0x7f10ec06ca10 2024-04-20T10:21:17.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.260+0000 7f10f3200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f10ec0d05d0 con 0x7f10ec0d0010 2024-04-20T10:21:17.262 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.260+0000 7f10f3200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f10ec0d0b90 con 0x7f10ec15b1f0 2024-04-20T10:21:17.262 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.260+0000 7f10f1e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10ec06ca10 0x7f10ec1f1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:17.262 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.260+0000 7f10f1e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10ec06ca10 0x7f10ec1f1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37554/0 (socket says 172.21.15.121:37554) 2024-04-20T10:21:17.262 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.260+0000 7f10f1e00640 1 -- 172.21.15.121:0/4252748701 learned_addr learned my addr 172.21.15.121:0/4252748701 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:17.262 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.260+0000 7f10f1400640 1 --2- 172.21.15.121:0/4252748701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10ec0d0010 0x7f10ec1f4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:17.262 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.261+0000 7f10f1e00640 1 -- 172.21.15.121:0/4252748701 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10ec15b1f0 msgr2=0x7f10ec153f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:17.262 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.261+0000 7f10f1e00640 1 --2- 172.21.15.121:0/4252748701 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10ec15b1f0 0x7f10ec153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:17.262 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.261+0000 7f10f1e00640 1 -- 172.21.15.121:0/4252748701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10ec0d0010 msgr2=0x7f10ec1f4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:17.262 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.261+0000 7f10f1e00640 1 --2- 172.21.15.121:0/4252748701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10ec0d0010 0x7f10ec1f4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:17.262 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.261+0000 7f10f1e00640 1 -- 172.21.15.121:0/4252748701 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10ec1f23e0 con 0x7f10ec06ca10 2024-04-20T10:21:17.262 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.261+0000 7f10f1e00640 1 --2- 172.21.15.121:0/4252748701 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10ec06ca10 0x7f10ec1f1d90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f10e000d800 tx=0x7f10e000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:17.263 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.261+0000 7f10e6000640 1 -- 172.21.15.121:0/4252748701 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e000b660 con 0x7f10ec06ca10 2024-04-20T10:21:17.263 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.261+0000 7f10e6000640 1 -- 172.21.15.121:0/4252748701 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10e0019070 con 0x7f10ec06ca10 2024-04-20T10:21:17.263 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.261+0000 7f10f3200640 1 -- 172.21.15.121:0/4252748701 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10ec1f26c0 con 0x7f10ec06ca10 2024-04-20T10:21:17.263 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.261+0000 7f10e6000640 1 -- 172.21.15.121:0/4252748701 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e000ba20 con 0x7f10ec06ca10 2024-04-20T10:21:17.264 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.261+0000 7f10f3200640 1 -- 172.21.15.121:0/4252748701 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10ec1f2c80 con 0x7f10ec06ca10 2024-04-20T10:21:17.264 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.262+0000 7f10f3200640 1 -- 172.21.15.121:0/4252748701 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10ec075b30 con 0x7f10ec06ca10 2024-04-20T10:21:17.264 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.263+0000 7f10e6000640 1 -- 172.21.15.121:0/4252748701 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10e000bbc0 con 0x7f10ec06ca10 2024-04-20T10:21:17.265 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.265+0000 7f10e6000640 1 --2- 172.21.15.121:0/4252748701 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f10d006c8f0 0x7f10d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:17.265 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.265+0000 7f10e6000640 1 -- 172.21.15.121:0/4252748701 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f10e008c3d0 con 0x7f10ec06ca10 2024-04-20T10:21:17.265 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.265+0000 7f10f1400640 1 --2- 172.21.15.121:0/4252748701 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f10d006c8f0 0x7f10d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:17.266 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.266+0000 7f10f1400640 1 --2- 172.21.15.121:0/4252748701 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f10d006c8f0 0x7f10d006edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f10ec09a320 tx=0x7f10dc007c00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:17.270 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.269+0000 7f10e6000640 1 -- 172.21.15.121:0/4252748701 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f10e005a420 con 0x7f10ec06ca10 2024-04-20T10:21:17.471 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.471+0000 7f10f3200640 1 -- 172.21.15.121:0/4252748701 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f10ec072860 con 0x7f10ec06ca10 2024-04-20T10:21:17.472 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.472+0000 7f10e6000640 1 -- 172.21.15.121:0/4252748701 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f10e0059d20 con 0x7f10ec06ca10 2024-04-20T10:21:17.473 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 32 2024-04-20T10:21:17.475 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.474+0000 7f10f3200640 1 -- 172.21.15.121:0/4252748701 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f10d006c8f0 msgr2=0x7f10d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:17.475 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.474+0000 7f10f3200640 1 --2- 172.21.15.121:0/4252748701 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f10d006c8f0 0x7f10d006edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f10ec09a320 tx=0x7f10dc007c00 comp rx=0 tx=0).stop 2024-04-20T10:21:17.475 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.474+0000 7f10f3200640 1 -- 172.21.15.121:0/4252748701 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10ec06ca10 msgr2=0x7f10ec1f1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:17.475 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.474+0000 7f10f3200640 1 --2- 172.21.15.121:0/4252748701 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10ec06ca10 0x7f10ec1f1d90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f10e000d800 tx=0x7f10e000dcc0 comp rx=0 tx=0).stop 2024-04-20T10:21:17.476 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.475+0000 7f10f3200640 1 -- 172.21.15.121:0/4252748701 shutdown_connections 2024-04-20T10:21:17.476 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.475+0000 7f10f3200640 1 --2- 172.21.15.121:0/4252748701 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10ec15b1f0 0x7f10ec153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:17.476 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.475+0000 7f10f3200640 1 --2- 172.21.15.121:0/4252748701 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f10d006c8f0 0x7f10d006edb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:17.476 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.475+0000 7f10f3200640 1 --2- 172.21.15.121:0/4252748701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10ec0d0010 0x7f10ec1f4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:17.476 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.475+0000 7f10f3200640 1 --2- 172.21.15.121:0/4252748701 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10ec06ca10 0x7f10ec1f1d90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:17.476 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.475+0000 7f10f3200640 1 -- 172.21.15.121:0/4252748701 >> 172.21.15.121:0/4252748701 conn(0x7f10ec0b5270 msgr2=0x7f10ec0ae520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:17.476 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.475+0000 7f10f3200640 1 -- 172.21.15.121:0/4252748701 shutdown_connections 2024-04-20T10:21:17.476 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.475+0000 7f10f3200640 1 -- 172.21.15.121:0/4252748701 wait complete. 2024-04-20T10:21:17.494 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:21:17.702 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.701+0000 7f9ff9e00640 1 Processor -- start 2024-04-20T10:21:17.702 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.701+0000 7f9ff9e00640 1 -- start start 2024-04-20T10:21:17.703 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.702+0000 7f9ff9e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9ff415b3b0 0x7f9ff406eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:17.703 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.703+0000 7f9ff9e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9ff406dcb0 0x7f9ff4071b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:17.704 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.703+0000 7f9ff9e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ff40ade40 con 0x7f9ff4072090 2024-04-20T10:21:17.704 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.703+0000 7f9ff9e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f9ff40ae0e0 con 0x7f9ff406dcb0 2024-04-20T10:21:17.704 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.703+0000 7f9ff9e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f9ff415b790 con 0x7f9ff415b3b0 2024-04-20T10:21:17.704 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.703+0000 7f9fea000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9ff406dcb0 0x7f9ff4071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:17.704 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.703+0000 7f9fea000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9ff406dcb0 0x7f9ff4071b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37560/0 (socket says 172.21.15.121:37560) 2024-04-20T10:21:17.704 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.703+0000 7f9fea000640 1 -- 172.21.15.121:0/16313750 learned_addr learned my addr 172.21.15.121:0/16313750 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:17.705 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.703+0000 7f9ff2a00640 1 --2- 172.21.15.121:0/16313750 >> v2:172.21.15.123:3300/0 conn(0x7f9ff415b3b0 0x7f9ff406eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:17.705 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.704+0000 7f9fea000640 1 -- 172.21.15.121:0/16313750 >> v2:172.21.15.123:3300/0 conn(0x7f9ff415b3b0 msgr2=0x7f9ff406eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:17.705 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.704+0000 7f9fea000640 1 --2- 172.21.15.121:0/16313750 >> v2:172.21.15.123:3300/0 conn(0x7f9ff415b3b0 0x7f9ff406eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:17.705 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.704+0000 7f9fea000640 1 -- 172.21.15.121:0/16313750 >> v1:172.21.15.123:6789/0 conn(0x7f9ff4072090 legacy=0x7f9ff40d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:17.705 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.704+0000 7f9fea000640 1 -- 172.21.15.121:0/16313750 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff40ae380 con 0x7f9ff406dcb0 2024-04-20T10:21:17.705 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.705+0000 7f9fea000640 1 --2- 172.21.15.121:0/16313750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ff406dcb0 0x7f9ff4071b50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9ff406cfa0 tx=0x7f9fdc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a129a38ee4bde531 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:17.706 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.705+0000 7f9ff2000640 1 -- 172.21.15.121:0/16313750 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc035020 con 0x7f9ff406dcb0 2024-04-20T10:21:17.707 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.705+0000 7f9ff2000640 1 -- 172.21.15.121:0/16313750 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fdc031870 con 0x7f9ff406dcb0 2024-04-20T10:21:17.707 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.705+0000 7f9ff2000640 1 -- 172.21.15.121:0/16313750 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc031c40 con 0x7f9ff406dcb0 2024-04-20T10:21:17.707 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.706+0000 7f9ff9e00640 1 -- 172.21.15.121:0/16313750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ff406dcb0 msgr2=0x7f9ff4071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:17.707 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.706+0000 7f9ff9e00640 1 --2- 172.21.15.121:0/16313750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ff406dcb0 0x7f9ff4071b50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9ff406cfa0 tx=0x7f9fdc02b910 comp rx=0 tx=0).stop 2024-04-20T10:21:17.707 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.706+0000 7f9ff9e00640 1 -- 172.21.15.121:0/16313750 shutdown_connections 2024-04-20T10:21:17.707 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.706+0000 7f9ff9e00640 1 --2- 172.21.15.121:0/16313750 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ff406dcb0 0x7f9ff4071b50 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:17.707 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.706+0000 7f9ff9e00640 1 --2- 172.21.15.121:0/16313750 >> v2:172.21.15.123:3300/0 conn(0x7f9ff415b3b0 0x7f9ff406eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:17.707 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.706+0000 7f9ff9e00640 1 -- 172.21.15.121:0/16313750 >> 172.21.15.121:0/16313750 conn(0x7f9ff40b3350 msgr2=0x7f9ff40b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:17.707 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.706+0000 7f9ff9e00640 1 -- 172.21.15.121:0/16313750 shutdown_connections 2024-04-20T10:21:17.707 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.706+0000 7f9ff9e00640 1 -- 172.21.15.121:0/16313750 wait complete. 2024-04-20T10:21:17.708 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.708+0000 7f9ff9e00640 1 Processor -- start 2024-04-20T10:21:17.708 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.708+0000 7f9ff9e00640 1 -- start start 2024-04-20T10:21:17.711 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.709+0000 7f9ff9e00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ff406dcb0 0x7f9ff414e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:17.711 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.709+0000 7f9ff9e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ff415b3b0 0x7f9ff415d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:17.711 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.709+0000 7f9ff9e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ff415dc20 0x7f9ff4150370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:17.711 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.709+0000 7f9ff9e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff40aec20 con 0x7f9ff415dc20 2024-04-20T10:21:17.711 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.709+0000 7f9ff9e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff40ae940 con 0x7f9ff415b3b0 2024-04-20T10:21:17.711 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.709+0000 7f9ff9e00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f9ff406efe0 con 0x7f9ff406dcb0 2024-04-20T10:21:17.711 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.709+0000 7f9ff3400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ff415dc20 0x7f9ff4150370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:17.711 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.709+0000 7f9ff3400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ff415dc20 0x7f9ff4150370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37570/0 (socket says 172.21.15.121:37570) 2024-04-20T10:21:17.711 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.709+0000 7f9ff3400640 1 -- 172.21.15.121:0/2559071232 learned_addr learned my addr 172.21.15.121:0/2559071232 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:17.712 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.709+0000 7f9fea000640 1 --2- 172.21.15.121:0/2559071232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ff415b3b0 0x7f9ff415d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:17.712 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.709+0000 7f9ff2a00640 1 --2- 172.21.15.121:0/2559071232 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ff406dcb0 0x7f9ff414e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:17.712 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.710+0000 7f9ff3400640 1 -- 172.21.15.121:0/2559071232 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ff406dcb0 msgr2=0x7f9ff414e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:17.712 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.710+0000 7f9ff3400640 1 --2- 172.21.15.121:0/2559071232 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ff406dcb0 0x7f9ff414e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:17.712 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.710+0000 7f9ff3400640 1 -- 172.21.15.121:0/2559071232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ff415b3b0 msgr2=0x7f9ff415d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:17.712 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.710+0000 7f9ff3400640 1 --2- 172.21.15.121:0/2559071232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ff415b3b0 0x7f9ff415d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:17.712 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.710+0000 7f9ff3400640 1 -- 172.21.15.121:0/2559071232 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff415e680 con 0x7f9ff415dc20 2024-04-20T10:21:17.712 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.710+0000 7f9ff3400640 1 --2- 172.21.15.121:0/2559071232 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ff415dc20 0x7f9ff4150370 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00c410 tx=0x7f9fec00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:17.712 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.710+0000 7f9febe00640 1 -- 172.21.15.121:0/2559071232 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec017070 con 0x7f9ff415dc20 2024-04-20T10:21:17.712 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.710+0000 7f9febe00640 1 -- 172.21.15.121:0/2559071232 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fec005230 con 0x7f9ff415dc20 2024-04-20T10:21:17.712 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.710+0000 7f9ff9e00640 1 -- 172.21.15.121:0/2559071232 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ff415e0c0 con 0x7f9ff415dc20 2024-04-20T10:21:17.713 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.710+0000 7f9febe00640 1 -- 172.21.15.121:0/2559071232 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec011590 con 0x7f9ff415dc20 2024-04-20T10:21:17.713 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.710+0000 7f9ff9e00640 1 -- 172.21.15.121:0/2559071232 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ff40af150 con 0x7f9ff415dc20 2024-04-20T10:21:17.714 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.711+0000 7f9ff9e00640 1 -- 172.21.15.121:0/2559071232 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ff4068980 con 0x7f9ff415dc20 2024-04-20T10:21:17.714 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.712+0000 7f9febe00640 1 -- 172.21.15.121:0/2559071232 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9fec01a070 con 0x7f9ff415dc20 2024-04-20T10:21:17.714 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.713+0000 7f9febe00640 1 --2- 172.21.15.121:0/2559071232 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9fd006c8f0 0x7f9fd006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:17.714 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.713+0000 7f9febe00640 1 -- 172.21.15.121:0/2559071232 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9fec08b1c0 con 0x7f9ff415dc20 2024-04-20T10:21:17.714 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.714+0000 7f9ff2a00640 1 --2- 172.21.15.121:0/2559071232 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9fd006c8f0 0x7f9fd006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:17.715 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.714+0000 7f9ff2a00640 1 --2- 172.21.15.121:0/2559071232 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9fd006c8f0 0x7f9fd006edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9fe40099c0 tx=0x7f9fe4003990 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:17.718 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.718+0000 7f9febe00640 1 -- 172.21.15.121:0/2559071232 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9fec0592b0 con 0x7f9ff415dc20 2024-04-20T10:21:17.917 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.917+0000 7f9ff9e00640 1 -- 172.21.15.121:0/2559071232 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9ff4073540 con 0x7f9ff415dc20 2024-04-20T10:21:17.918 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.917+0000 7f9febe00640 1 -- 172.21.15.121:0/2559071232 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f9fec058bb0 con 0x7f9ff415dc20 2024-04-20T10:21:17.918 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2024-04-20T10:21:17.920 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.920+0000 7f9ff9e00640 1 -- 172.21.15.121:0/2559071232 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9fd006c8f0 msgr2=0x7f9fd006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:17.920 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.920+0000 7f9ff9e00640 1 --2- 172.21.15.121:0/2559071232 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9fd006c8f0 0x7f9fd006edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9fe40099c0 tx=0x7f9fe4003990 comp rx=0 tx=0).stop 2024-04-20T10:21:17.920 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.920+0000 7f9ff9e00640 1 -- 172.21.15.121:0/2559071232 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ff415dc20 msgr2=0x7f9ff4150370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:17.920 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.920+0000 7f9ff9e00640 1 --2- 172.21.15.121:0/2559071232 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ff415dc20 0x7f9ff4150370 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00c410 tx=0x7f9fec00c8d0 comp rx=0 tx=0).stop 2024-04-20T10:21:17.921 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.920+0000 7f9ff9e00640 1 -- 172.21.15.121:0/2559071232 shutdown_connections 2024-04-20T10:21:17.921 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.920+0000 7f9ff9e00640 1 --2- 172.21.15.121:0/2559071232 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ff415dc20 0x7f9ff4150370 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:17.921 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.920+0000 7f9ff9e00640 1 --2- 172.21.15.121:0/2559071232 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9fd006c8f0 0x7f9fd006edb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:17.921 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.920+0000 7f9ff9e00640 1 --2- 172.21.15.121:0/2559071232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ff415b3b0 0x7f9ff415d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:17.921 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.920+0000 7f9ff9e00640 1 --2- 172.21.15.121:0/2559071232 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ff406dcb0 0x7f9ff414e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:17.921 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.920+0000 7f9ff9e00640 1 -- 172.21.15.121:0/2559071232 >> 172.21.15.121:0/2559071232 conn(0x7f9ff40b3350 msgr2=0x7f9ff4072d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:17.922 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.920+0000 7f9ff9e00640 1 -- 172.21.15.121:0/2559071232 shutdown_connections 2024-04-20T10:21:17.922 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:17.920+0000 7f9ff9e00640 1 -- 172.21.15.121:0/2559071232 wait complete. 2024-04-20T10:21:17.940 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T10:21:18.149 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.148+0000 7f0826200640 1 Processor -- start 2024-04-20T10:21:18.149 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.149+0000 7f0826200640 1 -- start start 2024-04-20T10:21:18.151 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.150+0000 7f0826200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f082015e9c0 0x7f08200d1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:18.151 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.150+0000 7f0826200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f08200b11c0 0x7f08200d0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:18.151 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.150+0000 7f0826200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08200d0c80 con 0x7f08200d4930 2024-04-20T10:21:18.151 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.150+0000 7f0826200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f08200d0f20 con 0x7f08200b11c0 2024-04-20T10:21:18.151 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.150+0000 7f0826200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f08200d2ce0 con 0x7f082015e9c0 2024-04-20T10:21:18.151 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.151+0000 7f081f400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f082015e9c0 0x7f08200d1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:18.151 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.151+0000 7f081ea00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f08200b11c0 0x7f08200d0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:18.151 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.151+0000 7f081f400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f082015e9c0 0x7f08200d1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.121:46294/0 (socket says 172.21.15.121:46294) 2024-04-20T10:21:18.152 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.151+0000 7f081f400640 1 -- 172.21.15.121:0/312820043 learned_addr learned my addr 172.21.15.121:0/312820043 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:18.152 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.151+0000 7f081ea00640 1 -- 172.21.15.121:0/312820043 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f082015e9c0 msgr2=0x7f08200d1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:18.152 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.151+0000 7f081ea00640 1 --2- 172.21.15.121:0/312820043 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f082015e9c0 0x7f08200d1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:18.152 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.151+0000 7f081ea00640 1 -- 172.21.15.121:0/312820043 >> v1:172.21.15.123:6789/0 conn(0x7f08200d4930 legacy=0x7f08200b60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:18.152 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.151+0000 7f081ea00640 1 -- 172.21.15.121:0/312820043 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08200d23b0 con 0x7f08200b11c0 2024-04-20T10:21:18.153 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.153+0000 7f081ea00640 1 --2- 172.21.15.121:0/312820043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08200b11c0 0x7f08200d0740 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0810009d90 tx=0x7f081002be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbeb0ec29a82db51 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:18.155 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.154+0000 7f081e000640 1 -- 172.21.15.121:0/312820043 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0810039070 con 0x7f08200b11c0 2024-04-20T10:21:18.155 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.154+0000 7f081e000640 1 -- 172.21.15.121:0/312820043 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f081002cbc0 con 0x7f08200b11c0 2024-04-20T10:21:18.155 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.154+0000 7f081e000640 1 -- 172.21.15.121:0/312820043 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0810034440 con 0x7f08200b11c0 2024-04-20T10:21:18.155 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.154+0000 7f0826200640 1 -- 172.21.15.121:0/312820043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08200b11c0 msgr2=0x7f08200d0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:18.155 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.154+0000 7f0826200640 1 --2- 172.21.15.121:0/312820043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08200b11c0 0x7f08200d0740 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0810009d90 tx=0x7f081002be10 comp rx=0 tx=0).stop 2024-04-20T10:21:18.155 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.155+0000 7f0826200640 1 -- 172.21.15.121:0/312820043 shutdown_connections 2024-04-20T10:21:18.155 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.155+0000 7f0826200640 1 --2- 172.21.15.121:0/312820043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08200b11c0 0x7f08200d0740 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:18.155 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.155+0000 7f0826200640 1 --2- 172.21.15.121:0/312820043 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f082015e9c0 0x7f08200d1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:18.155 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.155+0000 7f0826200640 1 -- 172.21.15.121:0/312820043 >> 172.21.15.121:0/312820043 conn(0x7f08200b9600 msgr2=0x7f08200b9a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:18.156 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.155+0000 7f0826200640 1 -- 172.21.15.121:0/312820043 shutdown_connections 2024-04-20T10:21:18.156 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.155+0000 7f0826200640 1 -- 172.21.15.121:0/312820043 wait complete. 2024-04-20T10:21:18.157 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.156+0000 7f0826200640 1 Processor -- start 2024-04-20T10:21:18.157 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.157+0000 7f0826200640 1 -- start start 2024-04-20T10:21:18.159 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.157+0000 7f0826200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08200b11c0 0x7f0820161850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:18.159 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.157+0000 7f0826200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f08200d4930 0x7f08200b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:18.159 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.157+0000 7f0826200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f082015e9c0 0x7f082015fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:18.159 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.158+0000 7f0826200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f08200d4de0 con 0x7f082015e9c0 2024-04-20T10:21:18.159 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.158+0000 7f0826200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f08200d2970 con 0x7f08200b11c0 2024-04-20T10:21:18.159 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.158+0000 7f0826200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f08200d4f60 con 0x7f08200d4930 2024-04-20T10:21:18.159 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.158+0000 7f081fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f082015e9c0 0x7f082015fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:18.159 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.158+0000 7f081fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f082015e9c0 0x7f082015fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37572/0 (socket says 172.21.15.121:37572) 2024-04-20T10:21:18.159 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.158+0000 7f081fe00640 1 -- 172.21.15.121:0/1244748635 learned_addr learned my addr 172.21.15.121:0/1244748635 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:18.159 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.158+0000 7f081f400640 1 --2- 172.21.15.121:0/1244748635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08200b11c0 0x7f0820161850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:18.160 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.158+0000 7f081ea00640 1 --2- 172.21.15.121:0/1244748635 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f08200d4930 0x7f08200b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:18.160 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.158+0000 7f081fe00640 1 -- 172.21.15.121:0/1244748635 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f08200d4930 msgr2=0x7f08200b5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:18.160 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.158+0000 7f081fe00640 1 --2- 172.21.15.121:0/1244748635 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f08200d4930 0x7f08200b5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:18.160 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.158+0000 7f081fe00640 1 -- 172.21.15.121:0/1244748635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08200b11c0 msgr2=0x7f0820161850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:18.160 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.158+0000 7f081fe00640 1 --2- 172.21.15.121:0/1244748635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08200b11c0 0x7f0820161850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:18.160 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.158+0000 7f081fe00640 1 -- 172.21.15.121:0/1244748635 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f082015eff0 con 0x7f082015e9c0 2024-04-20T10:21:18.160 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.159+0000 7f081fe00640 1 --2- 172.21.15.121:0/1244748635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f082015e9c0 0x7f082015fcb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f081800c400 tx=0x7f081800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:18.161 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.159+0000 7f0817400640 1 -- 172.21.15.121:0/1244748635 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0818017070 con 0x7f082015e9c0 2024-04-20T10:21:18.161 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.159+0000 7f0817400640 1 -- 172.21.15.121:0/1244748635 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08180052d0 con 0x7f082015e9c0 2024-04-20T10:21:18.161 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.159+0000 7f0817400640 1 -- 172.21.15.121:0/1244748635 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0818003810 con 0x7f082015e9c0 2024-04-20T10:21:18.161 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.159+0000 7f0826200640 1 -- 172.21.15.121:0/1244748635 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08200b5370 con 0x7f082015e9c0 2024-04-20T10:21:18.161 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.159+0000 7f0826200640 1 -- 172.21.15.121:0/1244748635 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08200b6390 con 0x7f082015e9c0 2024-04-20T10:21:18.161 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.161+0000 7f0826200640 1 -- 172.21.15.121:0/1244748635 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0820068980 con 0x7f082015e9c0 2024-04-20T10:21:18.162 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.162+0000 7f0817400640 1 -- 172.21.15.121:0/1244748635 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f081801a070 con 0x7f082015e9c0 2024-04-20T10:21:18.163 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.162+0000 7f0817400640 1 --2- 172.21.15.121:0/1244748635 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f07fc06c8f0 0x7f07fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:18.163 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.163+0000 7f0817400640 1 -- 172.21.15.121:0/1244748635 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f081808b1d0 con 0x7f082015e9c0 2024-04-20T10:21:18.163 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.163+0000 7f081f400640 1 --2- 172.21.15.121:0/1244748635 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f07fc06c8f0 0x7f07fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:18.164 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.164+0000 7f081f400640 1 --2- 172.21.15.121:0/1244748635 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f07fc06c8f0 0x7f07fc06edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0808009ac0 tx=0x7f0808008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:18.168 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.168+0000 7f0817400640 1 -- 172.21.15.121:0/1244748635 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0818059350 con 0x7f082015e9c0 2024-04-20T10:21:18.366 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.366+0000 7f0826200640 1 -- 172.21.15.121:0/1244748635 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f08200ce740 con 0x7f082015e9c0 2024-04-20T10:21:18.367 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.367+0000 7f0817400640 1 -- 172.21.15.121:0/1244748635 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f0818058c50 con 0x7f082015e9c0 2024-04-20T10:21:18.367 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 32 2024-04-20T10:21:18.371 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.370+0000 7f0826200640 1 -- 172.21.15.121:0/1244748635 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f07fc06c8f0 msgr2=0x7f07fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:18.371 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.370+0000 7f0826200640 1 --2- 172.21.15.121:0/1244748635 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f07fc06c8f0 0x7f07fc06edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0808009ac0 tx=0x7f0808008040 comp rx=0 tx=0).stop 2024-04-20T10:21:18.371 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.370+0000 7f0826200640 1 -- 172.21.15.121:0/1244748635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f082015e9c0 msgr2=0x7f082015fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:18.371 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.370+0000 7f0826200640 1 --2- 172.21.15.121:0/1244748635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f082015e9c0 0x7f082015fcb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f081800c400 tx=0x7f081800c8c0 comp rx=0 tx=0).stop 2024-04-20T10:21:18.371 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.370+0000 7f0826200640 1 -- 172.21.15.121:0/1244748635 shutdown_connections 2024-04-20T10:21:18.371 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.370+0000 7f0826200640 1 --2- 172.21.15.121:0/1244748635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f082015e9c0 0x7f082015fcb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:18.371 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.370+0000 7f0826200640 1 --2- 172.21.15.121:0/1244748635 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f08200d4930 0x7f08200b5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:18.371 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.370+0000 7f0826200640 1 --2- 172.21.15.121:0/1244748635 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f07fc06c8f0 0x7f07fc06edb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:18.371 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.370+0000 7f0826200640 1 --2- 172.21.15.121:0/1244748635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08200b11c0 0x7f0820161850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:18.371 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.370+0000 7f0826200640 1 -- 172.21.15.121:0/1244748635 >> 172.21.15.121:0/1244748635 conn(0x7f08200b9600 msgr2=0x7f08200b15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:18.371 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.370+0000 7f0826200640 1 -- 172.21.15.121:0/1244748635 shutdown_connections 2024-04-20T10:21:18.372 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.370+0000 7f0826200640 1 -- 172.21.15.121:0/1244748635 wait complete. 2024-04-20T10:21:18.391 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T10:21:18.582 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.581+0000 7f0959800640 1 Processor -- start 2024-04-20T10:21:18.582 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.582+0000 7f0959800640 1 -- start start 2024-04-20T10:21:18.586 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.583+0000 7f0959800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0954162c00 0x7f09540d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:18.586 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.583+0000 7f0959800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f095406fc30 0x7f09540d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:18.586 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.583+0000 7f0959800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f095406df70 con 0x7f095406f090 2024-04-20T10:21:18.587 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.583+0000 7f0959800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f0954162fe0 con 0x7f0954162c00 2024-04-20T10:21:18.587 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.583+0000 7f0959800640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f095406ec00 con 0x7f095406fc30 2024-04-20T10:21:18.587 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.583+0000 7f094be00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f095406f090 0x7f095406b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:51810/0 (socket says 172.21.15.121:51810) 2024-04-20T10:21:18.587 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.583+0000 7f094be00640 1 -- 172.21.15.121:0/1822893188 learned_addr learned my addr 172.21.15.121:0/1822893188 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:18.587 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.584+0000 7f0952a00640 1 --2- 172.21.15.121:0/1822893188 >> v2:172.21.15.123:3300/0 conn(0x7f0954162c00 0x7f09540d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:18.587 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.584+0000 7f0953400640 1 --2- 172.21.15.121:0/1822893188 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f095406fc30 0x7f09540d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:18.587 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.584+0000 7f0952000640 1 -- 172.21.15.121:0/1822893188 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3877021134 0 0) 0x7f095406df70 con 0x7f095406f090 2024-04-20T10:21:18.587 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.584+0000 7f0952000640 1 -- 172.21.15.121:0/1822893188 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0938003520 con 0x7f095406f090 2024-04-20T10:21:18.587 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.584+0000 7f0952000640 1 -- 172.21.15.121:0/1822893188 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2870175318 0 0) 0x7f0938003520 con 0x7f095406f090 2024-04-20T10:21:18.587 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.584+0000 7f0952000640 1 -- 172.21.15.121:0/1822893188 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f095406fc30 msgr2=0x7f09540d08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:18.587 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.584+0000 7f0952000640 1 --2- 172.21.15.121:0/1822893188 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f095406fc30 0x7f09540d08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:18.588 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.584+0000 7f0952000640 1 -- 172.21.15.121:0/1822893188 >> v2:172.21.15.123:3300/0 conn(0x7f0954162c00 msgr2=0x7f09540d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:18.588 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.584+0000 7f0952000640 1 --2- 172.21.15.121:0/1822893188 >> v2:172.21.15.123:3300/0 conn(0x7f0954162c00 0x7f09540d0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:18.588 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.584+0000 7f0952000640 1 -- 172.21.15.121:0/1822893188 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f095406e310 con 0x7f095406f090 2024-04-20T10:21:18.588 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.584+0000 7f0952000640 1 -- 172.21.15.121:0/1822893188 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f093c001670 con 0x7f095406f090 2024-04-20T10:21:18.588 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.584+0000 7f0952000640 1 -- 172.21.15.121:0/1822893188 >> v1:172.21.15.121:6789/0 conn(0x7f095406f090 legacy=0x7f095406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:18.589 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.584+0000 7f0952000640 1 --2- 172.21.15.121:0/1822893188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09380039b0 0x7f0938003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:18.589 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.584+0000 7f0952000640 1 --2- 172.21.15.121:0/1822893188 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0938004800 0x7f0938004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:18.589 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.584+0000 7f0952000640 1 --2- 172.21.15.121:0/1822893188 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f095406fc30 0x7f09380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:18.589 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.584+0000 7f0952000640 1 -- 172.21.15.121:0/1822893188 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f095406ec00 con 0x7f0938004800 2024-04-20T10:21:18.589 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.584+0000 7f0952000640 1 -- 172.21.15.121:0/1822893188 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f093800b9c0 con 0x7f09380039b0 2024-04-20T10:21:18.589 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.584+0000 7f0952000640 1 -- 172.21.15.121:0/1822893188 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f093800bb40 con 0x7f095406fc30 2024-04-20T10:21:18.589 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.585+0000 7f094be00640 1 --2- 172.21.15.121:0/1822893188 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0938004800 0x7f0938004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:18.589 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.585+0000 7f0952a00640 1 --2- 172.21.15.121:0/1822893188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09380039b0 0x7f0938003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:18.590 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.585+0000 7f0953400640 1 --2- 172.21.15.121:0/1822893188 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f095406fc30 0x7f09380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:18.590 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.585+0000 7f094be00640 1 -- 172.21.15.121:0/1822893188 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f095406fc30 msgr2=0x7f09380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:18.590 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.585+0000 7f094be00640 1 --2- 172.21.15.121:0/1822893188 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f095406fc30 0x7f09380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:18.590 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.585+0000 7f094be00640 1 -- 172.21.15.121:0/1822893188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09380039b0 msgr2=0x7f0938003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:18.590 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.585+0000 7f094be00640 1 --2- 172.21.15.121:0/1822893188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09380039b0 0x7f0938003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:18.590 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.585+0000 7f094be00640 1 -- 172.21.15.121:0/1822893188 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0938003520 con 0x7f0938004800 2024-04-20T10:21:18.590 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.586+0000 7f094be00640 1 --2- 172.21.15.121:0/1822893188 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0938004800 0x7f0938004c80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f093c00c290 tx=0x7f0938005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b2cd9b46e5efb43 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:18.593 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.586+0000 7f0952000640 1 -- 172.21.15.121:0/1822893188 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093c007d50 con 0x7f0938004800 2024-04-20T10:21:18.593 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.586+0000 7f0952000640 1 -- 172.21.15.121:0/1822893188 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f093c030070 con 0x7f0938004800 2024-04-20T10:21:18.593 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.586+0000 7f0952000640 1 -- 172.21.15.121:0/1822893188 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093c005d30 con 0x7f0938004800 2024-04-20T10:21:18.593 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.587+0000 7f0959800640 1 -- 172.21.15.121:0/1822893188 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0938004800 msgr2=0x7f0938004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:18.593 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.587+0000 7f0959800640 1 --2- 172.21.15.121:0/1822893188 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0938004800 0x7f0938004c80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f093c00c290 tx=0x7f0938005830 comp rx=0 tx=0).stop 2024-04-20T10:21:18.593 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.587+0000 7f0953400640 1 -- 172.21.15.121:0/1822893188 reap_dead start 2024-04-20T10:21:18.593 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.587+0000 7f0959800640 1 -- 172.21.15.121:0/1822893188 shutdown_connections 2024-04-20T10:21:18.593 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.587+0000 7f0959800640 1 -- 172.21.15.121:0/1822893188 >> 172.21.15.121:0/1822893188 conn(0x7f09540b3350 msgr2=0x7f09540b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:18.593 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.587+0000 7f0959800640 1 -- 172.21.15.121:0/1822893188 shutdown_connections 2024-04-20T10:21:18.594 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.587+0000 7f0959800640 1 -- 172.21.15.121:0/1822893188 wait complete. 2024-04-20T10:21:18.594 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.589+0000 7f0959800640 1 Processor -- start 2024-04-20T10:21:18.594 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.589+0000 7f0959800640 1 -- start start 2024-04-20T10:21:18.594 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.590+0000 7f0959800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0938004800 0x7f09540b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:18.594 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.590+0000 7f0959800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09540cea20 0x7f09540cee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:18.594 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.590+0000 7f0959800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09541f7640 0x7f09541f7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:18.594 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.590+0000 7f0959800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f095406dbb0 con 0x7f0938004800 2024-04-20T10:21:18.594 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.590+0000 7f0959800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f095406e890 con 0x7f09540cea20 2024-04-20T10:21:18.594 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.590+0000 7f0959800640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f095406dd30 con 0x7f09541f7640 2024-04-20T10:21:18.594 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.590+0000 7f0952a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0938004800 0x7f09540b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:18.594 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.590+0000 7f0952a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0938004800 0x7f09540b06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37582/0 (socket says 172.21.15.121:37582) 2024-04-20T10:21:18.594 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.590+0000 7f0952a00640 1 -- 172.21.15.121:0/717716386 learned_addr learned my addr 172.21.15.121:0/717716386 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:18.594 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.591+0000 7f094be00640 1 --2- 172.21.15.121:0/717716386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09540cea20 0x7f09540cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:18.594 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.591+0000 7f0953400640 1 --2- 172.21.15.121:0/717716386 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09541f7640 0x7f09541f7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:18.595 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.591+0000 7f0952a00640 1 -- 172.21.15.121:0/717716386 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09541f7640 msgr2=0x7f09541f7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:18.595 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.591+0000 7f0952a00640 1 --2- 172.21.15.121:0/717716386 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09541f7640 0x7f09541f7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:18.595 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.591+0000 7f0952a00640 1 -- 172.21.15.121:0/717716386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09540cea20 msgr2=0x7f09540cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:18.595 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.591+0000 7f0952a00640 1 --2- 172.21.15.121:0/717716386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09540cea20 0x7f09540cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:18.595 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.591+0000 7f0952a00640 1 -- 172.21.15.121:0/717716386 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09541f8030 con 0x7f0938004800 2024-04-20T10:21:18.595 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.591+0000 7f0952a00640 1 --2- 172.21.15.121:0/717716386 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0938004800 0x7f09540b06a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0944011600 tx=0x7f0944011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:18.595 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.591+0000 7f094aa00640 1 -- 172.21.15.121:0/717716386 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094400f920 con 0x7f0938004800 2024-04-20T10:21:18.595 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.592+0000 7f094aa00640 1 -- 172.21.15.121:0/717716386 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f094400fac0 con 0x7f0938004800 2024-04-20T10:21:18.595 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.592+0000 7f0959800640 1 -- 172.21.15.121:0/717716386 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0954163330 con 0x7f0938004800 2024-04-20T10:21:18.595 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.592+0000 7f094aa00640 1 -- 172.21.15.121:0/717716386 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094400fe70 con 0x7f0938004800 2024-04-20T10:21:18.595 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.592+0000 7f0959800640 1 -- 172.21.15.121:0/717716386 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0954163bd0 con 0x7f0938004800 2024-04-20T10:21:18.596 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.593+0000 7f0959800640 1 -- 172.21.15.121:0/717716386 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0954068980 con 0x7f0938004800 2024-04-20T10:21:18.596 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.594+0000 7f094aa00640 1 -- 172.21.15.121:0/717716386 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0944015460 con 0x7f0938004800 2024-04-20T10:21:18.596 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.595+0000 7f094aa00640 1 --2- 172.21.15.121:0/717716386 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f093006c8f0 0x7f093006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:18.596 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.595+0000 7f094aa00640 1 -- 172.21.15.121:0/717716386 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f094408c8a0 con 0x7f0938004800 2024-04-20T10:21:18.596 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.596+0000 7f094be00640 1 --2- 172.21.15.121:0/717716386 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f093006c8f0 0x7f093006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:18.597 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.596+0000 7f094be00640 1 --2- 172.21.15.121:0/717716386 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f093006c8f0 0x7f093006edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f093c034000 tx=0x7f093c007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:18.600 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.600+0000 7f094aa00640 1 -- 172.21.15.121:0/717716386 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0944019030 con 0x7f0938004800 2024-04-20T10:21:18.811 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.810+0000 7f0959800640 1 -- 172.21.15.121:0/717716386 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0954071da0 con 0x7f0938004800 2024-04-20T10:21:18.811 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.811+0000 7f094aa00640 1 -- 172.21.15.121:0/717716386 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7f094405a8f0 con 0x7f0938004800 2024-04-20T10:21:18.811 INFO:teuthology.orchestra.run.smithi121.stderr:listed 0 entries 2024-04-20T10:21:18.813 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.813+0000 7f0959800640 1 -- 172.21.15.121:0/717716386 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f093006c8f0 msgr2=0x7f093006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:18.814 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.813+0000 7f0959800640 1 --2- 172.21.15.121:0/717716386 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f093006c8f0 0x7f093006edb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f093c034000 tx=0x7f093c007000 comp rx=0 tx=0).stop 2024-04-20T10:21:18.814 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.813+0000 7f0959800640 1 -- 172.21.15.121:0/717716386 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0938004800 msgr2=0x7f09540b06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:18.814 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.813+0000 7f0959800640 1 --2- 172.21.15.121:0/717716386 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0938004800 0x7f09540b06a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f0944011600 tx=0x7f0944011ac0 comp rx=0 tx=0).stop 2024-04-20T10:21:18.814 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.813+0000 7f0959800640 1 -- 172.21.15.121:0/717716386 shutdown_connections 2024-04-20T10:21:18.814 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.813+0000 7f0959800640 1 --2- 172.21.15.121:0/717716386 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09541f7640 0x7f09541f7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:18.814 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.813+0000 7f0959800640 1 --2- 172.21.15.121:0/717716386 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f093006c8f0 0x7f093006edb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:18.814 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.813+0000 7f0959800640 1 --2- 172.21.15.121:0/717716386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09540cea20 0x7f09540cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:18.814 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.813+0000 7f0959800640 1 --2- 172.21.15.121:0/717716386 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0938004800 0x7f09540b06a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:18.814 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.813+0000 7f0959800640 1 -- 172.21.15.121:0/717716386 >> 172.21.15.121:0/717716386 conn(0x7f09540b3350 msgr2=0x7f09540d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:18.814 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.813+0000 7f0959800640 1 -- 172.21.15.121:0/717716386 shutdown_connections 2024-04-20T10:21:18.815 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:18.813+0000 7f0959800640 1 -- 172.21.15.121:0/717716386 wait complete. 2024-04-20T10:21:18.831 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T10:21:18.831 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:21:18.874 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:21:19.097 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.097+0000 7f9b3b200640 1 Processor -- start 2024-04-20T10:21:19.098 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.097+0000 7f9b3b200640 1 -- start start 2024-04-20T10:21:19.099 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.099+0000 7f9b3b200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9b3c159e10 0x7f9b3c071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:19.099 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.099+0000 7f9b3b200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9b3c15a1f0 0x7f9b3c071fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:19.099 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.099+0000 7f9b3b200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9b3c06da90 0x7f9b3c160030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:19.100 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.099+0000 7f9b3b200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f9b3c072900 con 0x7f9b3c15a1f0 2024-04-20T10:21:19.100 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.099+0000 7f9b3b200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f9b3c15ca40 con 0x7f9b3c159e10 2024-04-20T10:21:19.100 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.099+0000 7f9b3b200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f9b3c15cc90 con 0x7f9b3c06da90 2024-04-20T10:21:19.100 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.099+0000 7f9b39400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9b3c15a1f0 0x7f9b3c071fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:19.100 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.099+0000 7f9b39400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9b3c15a1f0 0x7f9b3c071fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37594/0 (socket says 172.21.15.121:37594) 2024-04-20T10:21:19.100 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.099+0000 7f9b39400640 1 -- 172.21.15.121:0/2328617627 learned_addr learned my addr 172.21.15.121:0/2328617627 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:19.101 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.099+0000 7f9b39400640 1 -- 172.21.15.121:0/2328617627 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9b3c06da90 msgr2=0x7f9b3c160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:19.101 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.099+0000 7f9b39400640 1 --2- 172.21.15.121:0/2328617627 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9b3c06da90 0x7f9b3c160030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:19.101 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.099+0000 7f9b39400640 1 -- 172.21.15.121:0/2328617627 >> v2:172.21.15.123:3300/0 conn(0x7f9b3c159e10 msgr2=0x7f9b3c071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:19.101 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.099+0000 7f9b39400640 1 --2- 172.21.15.121:0/2328617627 >> v2:172.21.15.123:3300/0 conn(0x7f9b3c159e10 0x7f9b3c071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:19.101 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.099+0000 7f9b39400640 1 -- 172.21.15.121:0/2328617627 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b3c15cf70 con 0x7f9b3c15a1f0 2024-04-20T10:21:19.101 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.101+0000 7f9b39400640 1 --2- 172.21.15.121:0/2328617627 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b3c15a1f0 0x7f9b3c071fc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c073f30 tx=0x7f9b2802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2105abd172904295 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:19.102 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.101+0000 7f9b38a00640 1 -- 172.21.15.121:0/2328617627 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b28035020 con 0x7f9b3c15a1f0 2024-04-20T10:21:19.102 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.101+0000 7f9b38a00640 1 -- 172.21.15.121:0/2328617627 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b2802d040 con 0x7f9b3c15a1f0 2024-04-20T10:21:19.103 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.101+0000 7f9b38a00640 1 -- 172.21.15.121:0/2328617627 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b28031a80 con 0x7f9b3c15a1f0 2024-04-20T10:21:19.103 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.101+0000 7f9b3b200640 1 -- 172.21.15.121:0/2328617627 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b3c15a1f0 msgr2=0x7f9b3c071fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:19.103 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.101+0000 7f9b3b200640 1 --2- 172.21.15.121:0/2328617627 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b3c15a1f0 0x7f9b3c071fc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c073f30 tx=0x7f9b2802b900 comp rx=0 tx=0).stop 2024-04-20T10:21:19.103 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.101+0000 7f9b3b200640 1 -- 172.21.15.121:0/2328617627 shutdown_connections 2024-04-20T10:21:19.103 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.101+0000 7f9b3b200640 1 --2- 172.21.15.121:0/2328617627 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9b3c06da90 0x7f9b3c160030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:19.103 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.101+0000 7f9b3b200640 1 --2- 172.21.15.121:0/2328617627 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b3c15a1f0 0x7f9b3c071fc0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:19.103 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.101+0000 7f9b3b200640 1 --2- 172.21.15.121:0/2328617627 >> v2:172.21.15.123:3300/0 conn(0x7f9b3c159e10 0x7f9b3c071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:19.103 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.101+0000 7f9b3b200640 1 -- 172.21.15.121:0/2328617627 >> 172.21.15.121:0/2328617627 conn(0x7f9b3c0d0040 msgr2=0x7f9b3c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:19.103 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.102+0000 7f9b3b200640 1 -- 172.21.15.121:0/2328617627 shutdown_connections 2024-04-20T10:21:19.103 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.102+0000 7f9b3b200640 1 -- 172.21.15.121:0/2328617627 wait complete. 2024-04-20T10:21:19.104 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.103+0000 7f9b3b200640 1 Processor -- start 2024-04-20T10:21:19.104 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.103+0000 7f9b3b200640 1 -- start start 2024-04-20T10:21:19.105 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.104+0000 7f9b3b200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9b3c06da90 0x7f9b3c14bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:19.105 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.104+0000 7f9b3b200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b3c159e10 0x7f9b3c15fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:19.105 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.104+0000 7f9b3b200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b3c15a1f0 0x7f9b3c160370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:19.106 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.104+0000 7f9b3b200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3c15d810 con 0x7f9b3c159e10 2024-04-20T10:21:19.106 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.104+0000 7f9b3b200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3c15d530 con 0x7f9b3c15a1f0 2024-04-20T10:21:19.106 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.104+0000 7f9b3b200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f9b3c06b900 con 0x7f9b3c06da90 2024-04-20T10:21:19.106 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.104+0000 7f9b39400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b3c159e10 0x7f9b3c15fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:19.106 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.104+0000 7f9b39400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b3c159e10 0x7f9b3c15fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37606/0 (socket says 172.21.15.121:37606) 2024-04-20T10:21:19.106 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.104+0000 7f9b39400640 1 -- 172.21.15.121:0/3832693039 learned_addr learned my addr 172.21.15.121:0/3832693039 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:19.106 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.105+0000 7f9b3a800640 1 --2- 172.21.15.121:0/3832693039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b3c15a1f0 0x7f9b3c160370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:19.106 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.105+0000 7f9b39e00640 1 --2- 172.21.15.121:0/3832693039 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9b3c06da90 0x7f9b3c14bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:19.106 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.105+0000 7f9b39400640 1 -- 172.21.15.121:0/3832693039 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9b3c06da90 msgr2=0x7f9b3c14bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:19.106 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.105+0000 7f9b39400640 1 --2- 172.21.15.121:0/3832693039 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9b3c06da90 0x7f9b3c14bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:19.107 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.105+0000 7f9b39400640 1 -- 172.21.15.121:0/3832693039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b3c15a1f0 msgr2=0x7f9b3c160370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:19.107 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.105+0000 7f9b39400640 1 --2- 172.21.15.121:0/3832693039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b3c15a1f0 0x7f9b3c160370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:19.107 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.105+0000 7f9b39400640 1 -- 172.21.15.121:0/3832693039 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b3c14c300 con 0x7f9b3c159e10 2024-04-20T10:21:19.107 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.105+0000 7f9b39400640 1 --2- 172.21.15.121:0/3832693039 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b3c159e10 0x7f9b3c15fe30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9b280090b0 tx=0x7f9b280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:19.108 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.105+0000 7f9b32000640 1 -- 172.21.15.121:0/3832693039 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2803ac60 con 0x7f9b3c159e10 2024-04-20T10:21:19.108 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.106+0000 7f9b32000640 1 -- 172.21.15.121:0/3832693039 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b2803ae20 con 0x7f9b3c159e10 2024-04-20T10:21:19.108 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.106+0000 7f9b3b200640 1 -- 172.21.15.121:0/3832693039 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b3c15f9f0 con 0x7f9b3c159e10 2024-04-20T10:21:19.108 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.106+0000 7f9b32000640 1 -- 172.21.15.121:0/3832693039 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b28044630 con 0x7f9b3c159e10 2024-04-20T10:21:19.108 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.106+0000 7f9b3b200640 1 -- 172.21.15.121:0/3832693039 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b3c15ec90 con 0x7f9b3c159e10 2024-04-20T10:21:19.108 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.107+0000 7f9b3b200640 1 -- 172.21.15.121:0/3832693039 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b3c075b60 con 0x7f9b3c159e10 2024-04-20T10:21:19.108 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.108+0000 7f9b32000640 1 -- 172.21.15.121:0/3832693039 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b2804b060 con 0x7f9b3c159e10 2024-04-20T10:21:19.109 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.109+0000 7f9b32000640 1 --2- 172.21.15.121:0/3832693039 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9b1c06c8f0 0x7f9b1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:19.110 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.109+0000 7f9b32000640 1 -- 172.21.15.121:0/3832693039 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9b280adf20 con 0x7f9b3c159e10 2024-04-20T10:21:19.110 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.109+0000 7f9b39e00640 1 --2- 172.21.15.121:0/3832693039 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9b1c06c8f0 0x7f9b1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:19.110 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.110+0000 7f9b39e00640 1 --2- 172.21.15.121:0/3832693039 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9b1c06c8f0 0x7f9b1c06edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c009890 tx=0x7f9b2c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:19.113 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.113+0000 7f9b32000640 1 -- 172.21.15.121:0/3832693039 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9b2803f070 con 0x7f9b3c159e10 2024-04-20T10:21:19.316 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.316+0000 7f9b3b200640 1 -- 172.21.15.121:0/3832693039 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9b3c157000 con 0x7f9b3c159e10 2024-04-20T10:21:19.319 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.319+0000 7f9b32000640 1 -- 172.21.15.121:0/3832693039 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7f9b2807bf70 con 0x7f9b3c159e10 2024-04-20T10:21:19.320 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:21:19.320 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":25,"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","created":"2024-04-20T10:20:38.076076+0000","modified":"2024-04-20T10:21:07.336542+0000","last_up_change":"2024-04-20T10:20:50.162700+0000","last_in_change":"2024-04-20T10:20:41.510382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:20:51.141365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:21:00.436937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:21:01.026957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:21:04.862693+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"629a7b8c-a40a-4a2a-a3cc-d9f357774310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6809","nonce":1044588835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6811","nonce":1044588835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6815","nonce":1044588835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6813","nonce":1044588835}]},"public_addr":"172.21.15.121:6809/1044588835","cluster_addr":"172.21.15.121:6811/1044588835","heartbeat_back_addr":"172.21.15.121:6815/1044588835","heartbeat_front_addr":"172.21.15.121:6813/1044588835","state":["exists","up"]},{"osd":1,"uuid":"f2570399-aa50-4ff3-bf5d-3d09207290db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6801","nonce":3505199624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6803","nonce":3505199624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6807","nonce":3505199624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6805","nonce":3505199624}]},"public_addr":"172.21.15.121:6801/3505199624","cluster_addr":"172.21.15.121:6803/3505199624","heartbeat_back_addr":"172.21.15.121:6807/3505199624","heartbeat_front_addr":"172.21.15.121:6805/3505199624","state":["exists","up"]},{"osd":2,"uuid":"50893807-dc30-49f4-b41f-739366626658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6817","nonce":3656724207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6819","nonce":3656724207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6827","nonce":3656724207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6822","nonce":3656724207}]},"public_addr":"172.21.15.121:6817/3656724207","cluster_addr":"172.21.15.121:6819/3656724207","heartbeat_back_addr":"172.21.15.121:6827/3656724207","heartbeat_front_addr":"172.21.15.121:6822/3656724207","state":["exists","up"]},{"osd":3,"uuid":"dadb01bc-5491-4ffc-8824-d48a057240b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6823","nonce":2232342832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6826","nonce":2232342832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6831","nonce":2232342832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6829","nonce":2232342832}]},"public_addr":"172.21.15.121:6823/2232342832","cluster_addr":"172.21.15.121:6826/2232342832","heartbeat_back_addr":"172.21.15.121:6831/2232342832","heartbeat_front_addr":"172.21.15.121:6829/2232342832","state":["exists","up"]},{"osd":4,"uuid":"b3f3b4a1-b375-4526-b69e-344ab24c2da1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6801","nonce":99353111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6803","nonce":99353111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6807","nonce":99353111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6805","nonce":99353111}]},"public_addr":"172.21.15.123:6801/99353111","cluster_addr":"172.21.15.123:6803/99353111","heartbeat_back_addr":"172.21.15.123:6807/99353111","heartbeat_front_addr":"172.21.15.123:6805/99353111","state":["exists","up"]},{"osd":5,"uuid":"2ee24a08-1624-4961-9642-d7b510282484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6818","nonce":2504374688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6822","nonce":2504374688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6830","nonce":2504374688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6826","nonce":2504374688}]},"public_addr":"172.21.15.123:6818/2504374688","cluster_addr":"172.21.15.123:6822/2504374688","heartbeat_back_addr":"172.21.15.123:6830/2504374688","heartbeat_front_addr":"172.21.15.123:6826/2504374688","state":["exists","up"]},{"osd":6,"uuid":"186118cc-6128-49de-acca-55260945c1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6819","nonce":4139299599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6823","nonce":4139299599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6829","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6831","nonce":4139299599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6827","nonce":4139299599}]},"public_addr":"172.21.15.123:6819/4139299599","cluster_addr":"172.21.15.123:6823/4139299599","heartbeat_back_addr":"172.21.15.123:6831/4139299599","heartbeat_front_addr":"172.21.15.123:6827/4139299599","state":["exists","up"]},{"osd":7,"uuid":"a00aac6e-4bd9-415a-a93d-ae85210865ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6809","nonce":4043981069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6811","nonce":4043981069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6815","nonce":4043981069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6813","nonce":4043981069}]},"public_addr":"172.21.15.123:6809/4043981069","cluster_addr":"172.21.15.123:6811/4043981069","heartbeat_back_addr":"172.21.15.123:6815/4043981069","heartbeat_front_addr":"172.21.15.123:6813/4043981069","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.904540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.958585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.616367+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.885244+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.103127+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.167248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.884348+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.862356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:21:19.323 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.322+0000 7f9b3b200640 1 -- 172.21.15.121:0/3832693039 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9b1c06c8f0 msgr2=0x7f9b1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:19.323 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.322+0000 7f9b3b200640 1 --2- 172.21.15.121:0/3832693039 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9b1c06c8f0 0x7f9b1c06edb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c009890 tx=0x7f9b2c008040 comp rx=0 tx=0).stop 2024-04-20T10:21:19.323 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.322+0000 7f9b3b200640 1 -- 172.21.15.121:0/3832693039 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b3c159e10 msgr2=0x7f9b3c15fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:19.323 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.322+0000 7f9b3b200640 1 --2- 172.21.15.121:0/3832693039 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b3c159e10 0x7f9b3c15fe30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9b280090b0 tx=0x7f9b280046c0 comp rx=0 tx=0).stop 2024-04-20T10:21:19.323 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.322+0000 7f9b3b200640 1 -- 172.21.15.121:0/3832693039 shutdown_connections 2024-04-20T10:21:19.323 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.322+0000 7f9b3b200640 1 --2- 172.21.15.121:0/3832693039 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9b1c06c8f0 0x7f9b1c06edb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:19.323 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.322+0000 7f9b3b200640 1 --2- 172.21.15.121:0/3832693039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b3c15a1f0 0x7f9b3c160370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:19.323 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.322+0000 7f9b3b200640 1 --2- 172.21.15.121:0/3832693039 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b3c159e10 0x7f9b3c15fe30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:19.324 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.322+0000 7f9b3b200640 1 --2- 172.21.15.121:0/3832693039 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9b3c06da90 0x7f9b3c14bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:19.324 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.322+0000 7f9b3b200640 1 -- 172.21.15.121:0/3832693039 >> 172.21.15.121:0/3832693039 conn(0x7f9b3c0d0040 msgr2=0x7f9b3c155640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:19.324 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.322+0000 7f9b3b200640 1 -- 172.21.15.121:0/3832693039 shutdown_connections 2024-04-20T10:21:19.324 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.322+0000 7f9b3b200640 1 -- 172.21.15.121:0/3832693039 wait complete. 2024-04-20T10:21:19.341 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:20:51.141365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:21:00.436937+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:21:01.026957+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T10:21:04.862693+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T10:21:19.342 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:21:19.545 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.544+0000 7f0843e00640 1 Processor -- start 2024-04-20T10:21:19.545 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.544+0000 7f0843e00640 1 -- start start 2024-04-20T10:21:19.546 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.546+0000 7f0843e00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0844162210 0x7f0844162670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:19.546 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.546+0000 7f0843e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08440b3ba0 con 0x7f084415e770 2024-04-20T10:21:19.547 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.546+0000 7f0843e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08440d3590 con 0x7f08440b6400 2024-04-20T10:21:19.547 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.546+0000 7f0843e00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f084415eb50 con 0x7f0844162210 2024-04-20T10:21:19.547 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.546+0000 7f0842a00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f084415e770 0x7f08440b3490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:51818/0 (socket says 172.21.15.121:51818) 2024-04-20T10:21:19.547 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.546+0000 7f0842a00640 1 -- 172.21.15.121:0/2790807491 learned_addr learned my addr 172.21.15.121:0/2790807491 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:19.549 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.547+0000 7f0842000640 1 --2- 172.21.15.121:0/2790807491 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0844162210 0x7f0844162670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:19.549 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.547+0000 7f0841600640 1 -- 172.21.15.121:0/2790807491 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1255060510 0 0) 0x7f08440b3ba0 con 0x7f084415e770 2024-04-20T10:21:19.550 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.547+0000 7f0841600640 1 -- 172.21.15.121:0/2790807491 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0828003520 con 0x7f084415e770 2024-04-20T10:21:19.550 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.547+0000 7f0841600640 1 -- 172.21.15.121:0/2790807491 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 843355361 0 0) 0x7f0828003520 con 0x7f084415e770 2024-04-20T10:21:19.550 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.547+0000 7f0841600640 1 -- 172.21.15.121:0/2790807491 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0844162210 msgr2=0x7f0844162670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:19.550 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.547+0000 7f0841600640 1 --2- 172.21.15.121:0/2790807491 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0844162210 0x7f0844162670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:19.550 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.547+0000 7f0841600640 1 -- 172.21.15.121:0/2790807491 >> v1:172.21.15.123:6789/0 conn(0x7f08440b6400 legacy=0x7f08440d42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:19.550 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.547+0000 7f0841600640 1 -- 172.21.15.121:0/2790807491 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08440d3870 con 0x7f084415e770 2024-04-20T10:21:19.550 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.547+0000 7f0841600640 1 -- 172.21.15.121:0/2790807491 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f0834001670 con 0x7f084415e770 2024-04-20T10:21:19.550 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.547+0000 7f0841600640 1 -- 172.21.15.121:0/2790807491 >> v1:172.21.15.121:6789/0 conn(0x7f084415e770 legacy=0x7f08440b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:19.550 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.547+0000 7f0841600640 1 --2- 172.21.15.121:0/2790807491 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08280039e0 0x7f0828003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:19.550 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.547+0000 7f0841600640 1 --2- 172.21.15.121:0/2790807491 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0844162210 0x7f0828004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:19.550 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.547+0000 7f0841600640 1 --2- 172.21.15.121:0/2790807491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0828004d40 0x7f08280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:19.551 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.547+0000 7f0841600640 1 -- 172.21.15.121:0/2790807491 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f084415eb50 con 0x7f0828004d40 2024-04-20T10:21:19.551 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.547+0000 7f0841600640 1 -- 172.21.15.121:0/2790807491 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f082800baf0 con 0x7f08280039e0 2024-04-20T10:21:19.551 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.547+0000 7f0841600640 1 -- 172.21.15.121:0/2790807491 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f082800bcf0 con 0x7f0844162210 2024-04-20T10:21:19.551 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.548+0000 7f0843400640 1 --2- 172.21.15.121:0/2790807491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0828004d40 0x7f08280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:19.551 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.548+0000 7f0842a00640 1 --2- 172.21.15.121:0/2790807491 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08280039e0 0x7f0828003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:19.551 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.548+0000 7f0842000640 1 --2- 172.21.15.121:0/2790807491 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0844162210 0x7f0828004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:19.551 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.548+0000 7f0843400640 1 -- 172.21.15.121:0/2790807491 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0844162210 msgr2=0x7f0828004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:19.551 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.548+0000 7f0843400640 1 --2- 172.21.15.121:0/2790807491 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0844162210 0x7f0828004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:19.551 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.548+0000 7f0843400640 1 -- 172.21.15.121:0/2790807491 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08280039e0 msgr2=0x7f0828003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:19.551 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.548+0000 7f0843400640 1 --2- 172.21.15.121:0/2790807491 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08280039e0 0x7f0828003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:19.551 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.548+0000 7f0843400640 1 -- 172.21.15.121:0/2790807491 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0828003520 con 0x7f0828004d40 2024-04-20T10:21:19.551 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.549+0000 7f0843400640 1 --2- 172.21.15.121:0/2790807491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0828004d40 0x7f08280051c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f083c001e50 tx=0x7f0828005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5c5361d75b584ab server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:19.553 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.549+0000 7f0841600640 1 -- 172.21.15.121:0/2790807491 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083c001a70 con 0x7f0828004d40 2024-04-20T10:21:19.553 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.550+0000 7f0841600640 1 -- 172.21.15.121:0/2790807491 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f083c003b70 con 0x7f0828004d40 2024-04-20T10:21:19.553 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.550+0000 7f0841600640 1 -- 172.21.15.121:0/2790807491 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083c0050d0 con 0x7f0828004d40 2024-04-20T10:21:19.553 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.550+0000 7f0843e00640 1 -- 172.21.15.121:0/2790807491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0828004d40 msgr2=0x7f08280051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:19.553 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.550+0000 7f0843e00640 1 --2- 172.21.15.121:0/2790807491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0828004d40 0x7f08280051c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f083c001e50 tx=0x7f0828005830 comp rx=0 tx=0).stop 2024-04-20T10:21:19.553 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.550+0000 7f0843400640 1 -- 172.21.15.121:0/2790807491 reap_dead start 2024-04-20T10:21:19.553 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.550+0000 7f0843e00640 1 -- 172.21.15.121:0/2790807491 shutdown_connections 2024-04-20T10:21:19.553 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.550+0000 7f0843e00640 1 -- 172.21.15.121:0/2790807491 >> 172.21.15.121:0/2790807491 conn(0x7f084409d6d0 msgr2=0x7f08440b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:19.553 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.550+0000 7f0843e00640 1 -- 172.21.15.121:0/2790807491 shutdown_connections 2024-04-20T10:21:19.553 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.550+0000 7f0843e00640 1 -- 172.21.15.121:0/2790807491 wait complete. 2024-04-20T10:21:19.554 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.552+0000 7f0843e00640 1 Processor -- start 2024-04-20T10:21:19.554 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.552+0000 7f0843e00640 1 -- start start 2024-04-20T10:21:19.556 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.553+0000 7f0843e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0828004d40 0x7f08440cb400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:19.556 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.553+0000 7f0843e00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f08440ccfd0 0x7f084415c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:19.556 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.553+0000 7f0843e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0844153420 0x7f08441538a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:19.556 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.553+0000 7f0843e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f08440d3e30 con 0x7f0844153420 2024-04-20T10:21:19.556 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.553+0000 7f0843e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f08440b5ee0 con 0x7f0828004d40 2024-04-20T10:21:19.556 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.553+0000 7f0843e00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f08440d3120 con 0x7f08440ccfd0 2024-04-20T10:21:19.556 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.554+0000 7f0843400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0844153420 0x7f08441538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:19.556 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.554+0000 7f0843400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0844153420 0x7f08441538a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37622/0 (socket says 172.21.15.121:37622) 2024-04-20T10:21:19.556 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.554+0000 7f0843400640 1 -- 172.21.15.121:0/1810098473 learned_addr learned my addr 172.21.15.121:0/1810098473 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:19.556 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.554+0000 7f0842a00640 1 --2- 172.21.15.121:0/1810098473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0828004d40 0x7f08440cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:19.557 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.554+0000 7f0842000640 1 --2- 172.21.15.121:0/1810098473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f08440ccfd0 0x7f084415c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:19.557 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.554+0000 7f0843400640 1 -- 172.21.15.121:0/1810098473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f08440ccfd0 msgr2=0x7f084415c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:19.557 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.554+0000 7f0843400640 1 --2- 172.21.15.121:0/1810098473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f08440ccfd0 0x7f084415c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:19.557 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.554+0000 7f0843400640 1 -- 172.21.15.121:0/1810098473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0828004d40 msgr2=0x7f08440cb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:19.557 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.554+0000 7f0843400640 1 --2- 172.21.15.121:0/1810098473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0828004d40 0x7f08440cb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:19.557 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.554+0000 7f0843400640 1 -- 172.21.15.121:0/1810098473 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08440cb940 con 0x7f0844153420 2024-04-20T10:21:19.557 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.554+0000 7f0843400640 1 --2- 172.21.15.121:0/1810098473 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0844153420 0x7f08441538a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f083c001e50 tx=0x7f083c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:19.557 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.555+0000 7f083b400640 1 -- 172.21.15.121:0/1810098473 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083c007db0 con 0x7f0844153420 2024-04-20T10:21:19.557 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.555+0000 7f083b400640 1 -- 172.21.15.121:0/1810098473 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f083c003d20 con 0x7f0844153420 2024-04-20T10:21:19.557 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.555+0000 7f083b400640 1 -- 172.21.15.121:0/1810098473 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083c02f670 con 0x7f0844153420 2024-04-20T10:21:19.557 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.555+0000 7f0843e00640 1 -- 172.21.15.121:0/1810098473 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08440cbbe0 con 0x7f0844153420 2024-04-20T10:21:19.557 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.555+0000 7f0843e00640 1 -- 172.21.15.121:0/1810098473 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08440cc1a0 con 0x7f0844153420 2024-04-20T10:21:19.558 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.556+0000 7f0843e00640 1 -- 172.21.15.121:0/1810098473 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0844075b30 con 0x7f0844153420 2024-04-20T10:21:19.558 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.557+0000 7f083b400640 1 -- 172.21.15.121:0/1810098473 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f083c041030 con 0x7f0844153420 2024-04-20T10:21:19.559 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.558+0000 7f083b400640 1 --2- 172.21.15.121:0/1810098473 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f082406c8f0 0x7f082406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:19.559 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.558+0000 7f083b400640 1 -- 172.21.15.121:0/1810098473 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f083c0ad390 con 0x7f0844153420 2024-04-20T10:21:19.559 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.558+0000 7f0842a00640 1 --2- 172.21.15.121:0/1810098473 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f082406c8f0 0x7f082406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:19.559 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.559+0000 7f0842a00640 1 --2- 172.21.15.121:0/1810098473 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f082406c8f0 0x7f082406edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0834001cd0 tx=0x7f0834007040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:19.562 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.562+0000 7f083b400640 1 -- 172.21.15.121:0/1810098473 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f083c033020 con 0x7f0844153420 2024-04-20T10:21:19.762 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.761+0000 7f0843e00640 1 -- 172.21.15.121:0/1810098473 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f08440d50c0 con 0x7f0844153420 2024-04-20T10:21:19.763 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.762+0000 7f083b400640 1 -- 172.21.15.121:0/1810098473 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f083c07ad90 con 0x7f0844153420 2024-04-20T10:21:19.763 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2024-04-20T10:21:19.766 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.765+0000 7f0843e00640 1 -- 172.21.15.121:0/1810098473 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f082406c8f0 msgr2=0x7f082406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:19.766 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.765+0000 7f0843e00640 1 --2- 172.21.15.121:0/1810098473 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f082406c8f0 0x7f082406edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0834001cd0 tx=0x7f0834007040 comp rx=0 tx=0).stop 2024-04-20T10:21:19.766 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.765+0000 7f0843e00640 1 -- 172.21.15.121:0/1810098473 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0844153420 msgr2=0x7f08441538a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:19.766 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.765+0000 7f0843e00640 1 --2- 172.21.15.121:0/1810098473 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0844153420 0x7f08441538a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f083c001e50 tx=0x7f083c0014a0 comp rx=0 tx=0).stop 2024-04-20T10:21:19.766 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.765+0000 7f0843e00640 1 -- 172.21.15.121:0/1810098473 shutdown_connections 2024-04-20T10:21:19.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.765+0000 7f0843e00640 1 --2- 172.21.15.121:0/1810098473 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0844153420 0x7f08441538a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:19.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.765+0000 7f0843e00640 1 --2- 172.21.15.121:0/1810098473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f08440ccfd0 0x7f084415c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:19.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.765+0000 7f0843e00640 1 --2- 172.21.15.121:0/1810098473 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f082406c8f0 0x7f082406edb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:19.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.765+0000 7f0843e00640 1 --2- 172.21.15.121:0/1810098473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0828004d40 0x7f08440cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:19.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.765+0000 7f0843e00640 1 -- 172.21.15.121:0/1810098473 >> 172.21.15.121:0/1810098473 conn(0x7f084409d6d0 msgr2=0x7f08440d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:19.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.765+0000 7f0843e00640 1 -- 172.21.15.121:0/1810098473 shutdown_connections 2024-04-20T10:21:19.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.765+0000 7f0843e00640 1 -- 172.21.15.121:0/1810098473 wait complete. 2024-04-20T10:21:19.784 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:21:19.983 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.982+0000 7f5fc8a00640 1 Processor -- start 2024-04-20T10:21:19.983 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.983+0000 7f5fc8a00640 1 -- start start 2024-04-20T10:21:19.984 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.984+0000 7f5fc8a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5fc415a930 0x7f5fc406cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:19.984 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.984+0000 7f5fc8a00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5fc406ec20 0x7f5fc4072230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:19.984 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.984+0000 7f5fc8a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fc415be80 con 0x7f5fc406db90 2024-04-20T10:21:19.984 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.984+0000 7f5fc8a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f5fc406d2c0 con 0x7f5fc415a930 2024-04-20T10:21:19.984 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.984+0000 7f5fc8a00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f5fc406bce0 con 0x7f5fc406ec20 2024-04-20T10:21:19.985 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.984+0000 7f5fc3e00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5fc406ec20 0x7f5fc4072230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:19.985 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.985+0000 7f5fc3e00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5fc406ec20 0x7f5fc4072230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.121:46406/0 (socket says 172.21.15.121:46406) 2024-04-20T10:21:19.986 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.985+0000 7f5fc3e00640 1 -- 172.21.15.121:0/2354136060 learned_addr learned my addr 172.21.15.121:0/2354136060 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:19.986 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.985+0000 7f5fc3400640 1 --2- 172.21.15.121:0/2354136060 >> v2:172.21.15.123:3300/0 conn(0x7f5fc415a930 0x7f5fc406cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:19.986 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.985+0000 7f5fc3e00640 1 -- 172.21.15.121:0/2354136060 >> v2:172.21.15.123:3300/0 conn(0x7f5fc415a930 msgr2=0x7f5fc406cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:19.986 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.985+0000 7f5fc3e00640 1 --2- 172.21.15.121:0/2354136060 >> v2:172.21.15.123:3300/0 conn(0x7f5fc415a930 0x7f5fc406cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:19.986 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.985+0000 7f5fc3e00640 1 -- 172.21.15.121:0/2354136060 >> v1:172.21.15.123:6789/0 conn(0x7f5fc406db90 legacy=0x7f5fc406e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:19.986 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.985+0000 7f5fc3e00640 1 -- 172.21.15.121:0/2354136060 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fc406bfc0 con 0x7f5fc406ec20 2024-04-20T10:21:19.987 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.987+0000 7f5fc3e00640 1 --2- 172.21.15.121:0/2354136060 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5fc406ec20 0x7f5fc4072230 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc009e50 tx=0x7f5fbc02bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e40f0f152115d723 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:19.987 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.987+0000 7f5fc2a00640 1 -- 172.21.15.121:0/2354136060 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc02ca70 con 0x7f5fc406ec20 2024-04-20T10:21:19.987 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.987+0000 7f5fc2a00640 1 -- 172.21.15.121:0/2354136060 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fbc02cc10 con 0x7f5fc406ec20 2024-04-20T10:21:19.987 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.987+0000 7f5fc2a00640 1 -- 172.21.15.121:0/2354136060 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc0343f0 con 0x7f5fc406ec20 2024-04-20T10:21:19.988 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.988+0000 7f5fc8a00640 1 -- 172.21.15.121:0/2354136060 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5fc406ec20 msgr2=0x7f5fc4072230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:19.988 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.988+0000 7f5fc8a00640 1 --2- 172.21.15.121:0/2354136060 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5fc406ec20 0x7f5fc4072230 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc009e50 tx=0x7f5fbc02bed0 comp rx=0 tx=0).stop 2024-04-20T10:21:19.988 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.988+0000 7f5fc8a00640 1 -- 172.21.15.121:0/2354136060 shutdown_connections 2024-04-20T10:21:19.988 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.988+0000 7f5fc8a00640 1 --2- 172.21.15.121:0/2354136060 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5fc406ec20 0x7f5fc4072230 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:19.989 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.988+0000 7f5fc8a00640 1 --2- 172.21.15.121:0/2354136060 >> v2:172.21.15.123:3300/0 conn(0x7f5fc415a930 0x7f5fc406cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:19.989 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.988+0000 7f5fc8a00640 1 -- 172.21.15.121:0/2354136060 >> 172.21.15.121:0/2354136060 conn(0x7f5fc40b5270 msgr2=0x7f5fc40b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:19.989 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.988+0000 7f5fc8a00640 1 -- 172.21.15.121:0/2354136060 shutdown_connections 2024-04-20T10:21:19.989 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.988+0000 7f5fc8a00640 1 -- 172.21.15.121:0/2354136060 wait complete. 2024-04-20T10:21:19.990 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.990+0000 7f5fc8a00640 1 Processor -- start 2024-04-20T10:21:19.990 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.990+0000 7f5fc8a00640 1 -- start start 2024-04-20T10:21:19.992 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.991+0000 7f5fc8a00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5fc406db90 0x7f5fc4147e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:19.992 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.991+0000 7f5fc8a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fc406ec20 0x7f5fc415cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:19.992 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.991+0000 7f5fc8a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fc415a930 0x7f5fc415d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:19.992 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.991+0000 7f5fc8a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc415ad10 con 0x7f5fc415a930 2024-04-20T10:21:19.992 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.991+0000 7f5fc8a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc406c580 con 0x7f5fc406ec20 2024-04-20T10:21:19.992 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.991+0000 7f5fc8a00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f5fc406e040 con 0x7f5fc406db90 2024-04-20T10:21:19.992 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.991+0000 7f5fc3e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fc415a930 0x7f5fc415d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:19.992 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.991+0000 7f5fc3e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fc415a930 0x7f5fc415d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37628/0 (socket says 172.21.15.121:37628) 2024-04-20T10:21:19.992 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.991+0000 7f5fc3e00640 1 -- 172.21.15.121:0/3855994870 learned_addr learned my addr 172.21.15.121:0/3855994870 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:19.992 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.991+0000 7f5fc3400640 1 --2- 172.21.15.121:0/3855994870 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5fc406db90 0x7f5fc4147e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:19.992 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.991+0000 7f5fbbe00640 1 --2- 172.21.15.121:0/3855994870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fc406ec20 0x7f5fc415cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:19.992 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.992+0000 7f5fc3e00640 1 -- 172.21.15.121:0/3855994870 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5fc406db90 msgr2=0x7f5fc4147e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:19.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.992+0000 7f5fc3e00640 1 --2- 172.21.15.121:0/3855994870 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5fc406db90 0x7f5fc4147e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:19.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.992+0000 7f5fc3e00640 1 -- 172.21.15.121:0/3855994870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fc406ec20 msgr2=0x7f5fc415cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:19.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.992+0000 7f5fc3e00640 1 --2- 172.21.15.121:0/3855994870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fc406ec20 0x7f5fc415cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:19.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.992+0000 7f5fc3e00640 1 -- 172.21.15.121:0/3855994870 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fc40b1e50 con 0x7f5fc415a930 2024-04-20T10:21:19.994 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.992+0000 7f5fc3e00640 1 --2- 172.21.15.121:0/3855994870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fc415a930 0x7f5fc415d2e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc036040 tx=0x7f5fbc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:19.994 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.992+0000 7f5fbb400640 1 -- 172.21.15.121:0/3855994870 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc043020 con 0x7f5fc415a930 2024-04-20T10:21:19.994 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.992+0000 7f5fbb400640 1 -- 172.21.15.121:0/3855994870 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fbc03e450 con 0x7f5fc415a930 2024-04-20T10:21:19.994 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.992+0000 7f5fbb400640 1 -- 172.21.15.121:0/3855994870 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc009510 con 0x7f5fc415a930 2024-04-20T10:21:19.994 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.992+0000 7f5fc8a00640 1 -- 172.21.15.121:0/3855994870 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fc40b2130 con 0x7f5fc415a930 2024-04-20T10:21:19.994 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.992+0000 7f5fc8a00640 1 -- 172.21.15.121:0/3855994870 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fc40b26f0 con 0x7f5fc415a930 2024-04-20T10:21:19.994 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.994+0000 7f5fc8a00640 1 -- 172.21.15.121:0/3855994870 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fc40b28f0 con 0x7f5fc415a930 2024-04-20T10:21:19.995 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.994+0000 7f5fbb400640 1 -- 172.21.15.121:0/3855994870 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5fbc046020 con 0x7f5fc415a930 2024-04-20T10:21:19.995 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.995+0000 7f5fbb400640 1 --2- 172.21.15.121:0/3855994870 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f5fa006c8f0 0x7f5fa006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:19.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.995+0000 7f5fbb400640 1 -- 172.21.15.121:0/3855994870 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5fbc0ae6a0 con 0x7f5fc415a930 2024-04-20T10:21:19.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.996+0000 7f5fc3400640 1 --2- 172.21.15.121:0/3855994870 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f5fa006c8f0 0x7f5fa006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:19.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:19.996+0000 7f5fc3400640 1 --2- 172.21.15.121:0/3855994870 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f5fa006c8f0 0x7f5fa006edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5fc414bfc0 tx=0x7f5fb4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:20.000 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.000+0000 7f5fbb400640 1 -- 172.21.15.121:0/3855994870 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5fc40b28f0 con 0x7f5fc415a930 2024-04-20T10:21:20.198 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.198+0000 7f5fc8a00640 1 -- 172.21.15.121:0/3855994870 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5fc4075b30 con 0x7f5fc415a930 2024-04-20T10:21:20.199 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.199+0000 7f5fbb400640 1 -- 172.21.15.121:0/3855994870 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f5fbc04d030 con 0x7f5fc415a930 2024-04-20T10:21:20.199 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 32 2024-04-20T10:21:20.202 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.201+0000 7f5fc8a00640 1 -- 172.21.15.121:0/3855994870 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f5fa006c8f0 msgr2=0x7f5fa006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:20.202 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.201+0000 7f5fc8a00640 1 --2- 172.21.15.121:0/3855994870 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f5fa006c8f0 0x7f5fa006edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5fc414bfc0 tx=0x7f5fb4008040 comp rx=0 tx=0).stop 2024-04-20T10:21:20.202 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.201+0000 7f5fc8a00640 1 -- 172.21.15.121:0/3855994870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fc415a930 msgr2=0x7f5fc415d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:20.202 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.201+0000 7f5fc8a00640 1 --2- 172.21.15.121:0/3855994870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fc415a930 0x7f5fc415d2e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc036040 tx=0x7f5fbc004a40 comp rx=0 tx=0).stop 2024-04-20T10:21:20.203 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.202+0000 7f5fc8a00640 1 -- 172.21.15.121:0/3855994870 shutdown_connections 2024-04-20T10:21:20.203 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.202+0000 7f5fc8a00640 1 --2- 172.21.15.121:0/3855994870 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5fc415a930 0x7f5fc415d2e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:20.203 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.202+0000 7f5fc8a00640 1 --2- 172.21.15.121:0/3855994870 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f5fa006c8f0 0x7f5fa006edb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:20.203 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.202+0000 7f5fc8a00640 1 --2- 172.21.15.121:0/3855994870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fc406ec20 0x7f5fc415cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:20.203 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.202+0000 7f5fc8a00640 1 --2- 172.21.15.121:0/3855994870 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5fc406db90 0x7f5fc4147e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:20.203 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.202+0000 7f5fc8a00640 1 -- 172.21.15.121:0/3855994870 >> 172.21.15.121:0/3855994870 conn(0x7f5fc40b5270 msgr2=0x7f5fc41525a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:20.203 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.202+0000 7f5fc8a00640 1 -- 172.21.15.121:0/3855994870 shutdown_connections 2024-04-20T10:21:20.203 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.202+0000 7f5fc8a00640 1 -- 172.21.15.121:0/3855994870 wait complete. 2024-04-20T10:21:20.222 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:21:20.416 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.415+0000 7f2733600640 1 Processor -- start 2024-04-20T10:21:20.416 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.416+0000 7f2733600640 1 -- start start 2024-04-20T10:21:20.418 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.417+0000 7f2733600640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f272c0d4270 0x7f272c0d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:20.418 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.417+0000 7f2733600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f272c0d4650 0x7f272c0b1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:20.418 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.417+0000 7f2733600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f272c0d3820 con 0x7f272c0b17d0 2024-04-20T10:21:20.418 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.417+0000 7f2733600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f272c0d40c0 con 0x7f272c0d4650 2024-04-20T10:21:20.418 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.417+0000 7f2733600640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f272c0d3b00 con 0x7f272c0d4270 2024-04-20T10:21:20.418 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.417+0000 7f2731600640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f272c0b17d0 0x7f272c0d0380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:51832/0 (socket says 172.21.15.121:51832) 2024-04-20T10:21:20.418 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.417+0000 7f2731600640 1 -- 172.21.15.121:0/186849324 learned_addr learned my addr 172.21.15.121:0/186849324 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:20.421 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.418+0000 7f272be00640 1 --2- 172.21.15.121:0/186849324 >> v2:172.21.15.123:3300/0 conn(0x7f272c0d4650 0x7f272c0b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:20.421 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.418+0000 7f2730c00640 1 --2- 172.21.15.121:0/186849324 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f272c0d4270 0x7f272c0d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:20.421 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.418+0000 7f272b400640 1 -- 172.21.15.121:0/186849324 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3705776214 0 0) 0x7f272c0d3820 con 0x7f272c0b17d0 2024-04-20T10:21:20.421 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.418+0000 7f272b400640 1 -- 172.21.15.121:0/186849324 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2710003520 con 0x7f272c0b17d0 2024-04-20T10:21:20.421 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.418+0000 7f272b400640 1 -- 172.21.15.121:0/186849324 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 690433853 0 0) 0x7f2710003520 con 0x7f272c0b17d0 2024-04-20T10:21:20.422 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.418+0000 7f272b400640 1 -- 172.21.15.121:0/186849324 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f272c0d4270 msgr2=0x7f272c0d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:20.422 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.418+0000 7f272b400640 1 --2- 172.21.15.121:0/186849324 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f272c0d4270 0x7f272c0d1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:20.422 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.418+0000 7f272b400640 1 -- 172.21.15.121:0/186849324 >> v2:172.21.15.123:3300/0 conn(0x7f272c0d4650 msgr2=0x7f272c0b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:20.422 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.418+0000 7f272b400640 1 --2- 172.21.15.121:0/186849324 >> v2:172.21.15.123:3300/0 conn(0x7f272c0d4650 0x7f272c0b1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:20.422 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.418+0000 7f272b400640 1 -- 172.21.15.121:0/186849324 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f272c0d3de0 con 0x7f272c0b17d0 2024-04-20T10:21:20.422 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.418+0000 7f272b400640 1 -- 172.21.15.121:0/186849324 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f27240019f0 con 0x7f272c0b17d0 2024-04-20T10:21:20.422 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.418+0000 7f272b400640 1 -- 172.21.15.121:0/186849324 >> v1:172.21.15.121:6789/0 conn(0x7f272c0b17d0 legacy=0x7f272c0d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:20.422 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.418+0000 7f272b400640 1 --2- 172.21.15.121:0/186849324 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f272c0d4270 0x7f27100039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:20.422 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.418+0000 7f272b400640 1 --2- 172.21.15.121:0/186849324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2710003f80 0x7f2710004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:20.422 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.419+0000 7f272b400640 1 --2- 172.21.15.121:0/186849324 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2710004d40 0x7f27100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:20.422 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.419+0000 7f272b400640 1 -- 172.21.15.121:0/186849324 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f272c0d40c0 con 0x7f2710004d40 2024-04-20T10:21:20.423 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.419+0000 7f272b400640 1 -- 172.21.15.121:0/186849324 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f272c0d3b00 con 0x7f2710003f80 2024-04-20T10:21:20.423 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.419+0000 7f272b400640 1 -- 172.21.15.121:0/186849324 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f271000b9c0 con 0x7f272c0d4270 2024-04-20T10:21:20.423 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.419+0000 7f2731600640 1 --2- 172.21.15.121:0/186849324 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2710004d40 0x7f27100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:20.423 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.419+0000 7f2730c00640 1 --2- 172.21.15.121:0/186849324 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f272c0d4270 0x7f27100039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:20.423 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.419+0000 7f272be00640 1 --2- 172.21.15.121:0/186849324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2710003f80 0x7f2710004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:20.423 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.419+0000 7f2731600640 1 -- 172.21.15.121:0/186849324 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f272c0d4270 msgr2=0x7f27100039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:20.423 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.419+0000 7f2731600640 1 --2- 172.21.15.121:0/186849324 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f272c0d4270 0x7f27100039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:20.423 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.419+0000 7f2731600640 1 -- 172.21.15.121:0/186849324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2710003f80 msgr2=0x7f2710004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:20.423 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.419+0000 7f2731600640 1 --2- 172.21.15.121:0/186849324 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2710003f80 0x7f2710004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:20.423 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.419+0000 7f2731600640 1 -- 172.21.15.121:0/186849324 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2710003520 con 0x7f2710004d40 2024-04-20T10:21:20.423 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.420+0000 7f2731600640 1 --2- 172.21.15.121:0/186849324 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2710004d40 0x7f27100051c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f272400a7a0 tx=0x7f2710005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca56ed18f89fb57 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:20.423 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.421+0000 7f272b400640 1 -- 172.21.15.121:0/186849324 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272400a3f0 con 0x7f2710004d40 2024-04-20T10:21:20.423 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.421+0000 7f272b400640 1 -- 172.21.15.121:0/186849324 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2724034630 con 0x7f2710004d40 2024-04-20T10:21:20.423 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.421+0000 7f272b400640 1 -- 172.21.15.121:0/186849324 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2724034a80 con 0x7f2710004d40 2024-04-20T10:21:20.425 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.421+0000 7f2733600640 1 -- 172.21.15.121:0/186849324 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2710004d40 msgr2=0x7f27100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:20.425 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.421+0000 7f2733600640 1 --2- 172.21.15.121:0/186849324 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2710004d40 0x7f27100051c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f272400a7a0 tx=0x7f2710005830 comp rx=0 tx=0).stop 2024-04-20T10:21:20.425 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.421+0000 7f2731600640 1 -- 172.21.15.121:0/186849324 reap_dead start 2024-04-20T10:21:20.425 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.421+0000 7f2733600640 1 -- 172.21.15.121:0/186849324 shutdown_connections 2024-04-20T10:21:20.425 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.421+0000 7f2733600640 1 -- 172.21.15.121:0/186849324 >> 172.21.15.121:0/186849324 conn(0x7f272c0b99c0 msgr2=0x7f272c0b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:20.425 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.421+0000 7f2733600640 1 -- 172.21.15.121:0/186849324 shutdown_connections 2024-04-20T10:21:20.425 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.421+0000 7f2733600640 1 -- 172.21.15.121:0/186849324 wait complete. 2024-04-20T10:21:20.425 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.424+0000 7f2733600640 1 Processor -- start 2024-04-20T10:21:20.425 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.424+0000 7f2733600640 1 -- start start 2024-04-20T10:21:20.427 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.425+0000 7f2733600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2710004d40 0x7f272c06cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:20.427 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.425+0000 7f2733600640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f272c0c9490 0x7f272c14f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:20.427 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.425+0000 7f2733600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f272c160980 0x7f272c160e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:20.427 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.425+0000 7f2733600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f272c161f70 con 0x7f2710004d40 2024-04-20T10:21:20.427 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.425+0000 7f2733600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f272c0d53b0 con 0x7f272c160980 2024-04-20T10:21:20.428 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.425+0000 7f2733600640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f272c1620f0 con 0x7f272c0c9490 2024-04-20T10:21:20.428 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.425+0000 7f2730c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2710004d40 0x7f272c06cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:20.428 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.425+0000 7f2730c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2710004d40 0x7f272c06cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37654/0 (socket says 172.21.15.121:37654) 2024-04-20T10:21:20.428 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.425+0000 7f2730c00640 1 -- 172.21.15.121:0/4047829604 learned_addr learned my addr 172.21.15.121:0/4047829604 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:20.428 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.425+0000 7f2730c00640 1 -- 172.21.15.121:0/4047829604 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f272c0c9490 msgr2=0x7f272c14f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:20.428 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.425+0000 7f2730c00640 1 --2- 172.21.15.121:0/4047829604 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f272c0c9490 0x7f272c14f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:20.428 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.425+0000 7f2730c00640 1 -- 172.21.15.121:0/4047829604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f272c160980 msgr2=0x7f272c160e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:20.428 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.425+0000 7f2731600640 1 --2- 172.21.15.121:0/4047829604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f272c160980 0x7f272c160e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:20.428 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.426+0000 7f2730c00640 1 --2- 172.21.15.121:0/4047829604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f272c160980 0x7f272c160e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:20.428 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.426+0000 7f2730c00640 1 -- 172.21.15.121:0/4047829604 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f272c06d5c0 con 0x7f2710004d40 2024-04-20T10:21:20.429 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.426+0000 7f2730c00640 1 --2- 172.21.15.121:0/4047829604 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2710004d40 0x7f272c06cec0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f271c011630 tx=0x7f271c011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:20.429 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.426+0000 7f2728c00640 1 -- 172.21.15.121:0/4047829604 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c015870 con 0x7f2710004d40 2024-04-20T10:21:20.429 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.426+0000 7f2728c00640 1 -- 172.21.15.121:0/4047829604 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f271c015a10 con 0x7f2710004d40 2024-04-20T10:21:20.429 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.426+0000 7f2733600640 1 -- 172.21.15.121:0/4047829604 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f272c06d8a0 con 0x7f2710004d40 2024-04-20T10:21:20.430 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.426+0000 7f2728c00640 1 -- 172.21.15.121:0/4047829604 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f271c015dc0 con 0x7f2710004d40 2024-04-20T10:21:20.430 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.426+0000 7f2733600640 1 -- 172.21.15.121:0/4047829604 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f272c06de60 con 0x7f2710004d40 2024-04-20T10:21:20.432 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.428+0000 7f2733600640 1 -- 172.21.15.121:0/4047829604 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f272c161340 con 0x7f2710004d40 2024-04-20T10:21:20.432 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.428+0000 7f2728c00640 1 -- 172.21.15.121:0/4047829604 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f271c00f450 con 0x7f2710004d40 2024-04-20T10:21:20.432 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.429+0000 7f2728c00640 1 --2- 172.21.15.121:0/4047829604 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f270806c920 0x7f270806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:20.432 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.429+0000 7f2728c00640 1 -- 172.21.15.121:0/4047829604 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f271c08c750 con 0x7f2710004d40 2024-04-20T10:21:20.432 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.430+0000 7f272be00640 1 --2- 172.21.15.121:0/4047829604 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f270806c920 0x7f270806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:20.432 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.430+0000 7f272be00640 1 --2- 172.21.15.121:0/4047829604 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f270806c920 0x7f270806ede0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f272c161590 tx=0x7f2714008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:20.434 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.434+0000 7f2728c00640 1 -- 172.21.15.121:0/4047829604 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f271c019030 con 0x7f2710004d40 2024-04-20T10:21:20.632 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.632+0000 7f2733600640 1 -- 172.21.15.121:0/4047829604 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f272c068980 con 0x7f2710004d40 2024-04-20T10:21:20.633 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.633+0000 7f2728c00640 1 -- 172.21.15.121:0/4047829604 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f271c05a1e0 con 0x7f2710004d40 2024-04-20T10:21:20.633 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2024-04-20T10:21:20.636 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.635+0000 7f2733600640 1 -- 172.21.15.121:0/4047829604 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f270806c920 msgr2=0x7f270806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:20.636 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.635+0000 7f2733600640 1 --2- 172.21.15.121:0/4047829604 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f270806c920 0x7f270806ede0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f272c161590 tx=0x7f2714008040 comp rx=0 tx=0).stop 2024-04-20T10:21:20.636 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.635+0000 7f2733600640 1 -- 172.21.15.121:0/4047829604 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2710004d40 msgr2=0x7f272c06cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:20.636 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.635+0000 7f2733600640 1 --2- 172.21.15.121:0/4047829604 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2710004d40 0x7f272c06cec0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f271c011630 tx=0x7f271c011af0 comp rx=0 tx=0).stop 2024-04-20T10:21:20.636 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.635+0000 7f2733600640 1 -- 172.21.15.121:0/4047829604 shutdown_connections 2024-04-20T10:21:20.636 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.635+0000 7f2733600640 1 --2- 172.21.15.121:0/4047829604 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f270806c920 0x7f270806ede0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:20.637 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.635+0000 7f2733600640 1 --2- 172.21.15.121:0/4047829604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f272c160980 0x7f272c160e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:20.637 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.635+0000 7f2733600640 1 --2- 172.21.15.121:0/4047829604 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f272c0c9490 0x7f272c14f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:20.637 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.635+0000 7f2733600640 1 --2- 172.21.15.121:0/4047829604 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2710004d40 0x7f272c06cec0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:20.637 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.635+0000 7f2733600640 1 -- 172.21.15.121:0/4047829604 >> 172.21.15.121:0/4047829604 conn(0x7f272c0b99c0 msgr2=0x7f272c0d28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:20.637 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.636+0000 7f2733600640 1 -- 172.21.15.121:0/4047829604 shutdown_connections 2024-04-20T10:21:20.637 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.636+0000 7f2733600640 1 -- 172.21.15.121:0/4047829604 wait complete. 2024-04-20T10:21:20.655 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T10:21:20.860 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.859+0000 7f8325a00640 1 Processor -- start 2024-04-20T10:21:20.860 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.859+0000 7f8325a00640 1 -- start start 2024-04-20T10:21:20.862 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.861+0000 7f8325a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f83200d3b20 0x7f83200b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:20.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.861+0000 7f8325a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f832006c010 0x7f83200ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:20.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.861+0000 7f8325a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83200d0980 con 0x7f8320161a50 2024-04-20T10:21:20.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.861+0000 7f8325a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f83200d0c60 con 0x7f832006c010 2024-04-20T10:21:20.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.861+0000 7f8325a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f8320162ae0 con 0x7f83200d3b20 2024-04-20T10:21:20.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.861+0000 7f8325000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f832006c010 0x7f83200ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:20.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.862+0000 7f8325000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f832006c010 0x7f83200ceaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37664/0 (socket says 172.21.15.121:37664) 2024-04-20T10:21:20.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.862+0000 7f8325000640 1 -- 172.21.15.121:0/3332057122 learned_addr learned my addr 172.21.15.121:0/3332057122 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:20.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.862+0000 7f831f400640 1 --2- 172.21.15.121:0/3332057122 >> v2:172.21.15.123:3300/0 conn(0x7f83200d3b20 0x7f83200b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:20.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.862+0000 7f8325000640 1 -- 172.21.15.121:0/3332057122 >> v2:172.21.15.123:3300/0 conn(0x7f83200d3b20 msgr2=0x7f83200b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:20.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.862+0000 7f8325000640 1 --2- 172.21.15.121:0/3332057122 >> v2:172.21.15.123:3300/0 conn(0x7f83200d3b20 0x7f83200b3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:20.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.862+0000 7f8325000640 1 -- 172.21.15.121:0/3332057122 >> v1:172.21.15.123:6789/0 conn(0x7f8320161a50 legacy=0x7f832006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:20.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.862+0000 7f8325000640 1 -- 172.21.15.121:0/3332057122 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8320162490 con 0x7f832006c010 2024-04-20T10:21:20.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.862+0000 7f831f400640 1 --2- 172.21.15.121:0/3332057122 >> v2:172.21.15.123:3300/0 conn(0x7f83200d3b20 0x7f83200b3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:21:20.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.863+0000 7f8325000640 1 --2- 172.21.15.121:0/3332057122 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f832006c010 0x7f83200ceaa0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8318009870 tx=0x7f831802b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c68c6a9b428107 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:20.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.864+0000 7f831ea00640 1 -- 172.21.15.121:0/3332057122 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8318034070 con 0x7f832006c010 2024-04-20T10:21:20.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.864+0000 7f831ea00640 1 -- 172.21.15.121:0/3332057122 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8318030850 con 0x7f832006c010 2024-04-20T10:21:20.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.864+0000 7f831ea00640 1 -- 172.21.15.121:0/3332057122 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8318030c20 con 0x7f832006c010 2024-04-20T10:21:20.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.864+0000 7f8325a00640 1 -- 172.21.15.121:0/3332057122 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f832006c010 msgr2=0x7f83200ceaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:20.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.864+0000 7f8325a00640 1 --2- 172.21.15.121:0/3332057122 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f832006c010 0x7f83200ceaa0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8318009870 tx=0x7f831802b990 comp rx=0 tx=0).stop 2024-04-20T10:21:20.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.864+0000 7f8325a00640 1 -- 172.21.15.121:0/3332057122 shutdown_connections 2024-04-20T10:21:20.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.864+0000 7f8325a00640 1 --2- 172.21.15.121:0/3332057122 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f832006c010 0x7f83200ceaa0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:20.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.864+0000 7f8325a00640 1 --2- 172.21.15.121:0/3332057122 >> v2:172.21.15.123:3300/0 conn(0x7f83200d3b20 0x7f83200b3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:20.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.864+0000 7f8325a00640 1 -- 172.21.15.121:0/3332057122 >> 172.21.15.121:0/3332057122 conn(0x7f83200d0ed0 msgr2=0x7f83200d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:20.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.865+0000 7f8325a00640 1 -- 172.21.15.121:0/3332057122 shutdown_connections 2024-04-20T10:21:20.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.865+0000 7f8325a00640 1 -- 172.21.15.121:0/3332057122 wait complete. 2024-04-20T10:21:20.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.866+0000 7f8325a00640 1 Processor -- start 2024-04-20T10:21:20.867 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.866+0000 7f8325a00640 1 -- start start 2024-04-20T10:21:20.869 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.867+0000 7f8325a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f832006c010 0x7f83200caa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:20.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.867+0000 7f8325a00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8320161a50 0x7f832015fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:20.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.867+0000 7f8325a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8320160070 0x7f8320157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:20.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.867+0000 7f8325a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f83200d0600 con 0x7f8320160070 2024-04-20T10:21:20.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.867+0000 7f8325a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f83200d0480 con 0x7f832006c010 2024-04-20T10:21:20.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.867+0000 7f8325a00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f8320161e30 con 0x7f8320161a50 2024-04-20T10:21:20.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.868+0000 7f8325000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8320160070 0x7f8320157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:20.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.868+0000 7f8325000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8320160070 0x7f8320157250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37670/0 (socket says 172.21.15.121:37670) 2024-04-20T10:21:20.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.868+0000 7f8325000640 1 -- 172.21.15.121:0/124795956 learned_addr learned my addr 172.21.15.121:0/124795956 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:20.871 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.868+0000 7f831f400640 1 --2- 172.21.15.121:0/124795956 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8320161a50 0x7f832015fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:20.871 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.868+0000 7f831fe00640 1 --2- 172.21.15.121:0/124795956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f832006c010 0x7f83200caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:20.871 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.868+0000 7f8325000640 1 -- 172.21.15.121:0/124795956 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8320161a50 msgr2=0x7f832015fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:20.871 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.868+0000 7f8325000640 1 --2- 172.21.15.121:0/124795956 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8320161a50 0x7f832015fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:20.871 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.868+0000 7f8325000640 1 -- 172.21.15.121:0/124795956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f832006c010 msgr2=0x7f83200caa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:20.871 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.868+0000 7f8325000640 1 --2- 172.21.15.121:0/124795956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f832006c010 0x7f83200caa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:20.871 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.868+0000 7f8325000640 1 -- 172.21.15.121:0/124795956 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8320157f80 con 0x7f8320160070 2024-04-20T10:21:20.871 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.868+0000 7f831f400640 1 --2- 172.21.15.121:0/124795956 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8320161a50 0x7f832015fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:20.871 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.868+0000 7f831fe00640 1 --2- 172.21.15.121:0/124795956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f832006c010 0x7f83200caa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:20.871 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.868+0000 7f8325000640 1 --2- 172.21.15.121:0/124795956 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8320160070 0x7f8320157250 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8318009990 tx=0x7f8318004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:20.871 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.869+0000 7f8317e00640 1 -- 172.21.15.121:0/124795956 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8318048070 con 0x7f8320160070 2024-04-20T10:21:20.872 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.869+0000 7f8317e00640 1 -- 172.21.15.121:0/124795956 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8318030650 con 0x7f8320160070 2024-04-20T10:21:20.872 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.869+0000 7f8317e00640 1 -- 172.21.15.121:0/124795956 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8318034040 con 0x7f8320160070 2024-04-20T10:21:20.872 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.869+0000 7f8325a00640 1 -- 172.21.15.121:0/124795956 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8320158260 con 0x7f8320160070 2024-04-20T10:21:20.872 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.869+0000 7f8325a00640 1 -- 172.21.15.121:0/124795956 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83200caf40 con 0x7f8320160070 2024-04-20T10:21:20.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.870+0000 7f8325a00640 1 -- 172.21.15.121:0/124795956 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8320158780 con 0x7f8320160070 2024-04-20T10:21:20.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.871+0000 7f8317e00640 1 -- 172.21.15.121:0/124795956 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f831804b070 con 0x7f8320160070 2024-04-20T10:21:20.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.872+0000 7f8317e00640 1 --2- 172.21.15.121:0/124795956 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f82fc06c920 0x7f82fc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:20.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.872+0000 7f8317e00640 1 -- 172.21.15.121:0/124795956 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f83180ade70 con 0x7f8320160070 2024-04-20T10:21:20.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.872+0000 7f831fe00640 1 --2- 172.21.15.121:0/124795956 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f82fc06c920 0x7f82fc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:20.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.873+0000 7f831fe00640 1 --2- 172.21.15.121:0/124795956 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f82fc06c920 0x7f82fc06ede0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f83080015e0 tx=0x7f8308009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:20.876 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:20.876+0000 7f8317e00640 1 -- 172.21.15.121:0/124795956 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8320158780 con 0x7f8320160070 2024-04-20T10:21:21.062 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.061+0000 7f8325a00640 1 -- 172.21.15.121:0/124795956 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f8320162f70 con 0x7f8320160070 2024-04-20T10:21:21.063 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.062+0000 7f8317e00640 1 -- 172.21.15.121:0/124795956 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f8320162f70 con 0x7f8320160070 2024-04-20T10:21:21.063 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 32 2024-04-20T10:21:21.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.065+0000 7f8325a00640 1 -- 172.21.15.121:0/124795956 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f82fc06c920 msgr2=0x7f82fc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:21.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.065+0000 7f8325a00640 1 --2- 172.21.15.121:0/124795956 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f82fc06c920 0x7f82fc06ede0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f83080015e0 tx=0x7f8308009040 comp rx=0 tx=0).stop 2024-04-20T10:21:21.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.065+0000 7f8325a00640 1 -- 172.21.15.121:0/124795956 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8320160070 msgr2=0x7f8320157250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:21.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.065+0000 7f8325a00640 1 --2- 172.21.15.121:0/124795956 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8320160070 0x7f8320157250 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8318009990 tx=0x7f8318004730 comp rx=0 tx=0).stop 2024-04-20T10:21:21.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.065+0000 7f8325a00640 1 -- 172.21.15.121:0/124795956 shutdown_connections 2024-04-20T10:21:21.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.065+0000 7f8325a00640 1 --2- 172.21.15.121:0/124795956 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8320160070 0x7f8320157250 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:21.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.065+0000 7f8325a00640 1 --2- 172.21.15.121:0/124795956 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8320161a50 0x7f832015fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:21.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.065+0000 7f8325a00640 1 --2- 172.21.15.121:0/124795956 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f82fc06c920 0x7f82fc06ede0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:21.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.065+0000 7f8325a00640 1 --2- 172.21.15.121:0/124795956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f832006c010 0x7f83200caa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:21.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.065+0000 7f8325a00640 1 -- 172.21.15.121:0/124795956 >> 172.21.15.121:0/124795956 conn(0x7f83200d0ed0 msgr2=0x7f83200b57d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:21.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.065+0000 7f8325a00640 1 -- 172.21.15.121:0/124795956 shutdown_connections 2024-04-20T10:21:21.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.065+0000 7f8325a00640 1 -- 172.21.15.121:0/124795956 wait complete. 2024-04-20T10:21:21.082 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T10:21:21.273 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.272+0000 7f91a5400640 1 Processor -- start 2024-04-20T10:21:21.273 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.272+0000 7f91a5400640 1 -- start start 2024-04-20T10:21:21.274 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.274+0000 7f91a5400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f91a015e8a0 0x7f91a006e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:21.274 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.274+0000 7f91a5400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f91a006dbb0 0x7f91a00d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:21.274 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.274+0000 7f91a5400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91a006f060 con 0x7f91a00d08a0 2024-04-20T10:21:21.274 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.274+0000 7f91a5400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f91a006e7c0 con 0x7f91a006dbb0 2024-04-20T10:21:21.274 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.274+0000 7f91a5400640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f91a006eaa0 con 0x7f91a015e8a0 2024-04-20T10:21:21.275 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.274+0000 7f9197e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f91a006dbb0 0x7f91a00d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:21.275 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.274+0000 7f919f200640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f91a00d08a0 0x7f91a006b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:51844/0 (socket says 172.21.15.121:51844) 2024-04-20T10:21:21.276 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.274+0000 7f919f200640 1 -- 172.21.15.121:0/3320308628 learned_addr learned my addr 172.21.15.121:0/3320308628 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:21.276 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.275+0000 7f919e800640 1 --2- 172.21.15.121:0/3320308628 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f91a015e8a0 0x7f91a006e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:21.276 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.275+0000 7f9197e00640 1 -- 172.21.15.121:0/3320308628 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f91a015e8a0 msgr2=0x7f91a006e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:21.276 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.275+0000 7f9197e00640 1 --2- 172.21.15.121:0/3320308628 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f91a015e8a0 0x7f91a006e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:21.276 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.275+0000 7f9197e00640 1 -- 172.21.15.121:0/3320308628 >> v1:172.21.15.121:6789/0 conn(0x7f91a00d08a0 legacy=0x7f91a006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:21.276 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.275+0000 7f9197e00640 1 -- 172.21.15.121:0/3320308628 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a006ed80 con 0x7f91a006dbb0 2024-04-20T10:21:21.277 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.276+0000 7f9197e00640 1 --2- 172.21.15.121:0/3320308628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91a006dbb0 0x7f91a00d0360 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f91880097a0 tx=0x7f918802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d0e308e48f3648f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:21.277 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.276+0000 7f919de00640 1 -- 172.21.15.121:0/3320308628 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188035020 con 0x7f91a006dbb0 2024-04-20T10:21:21.277 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.276+0000 7f919de00640 1 -- 172.21.15.121:0/3320308628 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9188031810 con 0x7f91a006dbb0 2024-04-20T10:21:21.277 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.276+0000 7f919de00640 1 -- 172.21.15.121:0/3320308628 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188031be0 con 0x7f91a006dbb0 2024-04-20T10:21:21.278 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.277+0000 7f91a5400640 1 -- 172.21.15.121:0/3320308628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91a006dbb0 msgr2=0x7f91a00d0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:21.278 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.277+0000 7f91a5400640 1 --2- 172.21.15.121:0/3320308628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91a006dbb0 0x7f91a00d0360 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f91880097a0 tx=0x7f918802b8b0 comp rx=0 tx=0).stop 2024-04-20T10:21:21.278 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.277+0000 7f91a5400640 1 -- 172.21.15.121:0/3320308628 shutdown_connections 2024-04-20T10:21:21.278 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.277+0000 7f91a5400640 1 --2- 172.21.15.121:0/3320308628 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91a006dbb0 0x7f91a00d0360 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:21.278 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.277+0000 7f91a5400640 1 --2- 172.21.15.121:0/3320308628 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f91a015e8a0 0x7f91a006e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:21.278 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.277+0000 7f91a5400640 1 -- 172.21.15.121:0/3320308628 >> 172.21.15.121:0/3320308628 conn(0x7f91a00b3350 msgr2=0x7f91a00b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:21.278 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.277+0000 7f91a5400640 1 -- 172.21.15.121:0/3320308628 shutdown_connections 2024-04-20T10:21:21.278 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.277+0000 7f91a5400640 1 -- 172.21.15.121:0/3320308628 wait complete. 2024-04-20T10:21:21.279 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.279+0000 7f91a5400640 1 Processor -- start 2024-04-20T10:21:21.279 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.279+0000 7f91a5400640 1 -- start start 2024-04-20T10:21:21.281 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.280+0000 7f91a5400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91a006dbb0 0x7f91a0155950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:21.281 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.280+0000 7f91a5400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91a00d08a0 0x7f91a01611e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:21.281 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.280+0000 7f91a5400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f91a015e8a0 0x7f91a0161720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:21.281 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.280+0000 7f91a5400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f91a006fef0 con 0x7f91a006dbb0 2024-04-20T10:21:21.281 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.280+0000 7f91a5400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f91a006fd70 con 0x7f91a00d08a0 2024-04-20T10:21:21.282 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.280+0000 7f91a5400640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f91a00ae690 con 0x7f91a015e8a0 2024-04-20T10:21:21.282 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.280+0000 7f919e800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91a006dbb0 0x7f91a0155950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:21.282 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.280+0000 7f919e800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91a006dbb0 0x7f91a0155950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37698/0 (socket says 172.21.15.121:37698) 2024-04-20T10:21:21.282 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.280+0000 7f919e800640 1 -- 172.21.15.121:0/3440367901 learned_addr learned my addr 172.21.15.121:0/3440367901 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:21.282 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.281+0000 7f919f200640 1 --2- 172.21.15.121:0/3440367901 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f91a015e8a0 0x7f91a0161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:21.282 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.281+0000 7f919e800640 1 -- 172.21.15.121:0/3440367901 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f91a015e8a0 msgr2=0x7f91a0161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:21.282 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.281+0000 7f919e800640 1 --2- 172.21.15.121:0/3440367901 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f91a015e8a0 0x7f91a0161720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:21.282 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.281+0000 7f9197e00640 1 --2- 172.21.15.121:0/3440367901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91a00d08a0 0x7f91a01611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:21.282 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.281+0000 7f919e800640 1 -- 172.21.15.121:0/3440367901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91a00d08a0 msgr2=0x7f91a01611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:21.282 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.281+0000 7f919e800640 1 --2- 172.21.15.121:0/3440367901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91a00d08a0 0x7f91a01611e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:21.282 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.281+0000 7f919e800640 1 -- 172.21.15.121:0/3440367901 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a01544b0 con 0x7f91a006dbb0 2024-04-20T10:21:21.282 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.281+0000 7f919e800640 1 --2- 172.21.15.121:0/3440367901 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91a006dbb0 0x7f91a0155950 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f919000d620 tx=0x7f919000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:21.283 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.281+0000 7f9196a00640 1 -- 172.21.15.121:0/3440367901 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919000b620 con 0x7f91a006dbb0 2024-04-20T10:21:21.283 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.281+0000 7f9196a00640 1 -- 172.21.15.121:0/3440367901 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9190019070 con 0x7f91a006dbb0 2024-04-20T10:21:21.283 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.281+0000 7f91a5400640 1 -- 172.21.15.121:0/3440367901 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91a0155f50 con 0x7f91a006dbb0 2024-04-20T10:21:21.283 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.281+0000 7f9196a00640 1 -- 172.21.15.121:0/3440367901 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919000ba00 con 0x7f91a006dbb0 2024-04-20T10:21:21.284 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.281+0000 7f91a5400640 1 -- 172.21.15.121:0/3440367901 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91a0156510 con 0x7f91a006dbb0 2024-04-20T10:21:21.284 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.282+0000 7f91a5400640 1 -- 172.21.15.121:0/3440367901 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91a0068980 con 0x7f91a006dbb0 2024-04-20T10:21:21.284 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.283+0000 7f9196a00640 1 -- 172.21.15.121:0/3440367901 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f919000bba0 con 0x7f91a006dbb0 2024-04-20T10:21:21.284 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.284+0000 7f9196a00640 1 --2- 172.21.15.121:0/3440367901 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f917c06c920 0x7f917c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:21.285 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.284+0000 7f9196a00640 1 -- 172.21.15.121:0/3440367901 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f919008c2f0 con 0x7f91a006dbb0 2024-04-20T10:21:21.285 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.285+0000 7f9197e00640 1 --2- 172.21.15.121:0/3440367901 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f917c06c920 0x7f917c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:21.285 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.285+0000 7f9197e00640 1 --2- 172.21.15.121:0/3440367901 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f917c06c920 0x7f917c06ede0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9188036000 tx=0x7f9188008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:21.289 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.289+0000 7f9196a00640 1 -- 172.21.15.121:0/3440367901 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f919005a340 con 0x7f91a006dbb0 2024-04-20T10:21:21.501 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.501+0000 7f91a5400640 1 -- 172.21.15.121:0/3440367901 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f91a006d160 con 0x7f91a006dbb0 2024-04-20T10:21:21.502 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.502+0000 7f9196a00640 1 -- 172.21.15.121:0/3440367901 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7f9190059c40 con 0x7f91a006dbb0 2024-04-20T10:21:21.502 INFO:teuthology.orchestra.run.smithi121.stderr:listed 0 entries 2024-04-20T10:21:21.505 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.504+0000 7f91a5400640 1 -- 172.21.15.121:0/3440367901 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f917c06c920 msgr2=0x7f917c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:21.505 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.504+0000 7f91a5400640 1 --2- 172.21.15.121:0/3440367901 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f917c06c920 0x7f917c06ede0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9188036000 tx=0x7f9188008030 comp rx=0 tx=0).stop 2024-04-20T10:21:21.505 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.504+0000 7f91a5400640 1 -- 172.21.15.121:0/3440367901 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91a006dbb0 msgr2=0x7f91a0155950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:21.505 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.504+0000 7f91a5400640 1 --2- 172.21.15.121:0/3440367901 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91a006dbb0 0x7f91a0155950 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f919000d620 tx=0x7f919000dae0 comp rx=0 tx=0).stop 2024-04-20T10:21:21.505 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.504+0000 7f91a5400640 1 -- 172.21.15.121:0/3440367901 shutdown_connections 2024-04-20T10:21:21.505 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.504+0000 7f91a5400640 1 --2- 172.21.15.121:0/3440367901 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f91a015e8a0 0x7f91a0161720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:21.505 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.504+0000 7f91a5400640 1 --2- 172.21.15.121:0/3440367901 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f917c06c920 0x7f917c06ede0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:21.505 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.504+0000 7f91a5400640 1 --2- 172.21.15.121:0/3440367901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91a00d08a0 0x7f91a01611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:21.505 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.504+0000 7f91a5400640 1 --2- 172.21.15.121:0/3440367901 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91a006dbb0 0x7f91a0155950 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:21.505 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.504+0000 7f91a5400640 1 -- 172.21.15.121:0/3440367901 >> 172.21.15.121:0/3440367901 conn(0x7f91a00b3350 msgr2=0x7f91a015cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:21.506 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.504+0000 7f91a5400640 1 -- 172.21.15.121:0/3440367901 shutdown_connections 2024-04-20T10:21:21.506 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:21.504+0000 7f91a5400640 1 -- 172.21.15.121:0/3440367901 wait complete. 2024-04-20T10:21:21.523 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi121.front.sepia.ceph.com... 2024-04-20T10:21:21.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:21.523 DEBUG:teuthology.orchestra.run.smithi121:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-20T10:21:21.603 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi121.stderr:ceph-fuse[29669]: fuse finished with error 0 and tester_r 0 2024-04-20T10:21:21.609 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:21:25.304 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:21:25.311 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:21:27.710 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2024-04-20T10:21:27.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:27.711 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-20T10:21:27.782 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:21:27.988 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.988+0000 7f9ed8a00640 1 Processor -- start 2024-04-20T10:21:27.988 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.988+0000 7f9ed8a00640 1 -- start start 2024-04-20T10:21:27.990 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.990+0000 7f9ed8a00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ed4159e10 0x7f9ed4071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:27.990 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.990+0000 7f9ed8a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9ed415a1f0 0x7f9ed4071fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:27.990 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.990+0000 7f9ed8a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ed415cc90 con 0x7f9ed406da90 2024-04-20T10:21:27.990 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.990+0000 7f9ed8a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f9ed4072900 con 0x7f9ed415a1f0 2024-04-20T10:21:27.990 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.990+0000 7f9ed8a00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f9ed415ca40 con 0x7f9ed4159e10 2024-04-20T10:21:27.991 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.990+0000 7f9ecaa00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9ed415a1f0 0x7f9ed4071fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:27.991 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.990+0000 7f9ecaa00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9ed415a1f0 0x7f9ed4071fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54054/0 (socket says 172.21.15.121:54054) 2024-04-20T10:21:27.991 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.990+0000 7f9ecaa00640 1 -- 172.21.15.121:0/1504994377 learned_addr learned my addr 172.21.15.121:0/1504994377 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:27.991 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.991+0000 7f9ed3400640 1 --2- 172.21.15.121:0/1504994377 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ed4159e10 0x7f9ed4071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:27.991 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.991+0000 7f9ecaa00640 1 -- 172.21.15.121:0/1504994377 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ed4159e10 msgr2=0x7f9ed4071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:27.991 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.991+0000 7f9ecaa00640 1 --2- 172.21.15.121:0/1504994377 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ed4159e10 0x7f9ed4071a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:27.991 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.991+0000 7f9ecaa00640 1 -- 172.21.15.121:0/1504994377 >> v1:172.21.15.123:6789/0 conn(0x7f9ed406da90 legacy=0x7f9ed4160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:27.991 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.991+0000 7f9ecaa00640 1 -- 172.21.15.121:0/1504994377 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ed415cf70 con 0x7f9ed415a1f0 2024-04-20T10:21:27.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.992+0000 7f9ecaa00640 1 --2- 172.21.15.121:0/1504994377 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ed415a1f0 0x7f9ed4071fc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9ed415cbc0 tx=0x7f9ebc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a1fce0ad3a45ed server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:27.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.992+0000 7f9ed2a00640 1 -- 172.21.15.121:0/1504994377 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc035020 con 0x7f9ed415a1f0 2024-04-20T10:21:27.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.992+0000 7f9ed2a00640 1 -- 172.21.15.121:0/1504994377 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ebc031640 con 0x7f9ed415a1f0 2024-04-20T10:21:27.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.992+0000 7f9ed2a00640 1 -- 172.21.15.121:0/1504994377 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc02d040 con 0x7f9ed415a1f0 2024-04-20T10:21:27.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.993+0000 7f9ed8a00640 1 -- 172.21.15.121:0/1504994377 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ed415a1f0 msgr2=0x7f9ed4071fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:27.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.993+0000 7f9ed8a00640 1 --2- 172.21.15.121:0/1504994377 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ed415a1f0 0x7f9ed4071fc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9ed415cbc0 tx=0x7f9ebc02b8f0 comp rx=0 tx=0).stop 2024-04-20T10:21:27.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.993+0000 7f9ed8a00640 1 -- 172.21.15.121:0/1504994377 shutdown_connections 2024-04-20T10:21:27.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.993+0000 7f9ed8a00640 1 --2- 172.21.15.121:0/1504994377 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ed415a1f0 0x7f9ed4071fc0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:27.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.993+0000 7f9ed8a00640 1 --2- 172.21.15.121:0/1504994377 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ed4159e10 0x7f9ed4071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:27.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.993+0000 7f9ed8a00640 1 -- 172.21.15.121:0/1504994377 >> 172.21.15.121:0/1504994377 conn(0x7f9ed40d0040 msgr2=0x7f9ed40d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:27.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.993+0000 7f9ed8a00640 1 -- 172.21.15.121:0/1504994377 shutdown_connections 2024-04-20T10:21:27.994 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.993+0000 7f9ed8a00640 1 -- 172.21.15.121:0/1504994377 wait complete. 2024-04-20T10:21:27.994 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.995+0000 7f9ed8a00640 1 Processor -- start 2024-04-20T10:21:27.994 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.995+0000 7f9ed8a00640 1 -- start start 2024-04-20T10:21:27.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.996+0000 7f9ed8a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ed406da90 0x7f9ed41f2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:27.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.996+0000 7f9ed8a00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ed4159e10 0x7f9ed415b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:27.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.996+0000 7f9ed8a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ed415a1f0 0x7f9ed409b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:27.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.996+0000 7f9ed8a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed415d810 con 0x7f9ed406da90 2024-04-20T10:21:27.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.996+0000 7f9ed8a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed415d530 con 0x7f9ed415a1f0 2024-04-20T10:21:27.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.996+0000 7f9ed8a00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f9ed406b900 con 0x7f9ed4159e10 2024-04-20T10:21:27.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.996+0000 7f9ed3400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ed406da90 0x7f9ed41f2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:27.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.996+0000 7f9ed3400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ed406da90 0x7f9ed41f2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54064/0 (socket says 172.21.15.121:54064) 2024-04-20T10:21:27.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.996+0000 7f9ed3400640 1 -- 172.21.15.121:0/3438478604 learned_addr learned my addr 172.21.15.121:0/3438478604 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:27.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.997+0000 7f9ecaa00640 1 --2- 172.21.15.121:0/3438478604 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ed4159e10 0x7f9ed415b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:27.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.997+0000 7f9ed3e00640 1 --2- 172.21.15.121:0/3438478604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ed415a1f0 0x7f9ed409b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:27.998 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.997+0000 7f9ed3400640 1 -- 172.21.15.121:0/3438478604 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ed4159e10 msgr2=0x7f9ed415b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:27.998 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.997+0000 7f9ed3400640 1 --2- 172.21.15.121:0/3438478604 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ed4159e10 0x7f9ed415b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:27.998 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.997+0000 7f9ed3400640 1 -- 172.21.15.121:0/3438478604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ed415a1f0 msgr2=0x7f9ed409b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:27.998 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.997+0000 7f9ed3400640 1 --2- 172.21.15.121:0/3438478604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ed415a1f0 0x7f9ed409b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:27.998 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.997+0000 7f9ed3400640 1 -- 172.21.15.121:0/3438478604 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ed415a5d0 con 0x7f9ed406da90 2024-04-20T10:21:27.998 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.997+0000 7f9ed3e00640 1 --2- 172.21.15.121:0/3438478604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ed415a1f0 0x7f9ed409b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:27.998 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.997+0000 7f9ed3400640 1 --2- 172.21.15.121:0/3438478604 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ed406da90 0x7f9ed41f2a30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4007ef0 tx=0x7f9ec400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:27.999 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.997+0000 7f9ecbe00640 1 -- 172.21.15.121:0/3438478604 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4014620 con 0x7f9ed406da90 2024-04-20T10:21:27.999 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.998+0000 7f9ecbe00640 1 -- 172.21.15.121:0/3438478604 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ec4019070 con 0x7f9ed406da90 2024-04-20T10:21:28.000 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.998+0000 7f9ecbe00640 1 -- 172.21.15.121:0/3438478604 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4014a00 con 0x7f9ed406da90 2024-04-20T10:21:28.000 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.998+0000 7f9ed8a00640 1 -- 172.21.15.121:0/3438478604 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ed41f2f70 con 0x7f9ed406da90 2024-04-20T10:21:28.000 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.998+0000 7f9ed8a00640 1 -- 172.21.15.121:0/3438478604 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ed415ab50 con 0x7f9ed406da90 2024-04-20T10:21:28.000 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:27.999+0000 7f9ed8a00640 1 -- 172.21.15.121:0/3438478604 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ed4075b60 con 0x7f9ed406da90 2024-04-20T10:21:28.000 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.000+0000 7f9ecbe00640 1 -- 172.21.15.121:0/3438478604 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9ec4014ba0 con 0x7f9ed406da90 2024-04-20T10:21:28.001 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.001+0000 7f9ecbe00640 1 --2- 172.21.15.121:0/3438478604 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9eb006c8f0 0x7f9eb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:28.001 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.001+0000 7f9ecbe00640 1 -- 172.21.15.121:0/3438478604 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9ec408c8a0 con 0x7f9ed406da90 2024-04-20T10:21:28.001 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.001+0000 7f9ecaa00640 1 --2- 172.21.15.121:0/3438478604 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9eb006c8f0 0x7f9eb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:28.001 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.002+0000 7f9ecaa00640 1 --2- 172.21.15.121:0/3438478604 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9eb006c8f0 0x7f9eb006edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc036000 tx=0x7f9ebc03c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:28.004 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.005+0000 7f9ecbe00640 1 -- 172.21.15.121:0/3438478604 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9ec405a8f0 con 0x7f9ed406da90 2024-04-20T10:21:28.211 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.211+0000 7f9ed8a00640 1 -- 172.21.15.121:0/3438478604 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9ed415be50 con 0x7f9ed406da90 2024-04-20T10:21:28.214 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.215+0000 7f9ecbe00640 1 -- 172.21.15.121:0/3438478604 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7f9ec405a1f0 con 0x7f9ed406da90 2024-04-20T10:21:28.215 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:21:28.215 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":25,"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","created":"2024-04-20T10:20:38.076076+0000","modified":"2024-04-20T10:21:07.336542+0000","last_up_change":"2024-04-20T10:20:50.162700+0000","last_in_change":"2024-04-20T10:20:41.510382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:20:51.141365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:21:00.436937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:21:01.026957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:21:04.862693+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"629a7b8c-a40a-4a2a-a3cc-d9f357774310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6809","nonce":1044588835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6811","nonce":1044588835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6815","nonce":1044588835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6813","nonce":1044588835}]},"public_addr":"172.21.15.121:6809/1044588835","cluster_addr":"172.21.15.121:6811/1044588835","heartbeat_back_addr":"172.21.15.121:6815/1044588835","heartbeat_front_addr":"172.21.15.121:6813/1044588835","state":["exists","up"]},{"osd":1,"uuid":"f2570399-aa50-4ff3-bf5d-3d09207290db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6801","nonce":3505199624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6803","nonce":3505199624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6807","nonce":3505199624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6805","nonce":3505199624}]},"public_addr":"172.21.15.121:6801/3505199624","cluster_addr":"172.21.15.121:6803/3505199624","heartbeat_back_addr":"172.21.15.121:6807/3505199624","heartbeat_front_addr":"172.21.15.121:6805/3505199624","state":["exists","up"]},{"osd":2,"uuid":"50893807-dc30-49f4-b41f-739366626658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6817","nonce":3656724207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6819","nonce":3656724207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6827","nonce":3656724207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6822","nonce":3656724207}]},"public_addr":"172.21.15.121:6817/3656724207","cluster_addr":"172.21.15.121:6819/3656724207","heartbeat_back_addr":"172.21.15.121:6827/3656724207","heartbeat_front_addr":"172.21.15.121:6822/3656724207","state":["exists","up"]},{"osd":3,"uuid":"dadb01bc-5491-4ffc-8824-d48a057240b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6823","nonce":2232342832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6826","nonce":2232342832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6831","nonce":2232342832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6829","nonce":2232342832}]},"public_addr":"172.21.15.121:6823/2232342832","cluster_addr":"172.21.15.121:6826/2232342832","heartbeat_back_addr":"172.21.15.121:6831/2232342832","heartbeat_front_addr":"172.21.15.121:6829/2232342832","state":["exists","up"]},{"osd":4,"uuid":"b3f3b4a1-b375-4526-b69e-344ab24c2da1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6801","nonce":99353111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6803","nonce":99353111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6807","nonce":99353111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6805","nonce":99353111}]},"public_addr":"172.21.15.123:6801/99353111","cluster_addr":"172.21.15.123:6803/99353111","heartbeat_back_addr":"172.21.15.123:6807/99353111","heartbeat_front_addr":"172.21.15.123:6805/99353111","state":["exists","up"]},{"osd":5,"uuid":"2ee24a08-1624-4961-9642-d7b510282484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6818","nonce":2504374688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6822","nonce":2504374688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6830","nonce":2504374688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6826","nonce":2504374688}]},"public_addr":"172.21.15.123:6818/2504374688","cluster_addr":"172.21.15.123:6822/2504374688","heartbeat_back_addr":"172.21.15.123:6830/2504374688","heartbeat_front_addr":"172.21.15.123:6826/2504374688","state":["exists","up"]},{"osd":6,"uuid":"186118cc-6128-49de-acca-55260945c1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6819","nonce":4139299599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6823","nonce":4139299599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6829","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6831","nonce":4139299599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6827","nonce":4139299599}]},"public_addr":"172.21.15.123:6819/4139299599","cluster_addr":"172.21.15.123:6823/4139299599","heartbeat_back_addr":"172.21.15.123:6831/4139299599","heartbeat_front_addr":"172.21.15.123:6827/4139299599","state":["exists","up"]},{"osd":7,"uuid":"a00aac6e-4bd9-415a-a93d-ae85210865ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6809","nonce":4043981069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6811","nonce":4043981069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6815","nonce":4043981069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6813","nonce":4043981069}]},"public_addr":"172.21.15.123:6809/4043981069","cluster_addr":"172.21.15.123:6811/4043981069","heartbeat_back_addr":"172.21.15.123:6815/4043981069","heartbeat_front_addr":"172.21.15.123:6813/4043981069","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.904540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.958585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.616367+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.885244+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.103127+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.167248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.884348+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.862356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:21:28.218 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.218+0000 7f9ed8a00640 1 -- 172.21.15.121:0/3438478604 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9eb006c8f0 msgr2=0x7f9eb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:28.218 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.218+0000 7f9ed8a00640 1 --2- 172.21.15.121:0/3438478604 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9eb006c8f0 0x7f9eb006edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc036000 tx=0x7f9ebc03c040 comp rx=0 tx=0).stop 2024-04-20T10:21:28.218 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.218+0000 7f9ed8a00640 1 -- 172.21.15.121:0/3438478604 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ed406da90 msgr2=0x7f9ed41f2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:28.218 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.218+0000 7f9ed8a00640 1 --2- 172.21.15.121:0/3438478604 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ed406da90 0x7f9ed41f2a30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9ec4007ef0 tx=0x7f9ec400e4b0 comp rx=0 tx=0).stop 2024-04-20T10:21:28.218 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.218+0000 7f9ed8a00640 1 -- 172.21.15.121:0/3438478604 shutdown_connections 2024-04-20T10:21:28.218 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.218+0000 7f9ed8a00640 1 --2- 172.21.15.121:0/3438478604 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9eb006c8f0 0x7f9eb006edb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:28.218 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.218+0000 7f9ed8a00640 1 --2- 172.21.15.121:0/3438478604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ed415a1f0 0x7f9ed409b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:28.218 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.218+0000 7f9ed8a00640 1 --2- 172.21.15.121:0/3438478604 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9ed4159e10 0x7f9ed415b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:28.218 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.218+0000 7f9ed8a00640 1 --2- 172.21.15.121:0/3438478604 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9ed406da90 0x7f9ed41f2a30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:28.219 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.218+0000 7f9ed8a00640 1 -- 172.21.15.121:0/3438478604 >> 172.21.15.121:0/3438478604 conn(0x7f9ed40d0040 msgr2=0x7f9ed40676c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:28.219 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.218+0000 7f9ed8a00640 1 -- 172.21.15.121:0/3438478604 shutdown_connections 2024-04-20T10:21:28.219 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.219+0000 7f9ed8a00640 1 -- 172.21.15.121:0/3438478604 wait complete. 2024-04-20T10:21:28.240 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:21:28.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.438+0000 7f6282c00640 1 Processor -- start 2024-04-20T10:21:28.438 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.438+0000 7f6282c00640 1 -- start start 2024-04-20T10:21:28.439 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.440+0000 7f6282c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f627c15b7e0 0x7f627c15bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:28.439 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.440+0000 7f6282c00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f627c06ccc0 con 0x7f627c15c190 2024-04-20T10:21:28.440 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.440+0000 7f6282c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f627c06cfa0 con 0x7f627c15b400 2024-04-20T10:21:28.440 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.440+0000 7f6282c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f627c06d280 con 0x7f627c15b7e0 2024-04-20T10:21:28.440 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.440+0000 7f6282200640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f627c15c190 0x7f627c06ec40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:49944/0 (socket says 172.21.15.121:49944) 2024-04-20T10:21:28.440 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.440+0000 7f6282200640 1 -- 172.21.15.121:0/3618786403 learned_addr learned my addr 172.21.15.121:0/3618786403 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:28.443 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.440+0000 7f6280e00640 1 --2- 172.21.15.121:0/3618786403 >> v2:172.21.15.123:3300/0 conn(0x7f627c15b7e0 0x7f627c15bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:28.443 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.441+0000 7f627be00640 1 -- 172.21.15.121:0/3618786403 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2163763553 0 0) 0x7f627c06ccc0 con 0x7f627c15c190 2024-04-20T10:21:28.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.441+0000 7f627be00640 1 -- 172.21.15.121:0/3618786403 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6264005730 con 0x7f627c15c190 2024-04-20T10:21:28.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.441+0000 7f627be00640 1 -- 172.21.15.121:0/3618786403 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 489468479 0 0) 0x7f6264005730 con 0x7f627c15c190 2024-04-20T10:21:28.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.441+0000 7f627be00640 1 -- 172.21.15.121:0/3618786403 >> v2:172.21.15.123:3300/0 conn(0x7f627c15b7e0 msgr2=0x7f627c15bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:28.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.441+0000 7f627be00640 1 --2- 172.21.15.121:0/3618786403 >> v2:172.21.15.123:3300/0 conn(0x7f627c15b7e0 0x7f627c15bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:28.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.441+0000 7f627be00640 1 -- 172.21.15.121:0/3618786403 >> v1:172.21.15.123:6789/0 conn(0x7f627c15b400 legacy=0x7f627c072d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:28.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.441+0000 7f627be00640 1 -- 172.21.15.121:0/3618786403 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f627c06e440 con 0x7f627c15c190 2024-04-20T10:21:28.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.441+0000 7f627be00640 1 -- 172.21.15.121:0/3618786403 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f62740019f0 con 0x7f627c15c190 2024-04-20T10:21:28.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.441+0000 7f627be00640 1 -- 172.21.15.121:0/3618786403 >> v1:172.21.15.121:6789/0 conn(0x7f627c15c190 legacy=0x7f627c06ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:28.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.441+0000 7f627be00640 1 --2- 172.21.15.121:0/3618786403 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6264005bc0 0x7f6264005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:28.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.441+0000 7f627be00640 1 --2- 172.21.15.121:0/3618786403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6264006a10 0x7f6264006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:28.445 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.441+0000 7f627be00640 1 --2- 172.21.15.121:0/3618786403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62640073d0 0x7f626400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:28.445 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.441+0000 7f627be00640 1 -- 172.21.15.121:0/3618786403 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f627c06d280 con 0x7f6264006a10 2024-04-20T10:21:28.445 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.441+0000 7f627be00640 1 -- 172.21.15.121:0/3618786403 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f626400e270 con 0x7f62640073d0 2024-04-20T10:21:28.445 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.441+0000 7f627be00640 1 -- 172.21.15.121:0/3618786403 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f626400e470 con 0x7f6264005bc0 2024-04-20T10:21:28.445 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.441+0000 7f6280e00640 1 --2- 172.21.15.121:0/3618786403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6264006a10 0x7f6264006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:28.445 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.442+0000 7f6281800640 1 --2- 172.21.15.121:0/3618786403 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6264005bc0 0x7f6264005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:28.445 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.442+0000 7f6282200640 1 --2- 172.21.15.121:0/3618786403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62640073d0 0x7f626400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:28.445 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.442+0000 7f6280e00640 1 -- 172.21.15.121:0/3618786403 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6264005bc0 msgr2=0x7f6264005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:28.445 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.442+0000 7f6280e00640 1 --2- 172.21.15.121:0/3618786403 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6264005bc0 0x7f6264005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:28.445 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.442+0000 7f6280e00640 1 -- 172.21.15.121:0/3618786403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62640073d0 msgr2=0x7f626400dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:28.445 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.442+0000 7f6280e00640 1 --2- 172.21.15.121:0/3618786403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62640073d0 0x7f626400dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:28.446 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.442+0000 7f6280e00640 1 -- 172.21.15.121:0/3618786403 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6264005730 con 0x7f6264006a10 2024-04-20T10:21:28.446 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.442+0000 7f6281800640 1 --2- 172.21.15.121:0/3618786403 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6264005bc0 0x7f6264005fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:28.446 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.442+0000 7f6282200640 1 --2- 172.21.15.121:0/3618786403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62640073d0 0x7f626400dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:28.446 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.442+0000 7f6282200640 1 -- 172.21.15.121:0/3618786403 reap_dead start 2024-04-20T10:21:28.446 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.443+0000 7f6280e00640 1 --2- 172.21.15.121:0/3618786403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6264006a10 0x7f6264006e90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6268009ac0 tx=0x7f6264007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=768fc071d509b8d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:28.446 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.443+0000 7f627be00640 1 -- 172.21.15.121:0/3618786403 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6268039070 con 0x7f6264006a10 2024-04-20T10:21:28.446 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.443+0000 7f627be00640 1 -- 172.21.15.121:0/3618786403 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f626802c6d0 con 0x7f6264006a10 2024-04-20T10:21:28.446 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.443+0000 7f627be00640 1 -- 172.21.15.121:0/3618786403 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6268009690 con 0x7f6264006a10 2024-04-20T10:21:28.446 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.444+0000 7f6282c00640 1 -- 172.21.15.121:0/3618786403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6264006a10 msgr2=0x7f6264006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:28.446 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.444+0000 7f6282c00640 1 --2- 172.21.15.121:0/3618786403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6264006a10 0x7f6264006e90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6268009ac0 tx=0x7f6264007a40 comp rx=0 tx=0).stop 2024-04-20T10:21:28.446 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.444+0000 7f6282c00640 1 -- 172.21.15.121:0/3618786403 shutdown_connections 2024-04-20T10:21:28.447 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.444+0000 7f6282c00640 1 --2- 172.21.15.121:0/3618786403 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6264006a10 0x7f6264006e90 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:28.447 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.444+0000 7f6282c00640 1 -- 172.21.15.121:0/3618786403 >> 172.21.15.121:0/3618786403 conn(0x7f627c0d0040 msgr2=0x7f627c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:28.447 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.444+0000 7f6282c00640 1 -- 172.21.15.121:0/3618786403 shutdown_connections 2024-04-20T10:21:28.447 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.444+0000 7f6282c00640 1 -- 172.21.15.121:0/3618786403 wait complete. 2024-04-20T10:21:28.449 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.446+0000 7f6282c00640 1 Processor -- start 2024-04-20T10:21:28.449 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.446+0000 7f6282c00640 1 -- start start 2024-04-20T10:21:28.449 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.447+0000 7f6282c00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6264006a10 0x7f627c09b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:28.450 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.447+0000 7f6282c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f627c09ad00 0x7f627c0ce570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:28.450 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.447+0000 7f6282c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f627c0ceab0 0x7f627c14f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:28.450 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.447+0000 7f6282c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f627c15c770 con 0x7f627c0ceab0 2024-04-20T10:21:28.450 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.447+0000 7f6282c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f627c15c5f0 con 0x7f627c09ad00 2024-04-20T10:21:28.450 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.447+0000 7f6282c00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f627c15bc50 con 0x7f6264006a10 2024-04-20T10:21:28.450 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.447+0000 7f6282200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f627c0ceab0 0x7f627c14f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:28.450 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.447+0000 7f6282200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f627c0ceab0 0x7f627c14f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54094/0 (socket says 172.21.15.121:54094) 2024-04-20T10:21:28.450 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.447+0000 7f6282200640 1 -- 172.21.15.121:0/3135730777 learned_addr learned my addr 172.21.15.121:0/3135730777 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:28.450 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.447+0000 7f6281800640 1 --2- 172.21.15.121:0/3135730777 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6264006a10 0x7f627c09b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:28.450 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.447+0000 7f6280e00640 1 --2- 172.21.15.121:0/3135730777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f627c09ad00 0x7f627c0ce570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:28.450 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.447+0000 7f6282200640 1 -- 172.21.15.121:0/3135730777 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6264006a10 msgr2=0x7f627c09b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:28.451 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.447+0000 7f6282200640 1 --2- 172.21.15.121:0/3135730777 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6264006a10 0x7f627c09b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:28.451 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.447+0000 7f6282200640 1 -- 172.21.15.121:0/3135730777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f627c09ad00 msgr2=0x7f627c0ce570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:28.451 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.447+0000 7f6282200640 1 --2- 172.21.15.121:0/3135730777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f627c09ad00 0x7f627c0ce570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:28.451 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.447+0000 7f6282200640 1 -- 172.21.15.121:0/3135730777 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f627c0cf280 con 0x7f627c0ceab0 2024-04-20T10:21:28.451 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.447+0000 7f6280e00640 1 --2- 172.21.15.121:0/3135730777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f627c09ad00 0x7f627c0ce570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:28.451 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.448+0000 7f6282200640 1 --2- 172.21.15.121:0/3135730777 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f627c0ceab0 0x7f627c14f590 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6274000f80 tx=0x7f627400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:28.451 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.448+0000 7f6279600640 1 -- 172.21.15.121:0/3135730777 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6274004d60 con 0x7f627c0ceab0 2024-04-20T10:21:28.451 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.448+0000 7f6279600640 1 -- 172.21.15.121:0/3135730777 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f627400bb80 con 0x7f627c0ceab0 2024-04-20T10:21:28.451 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.448+0000 7f6282c00640 1 -- 172.21.15.121:0/3135730777 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f627c0cf560 con 0x7f627c0ceab0 2024-04-20T10:21:28.451 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.448+0000 7f6279600640 1 -- 172.21.15.121:0/3135730777 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6274007110 con 0x7f627c0ceab0 2024-04-20T10:21:28.452 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.448+0000 7f6282c00640 1 -- 172.21.15.121:0/3135730777 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f627c15dcc0 con 0x7f627c0ceab0 2024-04-20T10:21:28.452 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.449+0000 7f6282c00640 1 -- 172.21.15.121:0/3135730777 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f627c075b60 con 0x7f627c0ceab0 2024-04-20T10:21:28.452 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.450+0000 7f6279600640 1 -- 172.21.15.121:0/3135730777 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f627401b020 con 0x7f627c0ceab0 2024-04-20T10:21:28.453 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.451+0000 7f6279600640 1 --2- 172.21.15.121:0/3135730777 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f625c06c8f0 0x7f625c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:28.453 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.451+0000 7f6279600640 1 -- 172.21.15.121:0/3135730777 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f627408c130 con 0x7f627c0ceab0 2024-04-20T10:21:28.453 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.451+0000 7f6281800640 1 --2- 172.21.15.121:0/3135730777 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f625c06c8f0 0x7f625c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:28.453 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.452+0000 7f6281800640 1 --2- 172.21.15.121:0/3135730777 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f625c06c8f0 0x7f625c06edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f627c069030 tx=0x7f6270009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:28.455 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.455+0000 7f6279600640 1 -- 172.21.15.121:0/3135730777 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f627405a180 con 0x7f627c0ceab0 2024-04-20T10:21:28.656 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.656+0000 7f6282c00640 1 -- 172.21.15.121:0/3135730777 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f627c0b5220 con 0x7f627c0ceab0 2024-04-20T10:21:28.657 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.658+0000 7f6279600640 1 -- 172.21.15.121:0/3135730777 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7f6274059a80 con 0x7f627c0ceab0 2024-04-20T10:21:28.658 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:21:28.658 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":25,"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","created":"2024-04-20T10:20:38.076076+0000","modified":"2024-04-20T10:21:07.336542+0000","last_up_change":"2024-04-20T10:20:50.162700+0000","last_in_change":"2024-04-20T10:20:41.510382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:20:51.141365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:21:00.436937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:21:01.026957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:21:04.862693+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"629a7b8c-a40a-4a2a-a3cc-d9f357774310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6809","nonce":1044588835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6811","nonce":1044588835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6815","nonce":1044588835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6813","nonce":1044588835}]},"public_addr":"172.21.15.121:6809/1044588835","cluster_addr":"172.21.15.121:6811/1044588835","heartbeat_back_addr":"172.21.15.121:6815/1044588835","heartbeat_front_addr":"172.21.15.121:6813/1044588835","state":["exists","up"]},{"osd":1,"uuid":"f2570399-aa50-4ff3-bf5d-3d09207290db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6801","nonce":3505199624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6803","nonce":3505199624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6807","nonce":3505199624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6805","nonce":3505199624}]},"public_addr":"172.21.15.121:6801/3505199624","cluster_addr":"172.21.15.121:6803/3505199624","heartbeat_back_addr":"172.21.15.121:6807/3505199624","heartbeat_front_addr":"172.21.15.121:6805/3505199624","state":["exists","up"]},{"osd":2,"uuid":"50893807-dc30-49f4-b41f-739366626658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6817","nonce":3656724207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6819","nonce":3656724207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6827","nonce":3656724207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6822","nonce":3656724207}]},"public_addr":"172.21.15.121:6817/3656724207","cluster_addr":"172.21.15.121:6819/3656724207","heartbeat_back_addr":"172.21.15.121:6827/3656724207","heartbeat_front_addr":"172.21.15.121:6822/3656724207","state":["exists","up"]},{"osd":3,"uuid":"dadb01bc-5491-4ffc-8824-d48a057240b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6823","nonce":2232342832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6826","nonce":2232342832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6831","nonce":2232342832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6829","nonce":2232342832}]},"public_addr":"172.21.15.121:6823/2232342832","cluster_addr":"172.21.15.121:6826/2232342832","heartbeat_back_addr":"172.21.15.121:6831/2232342832","heartbeat_front_addr":"172.21.15.121:6829/2232342832","state":["exists","up"]},{"osd":4,"uuid":"b3f3b4a1-b375-4526-b69e-344ab24c2da1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6801","nonce":99353111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6803","nonce":99353111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6807","nonce":99353111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6805","nonce":99353111}]},"public_addr":"172.21.15.123:6801/99353111","cluster_addr":"172.21.15.123:6803/99353111","heartbeat_back_addr":"172.21.15.123:6807/99353111","heartbeat_front_addr":"172.21.15.123:6805/99353111","state":["exists","up"]},{"osd":5,"uuid":"2ee24a08-1624-4961-9642-d7b510282484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6818","nonce":2504374688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6822","nonce":2504374688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6830","nonce":2504374688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6826","nonce":2504374688}]},"public_addr":"172.21.15.123:6818/2504374688","cluster_addr":"172.21.15.123:6822/2504374688","heartbeat_back_addr":"172.21.15.123:6830/2504374688","heartbeat_front_addr":"172.21.15.123:6826/2504374688","state":["exists","up"]},{"osd":6,"uuid":"186118cc-6128-49de-acca-55260945c1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6819","nonce":4139299599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6823","nonce":4139299599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6829","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6831","nonce":4139299599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6827","nonce":4139299599}]},"public_addr":"172.21.15.123:6819/4139299599","cluster_addr":"172.21.15.123:6823/4139299599","heartbeat_back_addr":"172.21.15.123:6831/4139299599","heartbeat_front_addr":"172.21.15.123:6827/4139299599","state":["exists","up"]},{"osd":7,"uuid":"a00aac6e-4bd9-415a-a93d-ae85210865ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6809","nonce":4043981069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6811","nonce":4043981069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6815","nonce":4043981069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6813","nonce":4043981069}]},"public_addr":"172.21.15.123:6809/4043981069","cluster_addr":"172.21.15.123:6811/4043981069","heartbeat_back_addr":"172.21.15.123:6815/4043981069","heartbeat_front_addr":"172.21.15.123:6813/4043981069","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.904540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.958585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.616367+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.885244+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.103127+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.167248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.884348+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.862356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:21:28.661 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.661+0000 7f6282c00640 1 -- 172.21.15.121:0/3135730777 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f625c06c8f0 msgr2=0x7f625c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:28.661 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.661+0000 7f6282c00640 1 --2- 172.21.15.121:0/3135730777 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f625c06c8f0 0x7f625c06edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f627c069030 tx=0x7f6270009040 comp rx=0 tx=0).stop 2024-04-20T10:21:28.661 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.661+0000 7f6282c00640 1 -- 172.21.15.121:0/3135730777 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f627c0ceab0 msgr2=0x7f627c14f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:28.661 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.661+0000 7f6282c00640 1 --2- 172.21.15.121:0/3135730777 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f627c0ceab0 0x7f627c14f590 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6274000f80 tx=0x7f627400b4a0 comp rx=0 tx=0).stop 2024-04-20T10:21:28.661 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.661+0000 7f6282c00640 1 -- 172.21.15.121:0/3135730777 shutdown_connections 2024-04-20T10:21:28.661 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.661+0000 7f6282c00640 1 --2- 172.21.15.121:0/3135730777 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f627c0ceab0 0x7f627c14f590 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:28.661 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.661+0000 7f6282c00640 1 --2- 172.21.15.121:0/3135730777 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f625c06c8f0 0x7f625c06edb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:28.661 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.661+0000 7f6282c00640 1 --2- 172.21.15.121:0/3135730777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f627c09ad00 0x7f627c0ce570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:28.661 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.661+0000 7f6282c00640 1 --2- 172.21.15.121:0/3135730777 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6264006a10 0x7f627c09b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:28.661 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.661+0000 7f6282c00640 1 -- 172.21.15.121:0/3135730777 >> 172.21.15.121:0/3135730777 conn(0x7f627c0d0040 msgr2=0x7f627c0b2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:28.661 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.661+0000 7f6282c00640 1 -- 172.21.15.121:0/3135730777 shutdown_connections 2024-04-20T10:21:28.662 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.661+0000 7f6282c00640 1 -- 172.21.15.121:0/3135730777 wait complete. 2024-04-20T10:21:28.680 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-20T10:21:28.889 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.889+0000 7f31f6c00640 1 Processor -- start 2024-04-20T10:21:28.889 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.890+0000 7f31f6c00640 1 -- start start 2024-04-20T10:21:28.891 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.891+0000 7f31f6c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f31f00d3b20 0x7f31f00b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:28.891 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.891+0000 7f31f6c00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31f006c010 0x7f31f00ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:28.891 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.891+0000 7f31f6c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31f00d0980 con 0x7f31f0161a50 2024-04-20T10:21:28.891 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.891+0000 7f31f6c00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f31f00d0c60 con 0x7f31f00d3b20 2024-04-20T10:21:28.891 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.891+0000 7f31f6c00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f31f0162ae0 con 0x7f31f006c010 2024-04-20T10:21:28.891 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.892+0000 7f31f4e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f31f00d3b20 0x7f31f00b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:28.891 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.892+0000 7f31f4e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f31f00d3b20 0x7f31f00b3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54102/0 (socket says 172.21.15.121:54102) 2024-04-20T10:21:28.891 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.892+0000 7f31f4e00640 1 -- 172.21.15.121:0/261381893 learned_addr learned my addr 172.21.15.121:0/261381893 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:28.892 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.892+0000 7f31f6200640 1 --2- 172.21.15.121:0/261381893 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31f006c010 0x7f31f00ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:28.892 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.892+0000 7f31f4e00640 1 -- 172.21.15.121:0/261381893 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31f006c010 msgr2=0x7f31f00ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:28.892 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.892+0000 7f31f4e00640 1 --2- 172.21.15.121:0/261381893 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31f006c010 0x7f31f00ceaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:28.892 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.892+0000 7f31f4e00640 1 -- 172.21.15.121:0/261381893 >> v1:172.21.15.123:6789/0 conn(0x7f31f0161a50 legacy=0x7f31f006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:28.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.892+0000 7f31f4e00640 1 -- 172.21.15.121:0/261381893 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f0162490 con 0x7f31f00d3b20 2024-04-20T10:21:28.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.893+0000 7f31f4e00640 1 --2- 172.21.15.121:0/261381893 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31f00d3b20 0x7f31f00b3180 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f31dc009870 tx=0x7f31dc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4376e7915923a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:28.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.894+0000 7f31efe00640 1 -- 172.21.15.121:0/261381893 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31dc035020 con 0x7f31f00d3b20 2024-04-20T10:21:28.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.894+0000 7f31efe00640 1 -- 172.21.15.121:0/261381893 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31dc031640 con 0x7f31f00d3b20 2024-04-20T10:21:28.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.894+0000 7f31efe00640 1 -- 172.21.15.121:0/261381893 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31dc02d040 con 0x7f31f00d3b20 2024-04-20T10:21:28.894 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.894+0000 7f31f6c00640 1 -- 172.21.15.121:0/261381893 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31f00d3b20 msgr2=0x7f31f00b3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:28.894 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.894+0000 7f31f6c00640 1 --2- 172.21.15.121:0/261381893 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31f00d3b20 0x7f31f00b3180 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f31dc009870 tx=0x7f31dc02b8f0 comp rx=0 tx=0).stop 2024-04-20T10:21:28.894 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.895+0000 7f31f6c00640 1 -- 172.21.15.121:0/261381893 shutdown_connections 2024-04-20T10:21:28.895 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.895+0000 7f31f6c00640 1 --2- 172.21.15.121:0/261381893 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31f006c010 0x7f31f00ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:28.895 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.895+0000 7f31f6c00640 1 --2- 172.21.15.121:0/261381893 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31f00d3b20 0x7f31f00b3180 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:28.895 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.895+0000 7f31f6c00640 1 -- 172.21.15.121:0/261381893 >> 172.21.15.121:0/261381893 conn(0x7f31f00d0ed0 msgr2=0x7f31f00d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:28.895 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.895+0000 7f31f6c00640 1 -- 172.21.15.121:0/261381893 shutdown_connections 2024-04-20T10:21:28.895 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.895+0000 7f31f6c00640 1 -- 172.21.15.121:0/261381893 wait complete. 2024-04-20T10:21:28.896 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.896+0000 7f31f6c00640 1 Processor -- start 2024-04-20T10:21:28.896 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.897+0000 7f31f6c00640 1 -- start start 2024-04-20T10:21:28.898 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.897+0000 7f31f6c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31f006c010 0x7f31f00cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:28.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.897+0000 7f31f6c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31f0161a50 0x7f31f014b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:28.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.897+0000 7f31f6c00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31f015bfa0 0x7f31f01e9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:28.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.898+0000 7f31f6c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f31f00d0600 con 0x7f31f006c010 2024-04-20T10:21:28.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.898+0000 7f31f6c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f31f00d0480 con 0x7f31f0161a50 2024-04-20T10:21:28.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.898+0000 7f31f6c00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f31f0161e30 con 0x7f31f015bfa0 2024-04-20T10:21:28.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.898+0000 7f31f5800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31f006c010 0x7f31f00cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:28.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.898+0000 7f31f5800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31f006c010 0x7f31f00cc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54106/0 (socket says 172.21.15.121:54106) 2024-04-20T10:21:28.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.898+0000 7f31f5800640 1 -- 172.21.15.121:0/3646826862 learned_addr learned my addr 172.21.15.121:0/3646826862 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:28.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.898+0000 7f31f4e00640 1 --2- 172.21.15.121:0/3646826862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31f0161a50 0x7f31f014b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:28.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.898+0000 7f31f6200640 1 --2- 172.21.15.121:0/3646826862 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31f015bfa0 0x7f31f01e9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:28.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.898+0000 7f31f5800640 1 -- 172.21.15.121:0/3646826862 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31f015bfa0 msgr2=0x7f31f01e9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:28.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.898+0000 7f31f5800640 1 --2- 172.21.15.121:0/3646826862 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31f015bfa0 0x7f31f01e9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:28.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.898+0000 7f31f5800640 1 -- 172.21.15.121:0/3646826862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31f0161a50 msgr2=0x7f31f014b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:28.900 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.898+0000 7f31f5800640 1 --2- 172.21.15.121:0/3646826862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31f0161a50 0x7f31f014b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:28.900 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.898+0000 7f31f5800640 1 -- 172.21.15.121:0/3646826862 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f00ccbc0 con 0x7f31f006c010 2024-04-20T10:21:28.900 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.898+0000 7f31f6200640 1 --2- 172.21.15.121:0/3646826862 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31f015bfa0 0x7f31f01e9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:28.900 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.899+0000 7f31f5800640 1 --2- 172.21.15.121:0/3646826862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31f006c010 0x7f31f00cc5e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f31e400b3f0 tx=0x7f31e400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:28.900 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.899+0000 7f31ed600640 1 -- 172.21.15.121:0/3646826862 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e4015020 con 0x7f31f006c010 2024-04-20T10:21:28.900 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.899+0000 7f31ed600640 1 -- 172.21.15.121:0/3646826862 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31e4011830 con 0x7f31f006c010 2024-04-20T10:21:28.900 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.899+0000 7f31f6c00640 1 -- 172.21.15.121:0/3646826862 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31f00ccea0 con 0x7f31f006c010 2024-04-20T10:21:28.900 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.899+0000 7f31ed600640 1 -- 172.21.15.121:0/3646826862 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e4011c20 con 0x7f31f006c010 2024-04-20T10:21:28.900 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.899+0000 7f31f6c00640 1 -- 172.21.15.121:0/3646826862 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31f00cd460 con 0x7f31f006c010 2024-04-20T10:21:28.901 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.900+0000 7f31f6c00640 1 -- 172.21.15.121:0/3646826862 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31f0075b30 con 0x7f31f006c010 2024-04-20T10:21:28.901 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.901+0000 7f31ed600640 1 -- 172.21.15.121:0/3646826862 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f31e4018020 con 0x7f31f006c010 2024-04-20T10:21:28.901 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.902+0000 7f31ed600640 1 --2- 172.21.15.121:0/3646826862 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f31d006c920 0x7f31d006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:28.901 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.902+0000 7f31ed600640 1 -- 172.21.15.121:0/3646826862 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f31e408ba50 con 0x7f31f006c010 2024-04-20T10:21:28.902 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.902+0000 7f31f4e00640 1 --2- 172.21.15.121:0/3646826862 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f31d006c920 0x7f31d006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:28.902 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.903+0000 7f31f4e00640 1 --2- 172.21.15.121:0/3646826862 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f31d006c920 0x7f31d006ede0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f31dc036000 tx=0x7f31dc03c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:28.906 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:28.906+0000 7f31ed600640 1 -- 172.21.15.121:0/3646826862 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f31e401f030 con 0x7f31f006c010 2024-04-20T10:21:29.150 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.151+0000 7f31f6c00640 1 -- 172.21.15.121:0/3646826862 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f31f00b8a90 con 0x7f31f006c010 2024-04-20T10:21:29.279 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.279+0000 7f31ed600640 1 -- 172.21.15.121:0/3646826862 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7f31e40172c0 con 0x7f31f006c010 2024-04-20T10:21:29.283 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.283+0000 7f31f6c00640 1 -- 172.21.15.121:0/3646826862 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f31d006c920 msgr2=0x7f31d006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:29.283 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.283+0000 7f31f6c00640 1 --2- 172.21.15.121:0/3646826862 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f31d006c920 0x7f31d006ede0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f31dc036000 tx=0x7f31dc03c040 comp rx=0 tx=0).stop 2024-04-20T10:21:29.283 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.283+0000 7f31f6c00640 1 -- 172.21.15.121:0/3646826862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31f006c010 msgr2=0x7f31f00cc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:29.283 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.283+0000 7f31f6c00640 1 --2- 172.21.15.121:0/3646826862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31f006c010 0x7f31f00cc5e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f31e400b3f0 tx=0x7f31e400b8b0 comp rx=0 tx=0).stop 2024-04-20T10:21:29.283 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.283+0000 7f31f6c00640 1 -- 172.21.15.121:0/3646826862 shutdown_connections 2024-04-20T10:21:29.283 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.283+0000 7f31f6c00640 1 --2- 172.21.15.121:0/3646826862 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31f015bfa0 0x7f31f01e9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:29.283 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.283+0000 7f31f6c00640 1 --2- 172.21.15.121:0/3646826862 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f31d006c920 0x7f31d006ede0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:29.283 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.283+0000 7f31f6c00640 1 --2- 172.21.15.121:0/3646826862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31f0161a50 0x7f31f014b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:29.283 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.283+0000 7f31f6c00640 1 --2- 172.21.15.121:0/3646826862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31f006c010 0x7f31f00cc5e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:29.283 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.283+0000 7f31f6c00640 1 -- 172.21.15.121:0/3646826862 >> 172.21.15.121:0/3646826862 conn(0x7f31f00d0ed0 msgr2=0x7f31f00b5340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:29.283 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.283+0000 7f31f6c00640 1 -- 172.21.15.121:0/3646826862 shutdown_connections 2024-04-20T10:21:29.284 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.283+0000 7f31f6c00640 1 -- 172.21.15.121:0/3646826862 wait complete. 2024-04-20T10:21:29.305 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-04-20T10:21:29.305 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-20T10:21:29.508 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.508+0000 7fd331000640 1 Processor -- start 2024-04-20T10:21:29.508 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.509+0000 7fd331000640 1 -- start start 2024-04-20T10:21:29.511 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.510+0000 7fd331000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd32c099900 0x7fd32c0d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:29.511 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.510+0000 7fd331000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd32c0ad160 0x7fd32c0ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:29.511 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.510+0000 7fd331000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd32c0afb40 con 0x7fd32c06db90 2024-04-20T10:21:29.511 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.510+0000 7fd331000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fd32c06baa0 con 0x7fd32c0ad160 2024-04-20T10:21:29.511 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.510+0000 7fd331000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fd32c06bd80 con 0x7fd32c099900 2024-04-20T10:21:29.511 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.510+0000 7fd32b400640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fd32c06db90 0x7fd32c06df70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:49950/0 (socket says 172.21.15.121:49950) 2024-04-20T10:21:29.512 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.510+0000 7fd32b400640 1 -- 172.21.15.121:0/507110839 learned_addr learned my addr 172.21.15.121:0/507110839 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:29.512 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.511+0000 7fd322a00640 1 --2- 172.21.15.121:0/507110839 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd32c099900 0x7fd32c0d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:29.512 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.511+0000 7fd32be00640 1 --2- 172.21.15.121:0/507110839 >> v2:172.21.15.123:3300/0 conn(0x7fd32c0ad160 0x7fd32c0ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:29.512 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.511+0000 7fd32aa00640 1 -- 172.21.15.121:0/507110839 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1926047664 0 0) 0x7fd32c0afb40 con 0x7fd32c06db90 2024-04-20T10:21:29.512 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.511+0000 7fd32aa00640 1 -- 172.21.15.121:0/507110839 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd310003520 con 0x7fd32c06db90 2024-04-20T10:21:29.512 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.511+0000 7fd32aa00640 1 -- 172.21.15.121:0/507110839 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3823471147 0 0) 0x7fd310003520 con 0x7fd32c06db90 2024-04-20T10:21:29.512 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.511+0000 7fd32aa00640 1 -- 172.21.15.121:0/507110839 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd32c099900 msgr2=0x7fd32c0d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:29.512 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.511+0000 7fd32aa00640 1 --2- 172.21.15.121:0/507110839 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd32c099900 0x7fd32c0d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:29.512 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.511+0000 7fd32aa00640 1 -- 172.21.15.121:0/507110839 >> v2:172.21.15.123:3300/0 conn(0x7fd32c0ad160 msgr2=0x7fd32c0ad5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:29.512 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.511+0000 7fd32aa00640 1 --2- 172.21.15.121:0/507110839 >> v2:172.21.15.123:3300/0 conn(0x7fd32c0ad160 0x7fd32c0ad5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:29.512 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.511+0000 7fd32aa00640 1 -- 172.21.15.121:0/507110839 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd32c06c060 con 0x7fd32c06db90 2024-04-20T10:21:29.512 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.511+0000 7fd32aa00640 1 -- 172.21.15.121:0/507110839 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7fd31c001670 con 0x7fd32c06db90 2024-04-20T10:21:29.512 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.512+0000 7fd32aa00640 1 -- 172.21.15.121:0/507110839 >> v1:172.21.15.121:6789/0 conn(0x7fd32c06db90 legacy=0x7fd32c06df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:29.513 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.512+0000 7fd32aa00640 1 --2- 172.21.15.121:0/507110839 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3100039b0 0x7fd310003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:29.513 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.512+0000 7fd32aa00640 1 --2- 172.21.15.121:0/507110839 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd32c099900 0x7fd310004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:29.513 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.512+0000 7fd32aa00640 1 --2- 172.21.15.121:0/507110839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd310004d40 0x7fd3100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:29.513 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.512+0000 7fd32aa00640 1 -- 172.21.15.121:0/507110839 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c06baa0 con 0x7fd3100039b0 2024-04-20T10:21:29.513 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.512+0000 7fd32aa00640 1 -- 172.21.15.121:0/507110839 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c06bd80 con 0x7fd310004d40 2024-04-20T10:21:29.513 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.512+0000 7fd32aa00640 1 -- 172.21.15.121:0/507110839 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fd31000b9c0 con 0x7fd32c099900 2024-04-20T10:21:29.513 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.512+0000 7fd32b400640 1 --2- 172.21.15.121:0/507110839 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3100039b0 0x7fd310003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:29.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.512+0000 7fd322a00640 1 --2- 172.21.15.121:0/507110839 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd32c099900 0x7fd310004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:29.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.512+0000 7fd32b400640 1 -- 172.21.15.121:0/507110839 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd32c099900 msgr2=0x7fd310004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:29.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.512+0000 7fd32b400640 1 --2- 172.21.15.121:0/507110839 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd32c099900 0x7fd310004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:29.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.512+0000 7fd32b400640 1 -- 172.21.15.121:0/507110839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd310004d40 msgr2=0x7fd3100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:29.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.512+0000 7fd32b400640 1 --2- 172.21.15.121:0/507110839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd310004d40 0x7fd3100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:29.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.512+0000 7fd32b400640 1 -- 172.21.15.121:0/507110839 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd310003520 con 0x7fd3100039b0 2024-04-20T10:21:29.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.513+0000 7fd32b400640 1 --2- 172.21.15.121:0/507110839 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3100039b0 0x7fd310003dd0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd31c00c190 tx=0x7fd310005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f512eb1985f046b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:29.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.514+0000 7fd32aa00640 1 -- 172.21.15.121:0/507110839 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c007c80 con 0x7fd3100039b0 2024-04-20T10:21:29.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.514+0000 7fd32aa00640 1 -- 172.21.15.121:0/507110839 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd31c030070 con 0x7fd3100039b0 2024-04-20T10:21:29.516 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.514+0000 7fd32aa00640 1 -- 172.21.15.121:0/507110839 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c005d30 con 0x7fd3100039b0 2024-04-20T10:21:29.516 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.514+0000 7fd331000640 1 -- 172.21.15.121:0/507110839 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3100039b0 msgr2=0x7fd310003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:29.516 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.514+0000 7fd331000640 1 --2- 172.21.15.121:0/507110839 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd3100039b0 0x7fd310003dd0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd31c00c190 tx=0x7fd310005830 comp rx=0 tx=0).stop 2024-04-20T10:21:29.516 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.514+0000 7fd32be00640 1 -- 172.21.15.121:0/507110839 reap_dead start 2024-04-20T10:21:29.516 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.514+0000 7fd331000640 1 -- 172.21.15.121:0/507110839 shutdown_connections 2024-04-20T10:21:29.516 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.514+0000 7fd331000640 1 -- 172.21.15.121:0/507110839 >> 172.21.15.121:0/507110839 conn(0x7fd32c0b5270 msgr2=0x7fd32c0b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:29.516 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.514+0000 7fd331000640 1 -- 172.21.15.121:0/507110839 shutdown_connections 2024-04-20T10:21:29.516 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.514+0000 7fd331000640 1 -- 172.21.15.121:0/507110839 wait complete. 2024-04-20T10:21:29.517 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.517+0000 7fd331000640 1 Processor -- start 2024-04-20T10:21:29.517 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.517+0000 7fd331000640 1 -- start start 2024-04-20T10:21:29.519 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.518+0000 7fd331000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3100039b0 0x7fd32c1552f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:29.519 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.518+0000 7fd331000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd32c0cf470 0x7fd32c15f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:29.519 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.518+0000 7fd331000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd32c153650 0x7fd32c160960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:29.519 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.518+0000 7fd331000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c162af0 con 0x7fd32c153650 2024-04-20T10:21:29.519 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.518+0000 7fd331000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c06c620 con 0x7fd3100039b0 2024-04-20T10:21:29.519 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.518+0000 7fd331000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fd32c162c70 con 0x7fd32c0cf470 2024-04-20T10:21:29.519 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.518+0000 7fd32be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd32c153650 0x7fd32c160960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:29.520 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.518+0000 7fd32be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd32c153650 0x7fd32c160960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54120/0 (socket says 172.21.15.121:54120) 2024-04-20T10:21:29.520 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.518+0000 7fd32be00640 1 -- 172.21.15.121:0/767847143 learned_addr learned my addr 172.21.15.121:0/767847143 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:29.520 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.519+0000 7fd322a00640 1 --2- 172.21.15.121:0/767847143 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd32c0cf470 0x7fd32c15f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:29.520 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.519+0000 7fd32b400640 1 --2- 172.21.15.121:0/767847143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3100039b0 0x7fd32c1552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:29.520 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.519+0000 7fd32be00640 1 -- 172.21.15.121:0/767847143 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd32c0cf470 msgr2=0x7fd32c15f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:29.520 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.519+0000 7fd32be00640 1 --2- 172.21.15.121:0/767847143 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd32c0cf470 0x7fd32c15f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:29.520 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.519+0000 7fd32be00640 1 -- 172.21.15.121:0/767847143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3100039b0 msgr2=0x7fd32c1552f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:29.520 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.519+0000 7fd32be00640 1 --2- 172.21.15.121:0/767847143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3100039b0 0x7fd32c1552f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:29.520 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.519+0000 7fd32be00640 1 -- 172.21.15.121:0/767847143 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd32c15fa80 con 0x7fd32c153650 2024-04-20T10:21:29.520 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.519+0000 7fd32be00640 1 --2- 172.21.15.121:0/767847143 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd32c153650 0x7fd32c160960 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0ccb40 tx=0x7fd32400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:29.520 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.519+0000 7fd323e00640 1 -- 172.21.15.121:0/767847143 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd32400bd10 con 0x7fd32c153650 2024-04-20T10:21:29.520 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.519+0000 7fd323e00640 1 -- 172.21.15.121:0/767847143 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd32401a070 con 0x7fd32c153650 2024-04-20T10:21:29.521 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.520+0000 7fd331000640 1 -- 172.21.15.121:0/767847143 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd32c15fd60 con 0x7fd32c153650 2024-04-20T10:21:29.521 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.520+0000 7fd323e00640 1 -- 172.21.15.121:0/767847143 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd324014640 con 0x7fd32c153650 2024-04-20T10:21:29.521 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.520+0000 7fd331000640 1 -- 172.21.15.121:0/767847143 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd32c0cce50 con 0x7fd32c153650 2024-04-20T10:21:29.521 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.521+0000 7fd331000640 1 -- 172.21.15.121:0/767847143 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd32c075b30 con 0x7fd32c153650 2024-04-20T10:21:29.521 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.522+0000 7fd323e00640 1 -- 172.21.15.121:0/767847143 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd324005d10 con 0x7fd32c153650 2024-04-20T10:21:29.522 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.523+0000 7fd323e00640 1 --2- 172.21.15.121:0/767847143 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fd30806c8f0 0x7fd30806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:29.522 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.523+0000 7fd323e00640 1 -- 172.21.15.121:0/767847143 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd32408c1d0 con 0x7fd32c153650 2024-04-20T10:21:29.523 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.523+0000 7fd32b400640 1 --2- 172.21.15.121:0/767847143 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fd30806c8f0 0x7fd30806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:29.523 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.524+0000 7fd32b400640 1 --2- 172.21.15.121:0/767847143 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fd30806c8f0 0x7fd30806edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd31c034000 tx=0x7fd31c007000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:29.527 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.528+0000 7fd323e00640 1 -- 172.21.15.121:0/767847143 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd324024020 con 0x7fd32c153650 2024-04-20T10:21:29.773 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:29.774+0000 7fd331000640 1 -- 172.21.15.121:0/767847143 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd32c06d1d0 con 0x7fd32c153650 2024-04-20T10:21:30.282 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.283+0000 7fd323e00640 1 -- 172.21.15.121:0/767847143 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7fd32405a220 con 0x7fd32c153650 2024-04-20T10:21:30.287 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.287+0000 7fd331000640 1 -- 172.21.15.121:0/767847143 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fd30806c8f0 msgr2=0x7fd30806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:30.287 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.287+0000 7fd331000640 1 --2- 172.21.15.121:0/767847143 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fd30806c8f0 0x7fd30806edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd31c034000 tx=0x7fd31c007000 comp rx=0 tx=0).stop 2024-04-20T10:21:30.288 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.287+0000 7fd331000640 1 -- 172.21.15.121:0/767847143 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd32c153650 msgr2=0x7fd32c160960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:30.288 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.287+0000 7fd331000640 1 --2- 172.21.15.121:0/767847143 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd32c153650 0x7fd32c160960 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0ccb40 tx=0x7fd32400b480 comp rx=0 tx=0).stop 2024-04-20T10:21:30.288 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.288+0000 7fd331000640 1 -- 172.21.15.121:0/767847143 shutdown_connections 2024-04-20T10:21:30.288 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.288+0000 7fd331000640 1 --2- 172.21.15.121:0/767847143 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd32c153650 0x7fd32c160960 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:30.288 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.288+0000 7fd331000640 1 --2- 172.21.15.121:0/767847143 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd32c0cf470 0x7fd32c15f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:30.288 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.288+0000 7fd331000640 1 --2- 172.21.15.121:0/767847143 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fd30806c8f0 0x7fd30806edb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:30.288 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.288+0000 7fd331000640 1 --2- 172.21.15.121:0/767847143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3100039b0 0x7fd32c1552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:30.288 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.288+0000 7fd331000640 1 -- 172.21.15.121:0/767847143 >> 172.21.15.121:0/767847143 conn(0x7fd32c0b5270 msgr2=0x7fd32c0b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:30.288 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.288+0000 7fd331000640 1 -- 172.21.15.121:0/767847143 shutdown_connections 2024-04-20T10:21:30.288 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.288+0000 7fd331000640 1 -- 172.21.15.121:0/767847143 wait complete. 2024-04-20T10:21:30.311 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-20T10:21:30.311 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:21:30.510 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.510+0000 7f95e7e00640 1 Processor -- start 2024-04-20T10:21:30.510 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.510+0000 7f95e7e00640 1 -- start start 2024-04-20T10:21:30.511 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.512+0000 7f95e7e00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f95e806cde0 0x7f95e80d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:30.511 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.512+0000 7f95e7e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95e806c190 con 0x7f95e8161d50 2024-04-20T10:21:30.512 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.512+0000 7f95e7e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95e806c470 con 0x7f95e80b3330 2024-04-20T10:21:30.512 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.512+0000 7f95e7e00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f95e8162130 con 0x7f95e806cde0 2024-04-20T10:21:30.512 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.512+0000 7f95e6a00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f95e8161d50 0x7f95e806c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:49954/0 (socket says 172.21.15.121:49954) 2024-04-20T10:21:30.512 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.512+0000 7f95e6a00640 1 -- 172.21.15.121:0/2427505671 learned_addr learned my addr 172.21.15.121:0/2427505671 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:30.514 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.512+0000 7f95dfe00640 1 --2- 172.21.15.121:0/2427505671 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f95e806cde0 0x7f95e80d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:30.514 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.512+0000 7f95e6000640 1 -- 172.21.15.121:0/2427505671 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3981112026 0 0) 0x7f95e806c190 con 0x7f95e8161d50 2024-04-20T10:21:30.514 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.513+0000 7f95e6000640 1 -- 172.21.15.121:0/2427505671 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f95cc003520 con 0x7f95e8161d50 2024-04-20T10:21:30.514 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.513+0000 7f95e6000640 1 -- 172.21.15.121:0/2427505671 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1677620816 0 0) 0x7f95cc003520 con 0x7f95e8161d50 2024-04-20T10:21:30.514 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.513+0000 7f95e6000640 1 -- 172.21.15.121:0/2427505671 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f95e806cde0 msgr2=0x7f95e80d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:30.514 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.513+0000 7f95e6000640 1 --2- 172.21.15.121:0/2427505671 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f95e806cde0 0x7f95e80d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:30.514 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.513+0000 7f95e6000640 1 -- 172.21.15.121:0/2427505671 >> v1:172.21.15.123:6789/0 conn(0x7f95e80b3330 legacy=0x7f95e80b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:30.514 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.513+0000 7f95e6000640 1 -- 172.21.15.121:0/2427505671 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e80d2e00 con 0x7f95e8161d50 2024-04-20T10:21:30.514 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.513+0000 7f95e6000640 1 -- 172.21.15.121:0/2427505671 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f95d8001670 con 0x7f95e8161d50 2024-04-20T10:21:30.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.513+0000 7f95e6000640 1 -- 172.21.15.121:0/2427505671 >> v1:172.21.15.121:6789/0 conn(0x7f95e8161d50 legacy=0x7f95e806c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:30.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.513+0000 7f95e6000640 1 --2- 172.21.15.121:0/2427505671 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f95e806cde0 0x7f95cc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:30.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.513+0000 7f95e6000640 1 --2- 172.21.15.121:0/2427505671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95cc003fb0 0x7f95cc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:30.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.513+0000 7f95e6000640 1 --2- 172.21.15.121:0/2427505671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95cc004d40 0x7f95cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:30.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.513+0000 7f95e6000640 1 -- 172.21.15.121:0/2427505671 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f95cc00bad0 con 0x7f95cc003fb0 2024-04-20T10:21:30.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.513+0000 7f95e6000640 1 -- 172.21.15.121:0/2427505671 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f95cc00bc80 con 0x7f95cc004d40 2024-04-20T10:21:30.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.513+0000 7f95e6000640 1 -- 172.21.15.121:0/2427505671 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f95cc00be80 con 0x7f95e806cde0 2024-04-20T10:21:30.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.513+0000 7f95dfe00640 1 --2- 172.21.15.121:0/2427505671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95cc003fb0 0x7f95cc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:30.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.514+0000 7f95dfe00640 1 -- 172.21.15.121:0/2427505671 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f95e806cde0 msgr2=0x7f95cc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:30.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.514+0000 7f95e6a00640 1 --2- 172.21.15.121:0/2427505671 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f95e806cde0 0x7f95cc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:30.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.514+0000 7f95e7400640 1 --2- 172.21.15.121:0/2427505671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95cc004d40 0x7f95cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:30.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.514+0000 7f95dfe00640 1 --2- 172.21.15.121:0/2427505671 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f95e806cde0 0x7f95cc0039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:30.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.514+0000 7f95dfe00640 1 -- 172.21.15.121:0/2427505671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95cc004d40 msgr2=0x7f95cc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:30.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.514+0000 7f95dfe00640 1 --2- 172.21.15.121:0/2427505671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95cc004d40 0x7f95cc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:30.516 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.514+0000 7f95dfe00640 1 -- 172.21.15.121:0/2427505671 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95cc003520 con 0x7f95cc003fb0 2024-04-20T10:21:30.516 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.514+0000 7f95e7400640 1 --2- 172.21.15.121:0/2427505671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95cc004d40 0x7f95cc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:30.517 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.515+0000 7f95dfe00640 1 --2- 172.21.15.121:0/2427505671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95cc003fb0 0x7f95cc004800 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f95d0009ac0 tx=0x7f95cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7231fae495feaed server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:30.517 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.515+0000 7f95e6000640 1 -- 172.21.15.121:0/2427505671 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d0039070 con 0x7f95cc003fb0 2024-04-20T10:21:30.517 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.515+0000 7f95e6000640 1 -- 172.21.15.121:0/2427505671 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95d002c600 con 0x7f95cc003fb0 2024-04-20T10:21:30.517 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.515+0000 7f95e6000640 1 -- 172.21.15.121:0/2427505671 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d0009690 con 0x7f95cc003fb0 2024-04-20T10:21:30.517 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.516+0000 7f95e7e00640 1 -- 172.21.15.121:0/2427505671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95cc003fb0 msgr2=0x7f95cc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:30.518 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.516+0000 7f95e7e00640 1 --2- 172.21.15.121:0/2427505671 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95cc003fb0 0x7f95cc004800 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f95d0009ac0 tx=0x7f95cc005830 comp rx=0 tx=0).stop 2024-04-20T10:21:30.518 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.516+0000 7f95e7400640 1 -- 172.21.15.121:0/2427505671 reap_dead start 2024-04-20T10:21:30.518 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.516+0000 7f95e7e00640 1 -- 172.21.15.121:0/2427505671 shutdown_connections 2024-04-20T10:21:30.518 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.516+0000 7f95e7e00640 1 -- 172.21.15.121:0/2427505671 >> 172.21.15.121:0/2427505671 conn(0x7f95e80d4450 msgr2=0x7f95e80d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:30.518 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.516+0000 7f95e7e00640 1 -- 172.21.15.121:0/2427505671 shutdown_connections 2024-04-20T10:21:30.518 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.516+0000 7f95e7e00640 1 -- 172.21.15.121:0/2427505671 wait complete. 2024-04-20T10:21:30.518 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.518+0000 7f95e7e00640 1 Processor -- start 2024-04-20T10:21:30.518 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.518+0000 7f95e7e00640 1 -- start start 2024-04-20T10:21:30.520 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.519+0000 7f95e7e00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f95cc003fb0 0x7f95e814db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:30.520 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.519+0000 7f95e7e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95e80b4f80 0x7f95e80c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:30.520 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.519+0000 7f95e7e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e80b53e0 0x7f95e80ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:30.520 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.519+0000 7f95e7e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f95e80d38e0 con 0x7f95e80b4f80 2024-04-20T10:21:30.520 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.519+0000 7f95e7e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f95e80d33c0 con 0x7f95e80b53e0 2024-04-20T10:21:30.520 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.519+0000 7f95e7e00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f95e80d3a60 con 0x7f95cc003fb0 2024-04-20T10:21:30.521 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.519+0000 7f95dfe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95e80b4f80 0x7f95e80c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:30.521 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.519+0000 7f95dfe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95e80b4f80 0x7f95e80c9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54134/0 (socket says 172.21.15.121:54134) 2024-04-20T10:21:30.521 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.519+0000 7f95dfe00640 1 -- 172.21.15.121:0/2860760675 learned_addr learned my addr 172.21.15.121:0/2860760675 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:30.521 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.520+0000 7f95e7400640 1 --2- 172.21.15.121:0/2860760675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e80b53e0 0x7f95e80ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:30.521 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.519+0000 7f95e6a00640 1 --2- 172.21.15.121:0/2860760675 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f95cc003fb0 0x7f95e814db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:30.521 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.520+0000 7f95dfe00640 1 -- 172.21.15.121:0/2860760675 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f95cc003fb0 msgr2=0x7f95e814db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:30.521 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.520+0000 7f95dfe00640 1 --2- 172.21.15.121:0/2860760675 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f95cc003fb0 0x7f95e814db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:30.521 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.520+0000 7f95dfe00640 1 -- 172.21.15.121:0/2860760675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e80b53e0 msgr2=0x7f95e80ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:30.521 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.520+0000 7f95dfe00640 1 --2- 172.21.15.121:0/2860760675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e80b53e0 0x7f95e80ca4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:30.521 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.520+0000 7f95dfe00640 1 -- 172.21.15.121:0/2860760675 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95e81f7b00 con 0x7f95e80b4f80 2024-04-20T10:21:30.521 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.520+0000 7f95e6a00640 1 --2- 172.21.15.121:0/2860760675 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f95cc003fb0 0x7f95e814db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:30.521 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.520+0000 7f95e7400640 1 --2- 172.21.15.121:0/2860760675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e80b53e0 0x7f95e80ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:30.521 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.520+0000 7f95dfe00640 1 --2- 172.21.15.121:0/2860760675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95e80b4f80 0x7f95e80c9f70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f95d0000c00 tx=0x7f95d0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:30.521 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.520+0000 7f95dea00640 1 -- 172.21.15.121:0/2860760675 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d0042020 con 0x7f95e80b4f80 2024-04-20T10:21:30.522 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.520+0000 7f95dea00640 1 -- 172.21.15.121:0/2860760675 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95d003a040 con 0x7f95e80b4f80 2024-04-20T10:21:30.522 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.520+0000 7f95dea00640 1 -- 172.21.15.121:0/2860760675 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d0039070 con 0x7f95e80b4f80 2024-04-20T10:21:30.522 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.520+0000 7f95e7e00640 1 -- 172.21.15.121:0/2860760675 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95e81f7de0 con 0x7f95e80b4f80 2024-04-20T10:21:30.522 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.520+0000 7f95e7e00640 1 -- 172.21.15.121:0/2860760675 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95e81f83a0 con 0x7f95e80b4f80 2024-04-20T10:21:30.523 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.521+0000 7f95e7e00640 1 -- 172.21.15.121:0/2860760675 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95e8075b60 con 0x7f95e80b4f80 2024-04-20T10:21:30.523 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.522+0000 7f95dea00640 1 -- 172.21.15.121:0/2860760675 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f95d0045020 con 0x7f95e80b4f80 2024-04-20T10:21:30.523 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.523+0000 7f95dea00640 1 --2- 172.21.15.121:0/2860760675 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f95c406c920 0x7f95c406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:30.523 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.524+0000 7f95dea00640 1 -- 172.21.15.121:0/2860760675 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f95d00ad3d0 con 0x7f95e80b4f80 2024-04-20T10:21:30.523 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.524+0000 7f95e6a00640 1 --2- 172.21.15.121:0/2860760675 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f95c406c920 0x7f95c406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:30.523 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.524+0000 7f95e6a00640 1 --2- 172.21.15.121:0/2860760675 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f95c406c920 0x7f95c406ede0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f95d80017e0 tx=0x7f95d8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:30.527 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.528+0000 7f95dea00640 1 -- 172.21.15.121:0/2860760675 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f95d004c070 con 0x7f95e80b4f80 2024-04-20T10:21:30.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.767+0000 7f95e7e00640 1 -- 172.21.15.121:0/2860760675 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f95e815ef90 con 0x7f95e80b4f80 2024-04-20T10:21:30.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.769+0000 7f95dea00640 1 -- 172.21.15.121:0/2860760675 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5204 (secure 0 0 0) 0x7f95d007b420 con 0x7f95e80b4f80 2024-04-20T10:21:30.769 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:21:30.769 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1376743826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1376743826},{"type":"v1","addr":"172.21.15.123:6833","nonce":1376743826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/999167058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":999167058},{"type":"v1","addr":"172.21.15.121:6835","nonce":999167058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2160736935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2160736935},{"type":"v1","addr":"172.21.15.121:6837","nonce":2160736935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:21:02.048590+0000","modified":"2024-04-20T10:21:29.775269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6835/3336775250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3336775250},{"type":"v1","addr":"172.21.15.123:6835","nonce":3336775250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1}]} 2024-04-20T10:21:30.769 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 8 2024-04-20T10:21:30.771 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.772+0000 7f95e7e00640 1 -- 172.21.15.121:0/2860760675 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f95c406c920 msgr2=0x7f95c406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:30.772 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.772+0000 7f95e7e00640 1 --2- 172.21.15.121:0/2860760675 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f95c406c920 0x7f95c406ede0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f95d80017e0 tx=0x7f95d8009040 comp rx=0 tx=0).stop 2024-04-20T10:21:30.772 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.772+0000 7f95e7e00640 1 -- 172.21.15.121:0/2860760675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95e80b4f80 msgr2=0x7f95e80c9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:30.772 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.772+0000 7f95e7e00640 1 --2- 172.21.15.121:0/2860760675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95e80b4f80 0x7f95e80c9f70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f95d0000c00 tx=0x7f95d0004730 comp rx=0 tx=0).stop 2024-04-20T10:21:30.772 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.772+0000 7f95e7e00640 1 -- 172.21.15.121:0/2860760675 shutdown_connections 2024-04-20T10:21:30.772 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.772+0000 7f95e7e00640 1 --2- 172.21.15.121:0/2860760675 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f95c406c920 0x7f95c406ede0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:30.772 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.772+0000 7f95e7e00640 1 --2- 172.21.15.121:0/2860760675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95e80b53e0 0x7f95e80ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:30.772 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.772+0000 7f95e7e00640 1 --2- 172.21.15.121:0/2860760675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f95e80b4f80 0x7f95e80c9f70 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:30.772 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.772+0000 7f95e7e00640 1 --2- 172.21.15.121:0/2860760675 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f95cc003fb0 0x7f95e814db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:30.772 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.772+0000 7f95e7e00640 1 -- 172.21.15.121:0/2860760675 >> 172.21.15.121:0/2860760675 conn(0x7f95e80d4450 msgr2=0x7f95e80b91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:30.772 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.772+0000 7f95e7e00640 1 -- 172.21.15.121:0/2860760675 shutdown_connections 2024-04-20T10:21:30.772 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.772+0000 7f95e7e00640 1 -- 172.21.15.121:0/2860760675 wait complete. 2024-04-20T10:21:30.790 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:21:30.985 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.986+0000 7f9182a00640 1 Processor -- start 2024-04-20T10:21:30.986 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.986+0000 7f9182a00640 1 -- start start 2024-04-20T10:21:30.989 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.987+0000 7f9182a00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f917c06dcb0 0x7f917c071b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:30.989 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.987+0000 7f9182a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f917c072090 0x7f917c0d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:30.989 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.987+0000 7f9182a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f917c0ade40 con 0x7f917c15b3b0 2024-04-20T10:21:30.989 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.987+0000 7f9182a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f917c06f1b0 con 0x7f917c072090 2024-04-20T10:21:30.989 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.987+0000 7f9182a00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f917c0ae0e0 con 0x7f917c06dcb0 2024-04-20T10:21:30.989 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.988+0000 7f917be00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f917c15b3b0 0x7f917c06eaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:49966/0 (socket says 172.21.15.121:49966) 2024-04-20T10:21:30.989 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.988+0000 7f917be00640 1 -- 172.21.15.121:0/3413200101 learned_addr learned my addr 172.21.15.121:0/3413200101 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:30.989 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.988+0000 7f9173400640 1 --2- 172.21.15.121:0/3413200101 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f917c06dcb0 0x7f917c071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:30.989 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.988+0000 7f9180a00640 1 --2- 172.21.15.121:0/3413200101 >> v2:172.21.15.123:3300/0 conn(0x7f917c072090 0x7f917c0d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:30.990 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.988+0000 7f917b400640 1 -- 172.21.15.121:0/3413200101 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 840903622 0 0) 0x7f917c0ade40 con 0x7f917c15b3b0 2024-04-20T10:21:30.990 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.988+0000 7f917b400640 1 -- 172.21.15.121:0/3413200101 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9160003520 con 0x7f917c15b3b0 2024-04-20T10:21:30.990 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.988+0000 7f917b400640 1 -- 172.21.15.121:0/3413200101 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2311246357 0 0) 0x7f9160003520 con 0x7f917c15b3b0 2024-04-20T10:21:30.990 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.988+0000 7f917b400640 1 -- 172.21.15.121:0/3413200101 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f917c06dcb0 msgr2=0x7f917c071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:30.990 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.988+0000 7f917b400640 1 --2- 172.21.15.121:0/3413200101 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f917c06dcb0 0x7f917c071b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:30.990 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.989+0000 7f917b400640 1 -- 172.21.15.121:0/3413200101 >> v2:172.21.15.123:3300/0 conn(0x7f917c072090 msgr2=0x7f917c0d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:30.990 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.989+0000 7f917b400640 1 --2- 172.21.15.121:0/3413200101 >> v2:172.21.15.123:3300/0 conn(0x7f917c072090 0x7f917c0d0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:30.990 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.989+0000 7f917b400640 1 -- 172.21.15.121:0/3413200101 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f917c15b790 con 0x7f917c15b3b0 2024-04-20T10:21:30.990 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.989+0000 7f917b400640 1 -- 172.21.15.121:0/3413200101 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f916c001690 con 0x7f917c15b3b0 2024-04-20T10:21:30.990 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.989+0000 7f917b400640 1 -- 172.21.15.121:0/3413200101 >> v1:172.21.15.121:6789/0 conn(0x7f917c15b3b0 legacy=0x7f917c06eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:30.990 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.989+0000 7f917b400640 1 --2- 172.21.15.121:0/3413200101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91600039b0 0x7f9160003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:30.990 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.989+0000 7f917b400640 1 --2- 172.21.15.121:0/3413200101 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f917c06dcb0 0x7f9160004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:30.990 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.989+0000 7f917b400640 1 --2- 172.21.15.121:0/3413200101 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9160004d40 0x7f91600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:30.991 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.989+0000 7f917b400640 1 -- 172.21.15.121:0/3413200101 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f917c06f1b0 con 0x7f9160004d40 2024-04-20T10:21:30.991 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.989+0000 7f917b400640 1 -- 172.21.15.121:0/3413200101 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f917c0ae0e0 con 0x7f91600039b0 2024-04-20T10:21:30.991 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.989+0000 7f917b400640 1 -- 172.21.15.121:0/3413200101 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f916000dbd0 con 0x7f917c06dcb0 2024-04-20T10:21:30.991 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.989+0000 7f9180a00640 1 --2- 172.21.15.121:0/3413200101 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9160004d40 0x7f91600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:30.991 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.989+0000 7f917be00640 1 --2- 172.21.15.121:0/3413200101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91600039b0 0x7f9160003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:30.991 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.989+0000 7f9173400640 1 --2- 172.21.15.121:0/3413200101 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f917c06dcb0 0x7f9160004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:30.991 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.989+0000 7f9180a00640 1 -- 172.21.15.121:0/3413200101 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f917c06dcb0 msgr2=0x7f9160004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:30.991 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.989+0000 7f9180a00640 1 --2- 172.21.15.121:0/3413200101 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f917c06dcb0 0x7f9160004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:30.991 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.989+0000 7f9180a00640 1 -- 172.21.15.121:0/3413200101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91600039b0 msgr2=0x7f9160003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:30.991 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.989+0000 7f9180a00640 1 --2- 172.21.15.121:0/3413200101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91600039b0 0x7f9160003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:30.991 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.989+0000 7f9180a00640 1 -- 172.21.15.121:0/3413200101 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9160003520 con 0x7f9160004d40 2024-04-20T10:21:30.991 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.989+0000 7f917be00640 1 --2- 172.21.15.121:0/3413200101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91600039b0 0x7f9160003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:30.991 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.989+0000 7f9173400640 1 --2- 172.21.15.121:0/3413200101 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f917c06dcb0 0x7f9160004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:30.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.990+0000 7f9180a00640 1 --2- 172.21.15.121:0/3413200101 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9160004d40 0x7f91600051c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9174009f90 tx=0x7f9160005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=574cda9010103dc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:30.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.991+0000 7f917b400640 1 -- 172.21.15.121:0/3413200101 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9174039070 con 0x7f9160004d40 2024-04-20T10:21:30.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.991+0000 7f917b400640 1 -- 172.21.15.121:0/3413200101 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f917402cc20 con 0x7f9160004d40 2024-04-20T10:21:30.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.991+0000 7f917b400640 1 -- 172.21.15.121:0/3413200101 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9174009c10 con 0x7f9160004d40 2024-04-20T10:21:30.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.991+0000 7f9182a00640 1 -- 172.21.15.121:0/3413200101 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9160004d40 msgr2=0x7f91600051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:30.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.991+0000 7f9182a00640 1 --2- 172.21.15.121:0/3413200101 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9160004d40 0x7f91600051c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9174009f90 tx=0x7f9160005830 comp rx=0 tx=0).stop 2024-04-20T10:21:30.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.991+0000 7f9180a00640 1 -- 172.21.15.121:0/3413200101 reap_dead start 2024-04-20T10:21:30.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.991+0000 7f9182a00640 1 -- 172.21.15.121:0/3413200101 shutdown_connections 2024-04-20T10:21:30.994 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.991+0000 7f9182a00640 1 -- 172.21.15.121:0/3413200101 >> 172.21.15.121:0/3413200101 conn(0x7f917c0b3350 msgr2=0x7f917c0b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:30.994 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.991+0000 7f9182a00640 1 -- 172.21.15.121:0/3413200101 shutdown_connections 2024-04-20T10:21:30.994 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.992+0000 7f9182a00640 1 -- 172.21.15.121:0/3413200101 wait complete. 2024-04-20T10:21:30.994 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.993+0000 7f9182a00640 1 Processor -- start 2024-04-20T10:21:30.994 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.993+0000 7f9182a00640 1 -- start start 2024-04-20T10:21:30.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.994+0000 7f9182a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9160004d40 0x7f917c15c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:30.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.994+0000 7f9182a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f917c1543d0 0x7f917c1506a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:30.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.994+0000 7f9182a00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f917c151ea0 0x7f917c152330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:30.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.994+0000 7f9182a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f917c0ae940 con 0x7f917c1543d0 2024-04-20T10:21:30.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.994+0000 7f9182a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f917c0ae6a0 con 0x7f9160004d40 2024-04-20T10:21:30.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.994+0000 7f9182a00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f917c0aec20 con 0x7f917c151ea0 2024-04-20T10:21:30.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.995+0000 7f9173400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f917c1543d0 0x7f917c1506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:30.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.995+0000 7f9173400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f917c1543d0 0x7f917c1506a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54140/0 (socket says 172.21.15.121:54140) 2024-04-20T10:21:30.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.995+0000 7f9173400640 1 -- 172.21.15.121:0/4267871166 learned_addr learned my addr 172.21.15.121:0/4267871166 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:30.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.995+0000 7f917be00640 1 --2- 172.21.15.121:0/4267871166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9160004d40 0x7f917c15c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:30.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.995+0000 7f9180a00640 1 --2- 172.21.15.121:0/4267871166 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f917c151ea0 0x7f917c152330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:30.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.995+0000 7f9173400640 1 -- 172.21.15.121:0/4267871166 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f917c151ea0 msgr2=0x7f917c152330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:30.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.995+0000 7f9173400640 1 --2- 172.21.15.121:0/4267871166 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f917c151ea0 0x7f917c152330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:30.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.995+0000 7f9173400640 1 -- 172.21.15.121:0/4267871166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9160004d40 msgr2=0x7f917c15c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:30.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.995+0000 7f9173400640 1 --2- 172.21.15.121:0/4267871166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9160004d40 0x7f917c15c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:30.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.995+0000 7f9173400640 1 -- 172.21.15.121:0/4267871166 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f917c15ccf0 con 0x7f917c1543d0 2024-04-20T10:21:30.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.996+0000 7f9173400640 1 --2- 172.21.15.121:0/4267871166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f917c1543d0 0x7f917c1506a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f91640108e0 tx=0x7f9164010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:30.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.996+0000 7f9178c00640 1 -- 172.21.15.121:0/4267871166 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916400f040 con 0x7f917c1543d0 2024-04-20T10:21:30.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.996+0000 7f9178c00640 1 -- 172.21.15.121:0/4267871166 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f916400ec20 con 0x7f917c1543d0 2024-04-20T10:21:30.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.996+0000 7f9178c00640 1 -- 172.21.15.121:0/4267871166 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916401c400 con 0x7f917c1543d0 2024-04-20T10:21:30.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.996+0000 7f9182a00640 1 -- 172.21.15.121:0/4267871166 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f917c15cfd0 con 0x7f917c1543d0 2024-04-20T10:21:30.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.996+0000 7f9182a00640 1 -- 172.21.15.121:0/4267871166 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f917c15d870 con 0x7f917c1543d0 2024-04-20T10:21:30.998 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.997+0000 7f9182a00640 1 -- 172.21.15.121:0/4267871166 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f917c071c70 con 0x7f917c1543d0 2024-04-20T10:21:30.998 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.998+0000 7f9178c00640 1 -- 172.21.15.121:0/4267871166 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f916401c5a0 con 0x7f917c1543d0 2024-04-20T10:21:30.998 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.999+0000 7f9178c00640 1 --2- 172.21.15.121:0/4267871166 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f915806c8f0 0x7f915806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:30.999 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.999+0000 7f9178c00640 1 -- 172.21.15.121:0/4267871166 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f916408c750 con 0x7f917c1543d0 2024-04-20T10:21:30.999 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:30.999+0000 7f917be00640 1 --2- 172.21.15.121:0/4267871166 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f915806c8f0 0x7f915806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:30.999 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.000+0000 7f917be00640 1 --2- 172.21.15.121:0/4267871166 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f915806c8f0 0x7f915806edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f916c001800 tx=0x7f916c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:31.003 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.004+0000 7f9178c00640 1 -- 172.21.15.121:0/4267871166 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f917c071c70 con 0x7f917c1543d0 2024-04-20T10:21:31.206 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.207+0000 7f9182a00640 1 -- 172.21.15.121:0/4267871166 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f917c068980 con 0x7f917c1543d0 2024-04-20T10:21:31.210 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.210+0000 7f9178c00640 1 -- 172.21.15.121:0/4267871166 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7f9164018030 con 0x7f917c1543d0 2024-04-20T10:21:31.210 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:21:31.210 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":25,"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","created":"2024-04-20T10:20:38.076076+0000","modified":"2024-04-20T10:21:07.336542+0000","last_up_change":"2024-04-20T10:20:50.162700+0000","last_in_change":"2024-04-20T10:20:41.510382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:20:51.141365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:21:00.436937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:21:01.026957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:21:04.862693+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"629a7b8c-a40a-4a2a-a3cc-d9f357774310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6809","nonce":1044588835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6811","nonce":1044588835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6815","nonce":1044588835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6813","nonce":1044588835}]},"public_addr":"172.21.15.121:6809/1044588835","cluster_addr":"172.21.15.121:6811/1044588835","heartbeat_back_addr":"172.21.15.121:6815/1044588835","heartbeat_front_addr":"172.21.15.121:6813/1044588835","state":["exists","up"]},{"osd":1,"uuid":"f2570399-aa50-4ff3-bf5d-3d09207290db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6801","nonce":3505199624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6803","nonce":3505199624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6807","nonce":3505199624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6805","nonce":3505199624}]},"public_addr":"172.21.15.121:6801/3505199624","cluster_addr":"172.21.15.121:6803/3505199624","heartbeat_back_addr":"172.21.15.121:6807/3505199624","heartbeat_front_addr":"172.21.15.121:6805/3505199624","state":["exists","up"]},{"osd":2,"uuid":"50893807-dc30-49f4-b41f-739366626658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6817","nonce":3656724207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6819","nonce":3656724207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6827","nonce":3656724207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6822","nonce":3656724207}]},"public_addr":"172.21.15.121:6817/3656724207","cluster_addr":"172.21.15.121:6819/3656724207","heartbeat_back_addr":"172.21.15.121:6827/3656724207","heartbeat_front_addr":"172.21.15.121:6822/3656724207","state":["exists","up"]},{"osd":3,"uuid":"dadb01bc-5491-4ffc-8824-d48a057240b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6823","nonce":2232342832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6826","nonce":2232342832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6831","nonce":2232342832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6829","nonce":2232342832}]},"public_addr":"172.21.15.121:6823/2232342832","cluster_addr":"172.21.15.121:6826/2232342832","heartbeat_back_addr":"172.21.15.121:6831/2232342832","heartbeat_front_addr":"172.21.15.121:6829/2232342832","state":["exists","up"]},{"osd":4,"uuid":"b3f3b4a1-b375-4526-b69e-344ab24c2da1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6801","nonce":99353111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6803","nonce":99353111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6807","nonce":99353111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6805","nonce":99353111}]},"public_addr":"172.21.15.123:6801/99353111","cluster_addr":"172.21.15.123:6803/99353111","heartbeat_back_addr":"172.21.15.123:6807/99353111","heartbeat_front_addr":"172.21.15.123:6805/99353111","state":["exists","up"]},{"osd":5,"uuid":"2ee24a08-1624-4961-9642-d7b510282484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6818","nonce":2504374688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6822","nonce":2504374688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6830","nonce":2504374688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6826","nonce":2504374688}]},"public_addr":"172.21.15.123:6818/2504374688","cluster_addr":"172.21.15.123:6822/2504374688","heartbeat_back_addr":"172.21.15.123:6830/2504374688","heartbeat_front_addr":"172.21.15.123:6826/2504374688","state":["exists","up"]},{"osd":6,"uuid":"186118cc-6128-49de-acca-55260945c1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6819","nonce":4139299599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6823","nonce":4139299599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6829","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6831","nonce":4139299599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6827","nonce":4139299599}]},"public_addr":"172.21.15.123:6819/4139299599","cluster_addr":"172.21.15.123:6823/4139299599","heartbeat_back_addr":"172.21.15.123:6831/4139299599","heartbeat_front_addr":"172.21.15.123:6827/4139299599","state":["exists","up"]},{"osd":7,"uuid":"a00aac6e-4bd9-415a-a93d-ae85210865ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6809","nonce":4043981069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6811","nonce":4043981069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6815","nonce":4043981069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6813","nonce":4043981069}]},"public_addr":"172.21.15.123:6809/4043981069","cluster_addr":"172.21.15.123:6811/4043981069","heartbeat_back_addr":"172.21.15.123:6815/4043981069","heartbeat_front_addr":"172.21.15.123:6813/4043981069","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.904540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.958585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.616367+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.885244+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.103127+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.167248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.884348+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.862356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:21:31.213 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.213+0000 7f9182a00640 1 -- 172.21.15.121:0/4267871166 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f915806c8f0 msgr2=0x7f915806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:31.213 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.213+0000 7f9182a00640 1 --2- 172.21.15.121:0/4267871166 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f915806c8f0 0x7f915806edb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f916c001800 tx=0x7f916c009040 comp rx=0 tx=0).stop 2024-04-20T10:21:31.213 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.213+0000 7f9182a00640 1 -- 172.21.15.121:0/4267871166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f917c1543d0 msgr2=0x7f917c1506a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:31.213 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.213+0000 7f9182a00640 1 --2- 172.21.15.121:0/4267871166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f917c1543d0 0x7f917c1506a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f91640108e0 tx=0x7f9164010da0 comp rx=0 tx=0).stop 2024-04-20T10:21:31.213 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.213+0000 7f9182a00640 1 -- 172.21.15.121:0/4267871166 shutdown_connections 2024-04-20T10:21:31.213 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.213+0000 7f9182a00640 1 --2- 172.21.15.121:0/4267871166 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f917c151ea0 0x7f917c152330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:31.213 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.213+0000 7f9182a00640 1 --2- 172.21.15.121:0/4267871166 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f917c1543d0 0x7f917c1506a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:31.213 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.213+0000 7f9182a00640 1 --2- 172.21.15.121:0/4267871166 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f915806c8f0 0x7f915806edb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:31.213 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.213+0000 7f9182a00640 1 --2- 172.21.15.121:0/4267871166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9160004d40 0x7f917c15c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:31.214 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.213+0000 7f9182a00640 1 -- 172.21.15.121:0/4267871166 >> 172.21.15.121:0/4267871166 conn(0x7f917c0b3350 msgr2=0x7f917c15ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:31.214 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.213+0000 7f9182a00640 1 -- 172.21.15.121:0/4267871166 shutdown_connections 2024-04-20T10:21:31.214 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.213+0000 7f9182a00640 1 -- 172.21.15.121:0/4267871166 wait complete. 2024-04-20T10:21:31.232 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-20T10:21:31.232 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T10:21:31.434 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.434+0000 7fdb37000640 1 Processor -- start 2024-04-20T10:21:31.434 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.435+0000 7fdb37000640 1 -- start start 2024-04-20T10:21:31.436 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.436+0000 7fdb37000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdb3015e550 0x7fdb300d06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:31.436 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.436+0000 7fdb37000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdb300d4380 0x7fdb300d4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:31.436 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.436+0000 7fdb37000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb300cf950 con 0x7fdb301621d0 2024-04-20T10:21:31.436 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.436+0000 7fdb37000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fdb30162e90 con 0x7fdb300d4380 2024-04-20T10:21:31.436 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.436+0000 7fdb37000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fdb300cfbf0 con 0x7fdb3015e550 2024-04-20T10:21:31.436 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.436+0000 7fdb35200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdb300d4380 0x7fdb300d4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:31.436 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.436+0000 7fdb35200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdb300d4380 0x7fdb300d4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54146/0 (socket says 172.21.15.121:54146) 2024-04-20T10:21:31.436 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.436+0000 7fdb35200640 1 -- 172.21.15.121:0/3751506100 learned_addr learned my addr 172.21.15.121:0/3751506100 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:31.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.437+0000 7fdb35200640 1 -- 172.21.15.121:0/3751506100 >> v2:172.21.15.123:3300/0 conn(0x7fdb3015e550 msgr2=0x7fdb300d06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:31.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.437+0000 7fdb35200640 1 --2- 172.21.15.121:0/3751506100 >> v2:172.21.15.123:3300/0 conn(0x7fdb3015e550 0x7fdb300d06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:31.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.437+0000 7fdb35200640 1 -- 172.21.15.121:0/3751506100 >> v1:172.21.15.123:6789/0 conn(0x7fdb301621d0 legacy=0x7fdb301625e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:31.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.437+0000 7fdb35200640 1 -- 172.21.15.121:0/3751506100 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb300cfed0 con 0x7fdb300d4380 2024-04-20T10:21:31.438 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.438+0000 7fdb35200640 1 --2- 172.21.15.121:0/3751506100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb300d4380 0x7fdb300d4810 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdb300b9680 tx=0x7fdb2002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1a51e6c387addc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:31.438 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.438+0000 7fdb2fe00640 1 -- 172.21.15.121:0/3751506100 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2002c9e0 con 0x7fdb300d4380 2024-04-20T10:21:31.439 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.438+0000 7fdb2fe00640 1 -- 172.21.15.121:0/3751506100 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb2002cb80 con 0x7fdb300d4380 2024-04-20T10:21:31.439 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.438+0000 7fdb2fe00640 1 -- 172.21.15.121:0/3751506100 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb200343f0 con 0x7fdb300d4380 2024-04-20T10:21:31.439 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.439+0000 7fdb37000640 1 -- 172.21.15.121:0/3751506100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb300d4380 msgr2=0x7fdb300d4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:31.439 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.439+0000 7fdb37000640 1 --2- 172.21.15.121:0/3751506100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb300d4380 0x7fdb300d4810 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdb300b9680 tx=0x7fdb2002be40 comp rx=0 tx=0).stop 2024-04-20T10:21:31.439 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.439+0000 7fdb37000640 1 -- 172.21.15.121:0/3751506100 shutdown_connections 2024-04-20T10:21:31.439 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.439+0000 7fdb37000640 1 --2- 172.21.15.121:0/3751506100 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb300d4380 0x7fdb300d4810 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:31.439 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.439+0000 7fdb37000640 1 --2- 172.21.15.121:0/3751506100 >> v2:172.21.15.123:3300/0 conn(0x7fdb3015e550 0x7fdb300d06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:31.439 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.439+0000 7fdb37000640 1 -- 172.21.15.121:0/3751506100 >> 172.21.15.121:0/3751506100 conn(0x7fdb3008bf80 msgr2=0x7fdb3009d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:31.439 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.439+0000 7fdb37000640 1 -- 172.21.15.121:0/3751506100 shutdown_connections 2024-04-20T10:21:31.439 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.439+0000 7fdb37000640 1 -- 172.21.15.121:0/3751506100 wait complete. 2024-04-20T10:21:31.440 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.441+0000 7fdb37000640 1 Processor -- start 2024-04-20T10:21:31.440 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.441+0000 7fdb37000640 1 -- start start 2024-04-20T10:21:31.443 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.442+0000 7fdb37000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb300d4380 0x7fdb301f1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:31.443 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.442+0000 7fdb37000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb3015e550 0x7fdb3006d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:31.443 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.442+0000 7fdb37000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb301621d0 0x7fdb3006d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:31.443 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.442+0000 7fdb37000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3015eab0 con 0x7fdb301621d0 2024-04-20T10:21:31.443 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.442+0000 7fdb37000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3015e930 con 0x7fdb3015e550 2024-04-20T10:21:31.443 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.442+0000 7fdb37000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fdb300d1fc0 con 0x7fdb300d4380 2024-04-20T10:21:31.443 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.442+0000 7fdb36600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb301621d0 0x7fdb3006d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:31.443 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.442+0000 7fdb36600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb301621d0 0x7fdb3006d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54156/0 (socket says 172.21.15.121:54156) 2024-04-20T10:21:31.443 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.442+0000 7fdb36600640 1 -- 172.21.15.121:0/696743954 learned_addr learned my addr 172.21.15.121:0/696743954 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:31.443 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.442+0000 7fdb36600640 1 -- 172.21.15.121:0/696743954 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb300d4380 msgr2=0x7fdb301f1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:31.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.442+0000 7fdb35c00640 1 --2- 172.21.15.121:0/696743954 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb300d4380 0x7fdb301f1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:31.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.442+0000 7fdb35200640 1 --2- 172.21.15.121:0/696743954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb3015e550 0x7fdb3006d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:31.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.442+0000 7fdb36600640 1 --2- 172.21.15.121:0/696743954 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb300d4380 0x7fdb301f1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:31.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.443+0000 7fdb36600640 1 -- 172.21.15.121:0/696743954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb3015e550 msgr2=0x7fdb3006d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:31.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.443+0000 7fdb36600640 1 --2- 172.21.15.121:0/696743954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb3015e550 0x7fdb3006d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:31.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.443+0000 7fdb36600640 1 -- 172.21.15.121:0/696743954 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb301f1ec0 con 0x7fdb301621d0 2024-04-20T10:21:31.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.443+0000 7fdb35c00640 1 --2- 172.21.15.121:0/696743954 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb300d4380 0x7fdb301f1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:31.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.443+0000 7fdb36600640 1 --2- 172.21.15.121:0/696743954 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb301621d0 0x7fdb3006d560 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdb2800b4c0 tx=0x7fdb2800b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:31.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.443+0000 7fdb2d600640 1 -- 172.21.15.121:0/696743954 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb28015020 con 0x7fdb301621d0 2024-04-20T10:21:31.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.443+0000 7fdb2d600640 1 -- 172.21.15.121:0/696743954 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb28011850 con 0x7fdb301621d0 2024-04-20T10:21:31.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.443+0000 7fdb2d600640 1 -- 172.21.15.121:0/696743954 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb28011c40 con 0x7fdb301621d0 2024-04-20T10:21:31.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.443+0000 7fdb37000640 1 -- 172.21.15.121:0/696743954 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb301f5190 con 0x7fdb301621d0 2024-04-20T10:21:31.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.443+0000 7fdb37000640 1 -- 172.21.15.121:0/696743954 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb301f7f40 con 0x7fdb301621d0 2024-04-20T10:21:31.445 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.444+0000 7fdb37000640 1 -- 172.21.15.121:0/696743954 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb30075b30 con 0x7fdb301621d0 2024-04-20T10:21:31.445 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.445+0000 7fdb2d600640 1 -- 172.21.15.121:0/696743954 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdb28018020 con 0x7fdb301621d0 2024-04-20T10:21:31.447 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.447+0000 7fdb2d600640 1 --2- 172.21.15.121:0/696743954 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fdb1406c8f0 0x7fdb1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:31.447 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.448+0000 7fdb2d600640 1 -- 172.21.15.121:0/696743954 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fdb2808bd70 con 0x7fdb301621d0 2024-04-20T10:21:31.447 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.448+0000 7fdb35c00640 1 --2- 172.21.15.121:0/696743954 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fdb1406c8f0 0x7fdb1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:31.447 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.448+0000 7fdb35c00640 1 --2- 172.21.15.121:0/696743954 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fdb1406c8f0 0x7fdb1406edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdb24006fd0 tx=0x7fdb24008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:31.451 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.452+0000 7fdb2d600640 1 -- 172.21.15.121:0/696743954 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdb2801f020 con 0x7fdb301621d0 2024-04-20T10:21:31.649 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.650+0000 7fdb37000640 1 -- 172.21.15.121:0/696743954 --> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdb301f5470 con 0x7fdb1406c8f0 2024-04-20T10:21:31.661 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.662+0000 7fdb2d600640 1 -- 172.21.15.121:0/696743954 <== mgr.4103 v2:172.21.15.121:6832/1408233272 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332538 (secure 0 0 0) 0x7fdb301f5470 con 0x7fdb1406c8f0 2024-04-20T10:21:31.664 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:21:31.668 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-04-20T10:21:31.670 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.670+0000 7fdb37000640 1 -- 172.21.15.121:0/696743954 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fdb1406c8f0 msgr2=0x7fdb1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:31.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.670+0000 7fdb37000640 1 --2- 172.21.15.121:0/696743954 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fdb1406c8f0 0x7fdb1406edb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdb24006fd0 tx=0x7fdb24008040 comp rx=0 tx=0).stop 2024-04-20T10:21:31.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.670+0000 7fdb37000640 1 -- 172.21.15.121:0/696743954 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb301621d0 msgr2=0x7fdb3006d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:31.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.670+0000 7fdb37000640 1 --2- 172.21.15.121:0/696743954 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb301621d0 0x7fdb3006d560 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdb2800b4c0 tx=0x7fdb2800b980 comp rx=0 tx=0).stop 2024-04-20T10:21:31.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.670+0000 7fdb37000640 1 -- 172.21.15.121:0/696743954 shutdown_connections 2024-04-20T10:21:31.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.670+0000 7fdb37000640 1 --2- 172.21.15.121:0/696743954 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb301621d0 0x7fdb3006d560 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:31.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.670+0000 7fdb37000640 1 --2- 172.21.15.121:0/696743954 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fdb1406c8f0 0x7fdb1406edb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:31.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.670+0000 7fdb37000640 1 --2- 172.21.15.121:0/696743954 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb3015e550 0x7fdb3006d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:31.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.670+0000 7fdb37000640 1 --2- 172.21.15.121:0/696743954 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb300d4380 0x7fdb301f1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:31.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.670+0000 7fdb37000640 1 -- 172.21.15.121:0/696743954 >> 172.21.15.121:0/696743954 conn(0x7fdb3008bf80 msgr2=0x7fdb3009d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:31.672 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.671+0000 7fdb37000640 1 -- 172.21.15.121:0/696743954 shutdown_connections 2024-04-20T10:21:31.672 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.671+0000 7fdb37000640 1 -- 172.21.15.121:0/696743954 wait complete. 2024-04-20T10:21:31.688 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-04-20T10:21:31.118017+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219436,"kb_used_data":3272,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755284,"statfs":{"total":773094113280,"available":772869410816,"internally_reserved":0,"allocated":3350528,"data_stored":2018488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004173"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.544361+0000","last_change":"2024-04-20T10:21:01.620269+0000","last_active":"2024-04-20T10:21:08.544361+0000","last_peered":"2024-04-20T10:21:08.544361+0000","last_clean":"2024-04-20T10:21:08.544361+0000","last_became_active":"2024-04-20T10:21:01.619751+0000","last_became_peered":"2024-04-20T10:21:01.619751+0000","last_unstale":"2024-04-20T10:21:08.544361+0000","last_undegraded":"2024-04-20T10:21:08.544361+0000","last_fullsized":"2024-04-20T10:21:08.544361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:28:01.100842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.544942+0000","last_change":"2024-04-20T10:21:02.642147+0000","last_active":"2024-04-20T10:21:08.544942+0000","last_peered":"2024-04-20T10:21:08.544942+0000","last_clean":"2024-04-20T10:21:08.544942+0000","last_became_active":"2024-04-20T10:21:02.641636+0000","last_became_peered":"2024-04-20T10:21:02.641636+0000","last_unstale":"2024-04-20T10:21:08.544942+0000","last_undegraded":"2024-04-20T10:21:08.544942+0000","last_fullsized":"2024-04-20T10:21:08.544942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:53:57.418208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540582+0000","last_change":"2024-04-20T10:21:06.909318+0000","last_active":"2024-04-20T10:21:08.540582+0000","last_peered":"2024-04-20T10:21:08.540582+0000","last_clean":"2024-04-20T10:21:08.540582+0000","last_became_active":"2024-04-20T10:21:06.907900+0000","last_became_peered":"2024-04-20T10:21:06.907900+0000","last_unstale":"2024-04-20T10:21:08.540582+0000","last_undegraded":"2024-04-20T10:21:08.540582+0000","last_fullsized":"2024-04-20T10:21:08.540582+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:35:58.510637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.545148+0000","last_change":"2024-04-20T10:21:01.609344+0000","last_active":"2024-04-20T10:21:08.545148+0000","last_peered":"2024-04-20T10:21:08.545148+0000","last_clean":"2024-04-20T10:21:08.545148+0000","last_became_active":"2024-04-20T10:21:01.608863+0000","last_became_peered":"2024-04-20T10:21:01.608863+0000","last_unstale":"2024-04-20T10:21:08.545148+0000","last_undegraded":"2024-04-20T10:21:08.545148+0000","last_fullsized":"2024-04-20T10:21:08.545148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:32:02.981747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.543880+0000","last_change":"2024-04-20T10:21:02.649607+0000","last_active":"2024-04-20T10:21:08.543880+0000","last_peered":"2024-04-20T10:21:08.543880+0000","last_clean":"2024-04-20T10:21:08.543880+0000","last_became_active":"2024-04-20T10:21:02.649283+0000","last_became_peered":"2024-04-20T10:21:02.649283+0000","last_unstale":"2024-04-20T10:21:08.543880+0000","last_undegraded":"2024-04-20T10:21:08.543880+0000","last_fullsized":"2024-04-20T10:21:08.543880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:06:35.349863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.885288+0000","last_change":"2024-04-20T10:21:06.910300+0000","last_active":"2024-04-20T10:21:07.885288+0000","last_peered":"2024-04-20T10:21:07.885288+0000","last_clean":"2024-04-20T10:21:07.885288+0000","last_became_active":"2024-04-20T10:21:06.909341+0000","last_became_peered":"2024-04-20T10:21:06.909341+0000","last_unstale":"2024-04-20T10:21:07.885288+0000","last_undegraded":"2024-04-20T10:21:07.885288+0000","last_fullsized":"2024-04-20T10:21:07.885288+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:06:39.624891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.391510+0000","last_change":"2024-04-20T10:21:01.623714+0000","last_active":"2024-04-20T10:21:08.391510+0000","last_peered":"2024-04-20T10:21:08.391510+0000","last_clean":"2024-04-20T10:21:08.391510+0000","last_became_active":"2024-04-20T10:21:01.623427+0000","last_became_peered":"2024-04-20T10:21:01.623427+0000","last_unstale":"2024-04-20T10:21:08.391510+0000","last_undegraded":"2024-04-20T10:21:08.391510+0000","last_fullsized":"2024-04-20T10:21:08.391510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:25:59.939855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541788+0000","last_change":"2024-04-20T10:21:02.636522+0000","last_active":"2024-04-20T10:21:08.541788+0000","last_peered":"2024-04-20T10:21:08.541788+0000","last_clean":"2024-04-20T10:21:08.541788+0000","last_became_active":"2024-04-20T10:21:02.633235+0000","last_became_peered":"2024-04-20T10:21:02.633235+0000","last_unstale":"2024-04-20T10:21:08.541788+0000","last_undegraded":"2024-04-20T10:21:08.541788+0000","last_fullsized":"2024-04-20T10:21:08.541788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:55:00.905038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.886695+0000","last_change":"2024-04-20T10:21:06.911285+0000","last_active":"2024-04-20T10:21:07.886695+0000","last_peered":"2024-04-20T10:21:07.886695+0000","last_clean":"2024-04-20T10:21:07.886695+0000","last_became_active":"2024-04-20T10:21:06.908548+0000","last_became_peered":"2024-04-20T10:21:06.908548+0000","last_unstale":"2024-04-20T10:21:07.886695+0000","last_undegraded":"2024-04-20T10:21:07.886695+0000","last_fullsized":"2024-04-20T10:21:07.886695+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:09:46.241216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.391163+0000","last_change":"2024-04-20T10:21:01.626685+0000","last_active":"2024-04-20T10:21:08.391163+0000","last_peered":"2024-04-20T10:21:08.391163+0000","last_clean":"2024-04-20T10:21:08.391163+0000","last_became_active":"2024-04-20T10:21:01.626354+0000","last_became_peered":"2024-04-20T10:21:01.626354+0000","last_unstale":"2024-04-20T10:21:08.391163+0000","last_undegraded":"2024-04-20T10:21:08.391163+0000","last_fullsized":"2024-04-20T10:21:08.391163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:07:50.284372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542097+0000","last_change":"2024-04-20T10:21:02.636800+0000","last_active":"2024-04-20T10:21:08.542097+0000","last_peered":"2024-04-20T10:21:08.542097+0000","last_clean":"2024-04-20T10:21:08.542097+0000","last_became_active":"2024-04-20T10:21:02.636523+0000","last_became_peered":"2024-04-20T10:21:02.636523+0000","last_unstale":"2024-04-20T10:21:08.542097+0000","last_undegraded":"2024-04-20T10:21:08.542097+0000","last_fullsized":"2024-04-20T10:21:08.542097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:31:07.363373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.885879+0000","last_change":"2024-04-20T10:21:06.893126+0000","last_active":"2024-04-20T10:21:07.885879+0000","last_peered":"2024-04-20T10:21:07.885879+0000","last_clean":"2024-04-20T10:21:07.885879+0000","last_became_active":"2024-04-20T10:21:06.892722+0000","last_became_peered":"2024-04-20T10:21:06.892722+0000","last_unstale":"2024-04-20T10:21:07.885879+0000","last_undegraded":"2024-04-20T10:21:07.885879+0000","last_fullsized":"2024-04-20T10:21:07.885879+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:10:19.481713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.038387+0000","last_change":"2024-04-20T10:21:02.633548+0000","last_active":"2024-04-20T10:21:08.038387+0000","last_peered":"2024-04-20T10:21:08.038387+0000","last_clean":"2024-04-20T10:21:08.038387+0000","last_became_active":"2024-04-20T10:21:02.631142+0000","last_became_peered":"2024-04-20T10:21:02.631142+0000","last_unstale":"2024-04-20T10:21:08.038387+0000","last_undegraded":"2024-04-20T10:21:08.038387+0000","last_fullsized":"2024-04-20T10:21:08.038387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:14:16.934710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.390972+0000","last_change":"2024-04-20T10:21:01.622399+0000","last_active":"2024-04-20T10:21:08.390972+0000","last_peered":"2024-04-20T10:21:08.390972+0000","last_clean":"2024-04-20T10:21:08.390972+0000","last_became_active":"2024-04-20T10:21:01.622072+0000","last_became_peered":"2024-04-20T10:21:01.622072+0000","last_unstale":"2024-04-20T10:21:08.390972+0000","last_undegraded":"2024-04-20T10:21:08.390972+0000","last_fullsized":"2024-04-20T10:21:08.390972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:35:02.394948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.887355+0000","last_change":"2024-04-20T10:21:06.897024+0000","last_active":"2024-04-20T10:21:07.887355+0000","last_peered":"2024-04-20T10:21:07.887355+0000","last_clean":"2024-04-20T10:21:07.887355+0000","last_became_active":"2024-04-20T10:21:06.896315+0000","last_became_peered":"2024-04-20T10:21:06.896315+0000","last_unstale":"2024-04-20T10:21:07.887355+0000","last_undegraded":"2024-04-20T10:21:07.887355+0000","last_fullsized":"2024-04-20T10:21:07.887355+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:21:29.812831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.544646+0000","last_change":"2024-04-20T10:21:02.646217+0000","last_active":"2024-04-20T10:21:08.544646+0000","last_peered":"2024-04-20T10:21:08.544646+0000","last_clean":"2024-04-20T10:21:08.544646+0000","last_became_active":"2024-04-20T10:21:02.645926+0000","last_became_peered":"2024-04-20T10:21:02.645926+0000","last_unstale":"2024-04-20T10:21:08.544646+0000","last_undegraded":"2024-04-20T10:21:08.544646+0000","last_fullsized":"2024-04-20T10:21:08.544646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:10:15.206686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441134+0000","last_change":"2024-04-20T10:21:01.617948+0000","last_active":"2024-04-20T10:21:08.441134+0000","last_peered":"2024-04-20T10:21:08.441134+0000","last_clean":"2024-04-20T10:21:08.441134+0000","last_became_active":"2024-04-20T10:21:01.617628+0000","last_became_peered":"2024-04-20T10:21:01.617628+0000","last_unstale":"2024-04-20T10:21:08.441134+0000","last_undegraded":"2024-04-20T10:21:08.441134+0000","last_fullsized":"2024-04-20T10:21:08.441134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:55:41.208744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.886801+0000","last_change":"2024-04-20T10:21:06.900164+0000","last_active":"2024-04-20T10:21:07.886801+0000","last_peered":"2024-04-20T10:21:07.886801+0000","last_clean":"2024-04-20T10:21:07.886801+0000","last_became_active":"2024-04-20T10:21:06.899783+0000","last_became_peered":"2024-04-20T10:21:06.899783+0000","last_unstale":"2024-04-20T10:21:07.886801+0000","last_undegraded":"2024-04-20T10:21:07.886801+0000","last_fullsized":"2024-04-20T10:21:07.886801+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:54:01.693236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541211+0000","last_change":"2024-04-20T10:21:02.629982+0000","last_active":"2024-04-20T10:21:08.541211+0000","last_peered":"2024-04-20T10:21:08.541211+0000","last_clean":"2024-04-20T10:21:08.541211+0000","last_became_active":"2024-04-20T10:21:02.629729+0000","last_became_peered":"2024-04-20T10:21:02.629729+0000","last_unstale":"2024-04-20T10:21:08.541211+0000","last_undegraded":"2024-04-20T10:21:08.541211+0000","last_fullsized":"2024-04-20T10:21:08.541211+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:31:53.442042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.440779+0000","last_change":"2024-04-20T10:21:01.604313+0000","last_active":"2024-04-20T10:21:08.440779+0000","last_peered":"2024-04-20T10:21:08.440779+0000","last_clean":"2024-04-20T10:21:08.440779+0000","last_became_active":"2024-04-20T10:21:01.603629+0000","last_became_peered":"2024-04-20T10:21:01.603629+0000","last_unstale":"2024-04-20T10:21:08.440779+0000","last_undegraded":"2024-04-20T10:21:08.440779+0000","last_fullsized":"2024-04-20T10:21:08.440779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:01:19.566351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441622+0000","last_change":"2024-04-20T10:21:06.903477+0000","last_active":"2024-04-20T10:21:08.441622+0000","last_peered":"2024-04-20T10:21:08.441622+0000","last_clean":"2024-04-20T10:21:08.441622+0000","last_became_active":"2024-04-20T10:21:06.903010+0000","last_became_peered":"2024-04-20T10:21:06.903010+0000","last_unstale":"2024-04-20T10:21:08.441622+0000","last_undegraded":"2024-04-20T10:21:08.441622+0000","last_fullsized":"2024-04-20T10:21:08.441622+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:46:45.448591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.885935+0000","last_change":"2024-04-20T10:21:02.614476+0000","last_active":"2024-04-20T10:21:07.885935+0000","last_peered":"2024-04-20T10:21:07.885935+0000","last_clean":"2024-04-20T10:21:07.885935+0000","last_became_active":"2024-04-20T10:21:02.614140+0000","last_became_peered":"2024-04-20T10:21:02.614140+0000","last_unstale":"2024-04-20T10:21:07.885935+0000","last_undegraded":"2024-04-20T10:21:07.885935+0000","last_fullsized":"2024-04-20T10:21:07.885935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:01:20.579875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.389864+0000","last_change":"2024-04-20T10:21:01.619958+0000","last_active":"2024-04-20T10:21:08.389864+0000","last_peered":"2024-04-20T10:21:08.389864+0000","last_clean":"2024-04-20T10:21:08.389864+0000","last_became_active":"2024-04-20T10:21:01.619688+0000","last_became_peered":"2024-04-20T10:21:01.619688+0000","last_unstale":"2024-04-20T10:21:08.389864+0000","last_undegraded":"2024-04-20T10:21:08.389864+0000","last_fullsized":"2024-04-20T10:21:08.389864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:45:51.165353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.887140+0000","last_change":"2024-04-20T10:21:06.908789+0000","last_active":"2024-04-20T10:21:07.887140+0000","last_peered":"2024-04-20T10:21:07.887140+0000","last_clean":"2024-04-20T10:21:07.887140+0000","last_became_active":"2024-04-20T10:21:06.908405+0000","last_became_peered":"2024-04-20T10:21:06.908405+0000","last_unstale":"2024-04-20T10:21:07.887140+0000","last_undegraded":"2024-04-20T10:21:07.887140+0000","last_fullsized":"2024-04-20T10:21:07.887140+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:02:30.991758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.544406+0000","last_change":"2024-04-20T10:21:02.649177+0000","last_active":"2024-04-20T10:21:08.544406+0000","last_peered":"2024-04-20T10:21:08.544406+0000","last_clean":"2024-04-20T10:21:08.544406+0000","last_became_active":"2024-04-20T10:21:02.648861+0000","last_became_peered":"2024-04-20T10:21:02.648861+0000","last_unstale":"2024-04-20T10:21:08.544406+0000","last_undegraded":"2024-04-20T10:21:08.544406+0000","last_fullsized":"2024-04-20T10:21:08.544406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:31:53.442042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.039602+0000","last_change":"2024-04-20T10:21:01.621228+0000","last_active":"2024-04-20T10:21:08.039602+0000","last_peered":"2024-04-20T10:21:08.039602+0000","last_clean":"2024-04-20T10:21:08.039602+0000","last_became_active":"2024-04-20T10:21:01.620525+0000","last_became_peered":"2024-04-20T10:21:01.620525+0000","last_unstale":"2024-04-20T10:21:08.039602+0000","last_undegraded":"2024-04-20T10:21:08.039602+0000","last_fullsized":"2024-04-20T10:21:08.039602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:50:27.716971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.885713+0000","last_change":"2024-04-20T10:21:06.903693+0000","last_active":"2024-04-20T10:21:07.885713+0000","last_peered":"2024-04-20T10:21:07.885713+0000","last_clean":"2024-04-20T10:21:07.885713+0000","last_became_active":"2024-04-20T10:21:06.903087+0000","last_became_peered":"2024-04-20T10:21:06.903087+0000","last_unstale":"2024-04-20T10:21:07.885713+0000","last_undegraded":"2024-04-20T10:21:07.885713+0000","last_fullsized":"2024-04-20T10:21:07.885713+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:22:40.880752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542691+0000","last_change":"2024-04-20T10:21:02.632360+0000","last_active":"2024-04-20T10:21:08.542691+0000","last_peered":"2024-04-20T10:21:08.542691+0000","last_clean":"2024-04-20T10:21:08.542691+0000","last_became_active":"2024-04-20T10:21:02.632062+0000","last_became_peered":"2024-04-20T10:21:02.632062+0000","last_unstale":"2024-04-20T10:21:08.542691+0000","last_undegraded":"2024-04-20T10:21:08.542691+0000","last_fullsized":"2024-04-20T10:21:08.542691+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:06:11.427561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441791+0000","last_change":"2024-04-20T10:21:01.622049+0000","last_active":"2024-04-20T10:21:08.441791+0000","last_peered":"2024-04-20T10:21:08.441791+0000","last_clean":"2024-04-20T10:21:08.441791+0000","last_became_active":"2024-04-20T10:21:01.621779+0000","last_became_peered":"2024-04-20T10:21:01.621779+0000","last_unstale":"2024-04-20T10:21:08.441791+0000","last_undegraded":"2024-04-20T10:21:08.441791+0000","last_fullsized":"2024-04-20T10:21:08.441791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:17:23.552169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540642+0000","last_change":"2024-04-20T10:21:06.908686+0000","last_active":"2024-04-20T10:21:08.540642+0000","last_peered":"2024-04-20T10:21:08.540642+0000","last_clean":"2024-04-20T10:21:08.540642+0000","last_became_active":"2024-04-20T10:21:06.907525+0000","last_became_peered":"2024-04-20T10:21:06.907525+0000","last_unstale":"2024-04-20T10:21:08.540642+0000","last_undegraded":"2024-04-20T10:21:08.540642+0000","last_fullsized":"2024-04-20T10:21:08.540642+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:38:05.651135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441452+0000","last_change":"2024-04-20T10:21:02.633193+0000","last_active":"2024-04-20T10:21:08.441452+0000","last_peered":"2024-04-20T10:21:08.441452+0000","last_clean":"2024-04-20T10:21:08.441452+0000","last_became_active":"2024-04-20T10:21:02.632896+0000","last_became_peered":"2024-04-20T10:21:02.632896+0000","last_unstale":"2024-04-20T10:21:08.441452+0000","last_undegraded":"2024-04-20T10:21:08.441452+0000","last_fullsized":"2024-04-20T10:21:08.441452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:06:35.349863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.391636+0000","last_change":"2024-04-20T10:21:01.624712+0000","last_active":"2024-04-20T10:21:08.391636+0000","last_peered":"2024-04-20T10:21:08.391636+0000","last_clean":"2024-04-20T10:21:08.391636+0000","last_became_active":"2024-04-20T10:21:01.624381+0000","last_became_peered":"2024-04-20T10:21:01.624381+0000","last_unstale":"2024-04-20T10:21:08.391636+0000","last_undegraded":"2024-04-20T10:21:08.391636+0000","last_fullsized":"2024-04-20T10:21:08.391636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:17:23.552169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.543354+0000","last_change":"2024-04-20T10:21:06.903399+0000","last_active":"2024-04-20T10:21:08.543354+0000","last_peered":"2024-04-20T10:21:08.543354+0000","last_clean":"2024-04-20T10:21:08.543354+0000","last_became_active":"2024-04-20T10:21:06.902529+0000","last_became_peered":"2024-04-20T10:21:06.902529+0000","last_unstale":"2024-04-20T10:21:08.543354+0000","last_undegraded":"2024-04-20T10:21:08.543354+0000","last_fullsized":"2024-04-20T10:21:08.543354+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:52:28.298927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542324+0000","last_change":"2024-04-20T10:21:02.637139+0000","last_active":"2024-04-20T10:21:08.542324+0000","last_peered":"2024-04-20T10:21:08.542324+0000","last_clean":"2024-04-20T10:21:08.542324+0000","last_became_active":"2024-04-20T10:21:02.636849+0000","last_became_peered":"2024-04-20T10:21:02.636849+0000","last_unstale":"2024-04-20T10:21:08.542324+0000","last_undegraded":"2024-04-20T10:21:08.542324+0000","last_fullsized":"2024-04-20T10:21:08.542324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:44:52.759993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441617+0000","last_change":"2024-04-20T10:21:01.606832+0000","last_active":"2024-04-20T10:21:08.441617+0000","last_peered":"2024-04-20T10:21:08.441617+0000","last_clean":"2024-04-20T10:21:08.441617+0000","last_became_active":"2024-04-20T10:21:01.606562+0000","last_became_peered":"2024-04-20T10:21:01.606562+0000","last_unstale":"2024-04-20T10:21:08.441617+0000","last_undegraded":"2024-04-20T10:21:08.441617+0000","last_fullsized":"2024-04-20T10:21:08.441617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:50:27.716971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.440686+0000","last_change":"2024-04-20T10:21:06.894216+0000","last_active":"2024-04-20T10:21:08.440686+0000","last_peered":"2024-04-20T10:21:08.440686+0000","last_clean":"2024-04-20T10:21:08.440686+0000","last_became_active":"2024-04-20T10:21:06.893762+0000","last_became_peered":"2024-04-20T10:21:06.893762+0000","last_unstale":"2024-04-20T10:21:08.440686+0000","last_undegraded":"2024-04-20T10:21:08.440686+0000","last_fullsized":"2024-04-20T10:21:08.440686+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:31:30.034892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.887359+0000","last_change":"2024-04-20T10:21:02.648574+0000","last_active":"2024-04-20T10:21:07.887359+0000","last_peered":"2024-04-20T10:21:07.887359+0000","last_clean":"2024-04-20T10:21:07.887359+0000","last_became_active":"2024-04-20T10:21:02.648275+0000","last_became_peered":"2024-04-20T10:21:02.648275+0000","last_unstale":"2024-04-20T10:21:07.887359+0000","last_undegraded":"2024-04-20T10:21:07.887359+0000","last_fullsized":"2024-04-20T10:21:07.887359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:55:42.222268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541444+0000","last_change":"2024-04-20T10:21:01.615460+0000","last_active":"2024-04-20T10:21:08.541444+0000","last_peered":"2024-04-20T10:21:08.541444+0000","last_clean":"2024-04-20T10:21:08.541444+0000","last_became_active":"2024-04-20T10:21:01.614405+0000","last_became_peered":"2024-04-20T10:21:01.614405+0000","last_unstale":"2024-04-20T10:21:08.541444+0000","last_undegraded":"2024-04-20T10:21:08.541444+0000","last_fullsized":"2024-04-20T10:21:08.541444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:51:46.790176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441597+0000","last_change":"2024-04-20T10:21:06.899961+0000","last_active":"2024-04-20T10:21:08.441597+0000","last_peered":"2024-04-20T10:21:08.441597+0000","last_clean":"2024-04-20T10:21:08.441597+0000","last_became_active":"2024-04-20T10:21:06.899574+0000","last_became_peered":"2024-04-20T10:21:06.899574+0000","last_unstale":"2024-04-20T10:21:08.441597+0000","last_undegraded":"2024-04-20T10:21:08.441597+0000","last_fullsized":"2024-04-20T10:21:08.441597+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:59:28.847178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.440949+0000","last_change":"2024-04-20T10:21:02.627320+0000","last_active":"2024-04-20T10:21:08.440949+0000","last_peered":"2024-04-20T10:21:08.440949+0000","last_clean":"2024-04-20T10:21:08.440949+0000","last_became_active":"2024-04-20T10:21:02.627008+0000","last_became_peered":"2024-04-20T10:21:02.627008+0000","last_unstale":"2024-04-20T10:21:08.440949+0000","last_undegraded":"2024-04-20T10:21:08.440949+0000","last_fullsized":"2024-04-20T10:21:08.440949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:01:55.363202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.881645+0000","last_change":"2024-04-20T10:21:01.611725+0000","last_active":"2024-04-20T10:21:07.881645+0000","last_peered":"2024-04-20T10:21:07.881645+0000","last_clean":"2024-04-20T10:21:07.881645+0000","last_became_active":"2024-04-20T10:21:01.609982+0000","last_became_peered":"2024-04-20T10:21:01.609982+0000","last_unstale":"2024-04-20T10:21:07.881645+0000","last_undegraded":"2024-04-20T10:21:07.881645+0000","last_fullsized":"2024-04-20T10:21:07.881645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:32:02.981747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441859+0000","last_change":"2024-04-20T10:21:06.897904+0000","last_active":"2024-04-20T10:21:08.441859+0000","last_peered":"2024-04-20T10:21:08.441859+0000","last_clean":"2024-04-20T10:21:08.441859+0000","last_became_active":"2024-04-20T10:21:06.896628+0000","last_became_peered":"2024-04-20T10:21:06.896628+0000","last_unstale":"2024-04-20T10:21:08.441859+0000","last_undegraded":"2024-04-20T10:21:08.441859+0000","last_fullsized":"2024-04-20T10:21:08.441859+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:35:58.510637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542234+0000","last_change":"2024-04-20T10:21:02.651679+0000","last_active":"2024-04-20T10:21:08.542234+0000","last_peered":"2024-04-20T10:21:08.542234+0000","last_clean":"2024-04-20T10:21:08.542234+0000","last_became_active":"2024-04-20T10:21:02.651336+0000","last_became_peered":"2024-04-20T10:21:02.651336+0000","last_unstale":"2024-04-20T10:21:08.542234+0000","last_undegraded":"2024-04-20T10:21:08.542234+0000","last_fullsized":"2024-04-20T10:21:08.542234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:17:24.565693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.389935+0000","last_change":"2024-04-20T10:21:01.620091+0000","last_active":"2024-04-20T10:21:08.389935+0000","last_peered":"2024-04-20T10:21:08.389935+0000","last_clean":"2024-04-20T10:21:08.389935+0000","last_became_active":"2024-04-20T10:21:01.619747+0000","last_became_peered":"2024-04-20T10:21:01.619747+0000","last_unstale":"2024-04-20T10:21:08.389935+0000","last_undegraded":"2024-04-20T10:21:08.389935+0000","last_fullsized":"2024-04-20T10:21:08.389935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:38:10.686751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.544184+0000","last_change":"2024-04-20T10:21:06.902502+0000","last_active":"2024-04-20T10:21:08.544184+0000","last_peered":"2024-04-20T10:21:08.544184+0000","last_clean":"2024-04-20T10:21:08.544184+0000","last_became_active":"2024-04-20T10:21:06.899513+0000","last_became_peered":"2024-04-20T10:21:06.899513+0000","last_unstale":"2024-04-20T10:21:08.544184+0000","last_undegraded":"2024-04-20T10:21:08.544184+0000","last_fullsized":"2024-04-20T10:21:08.544184+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:46:45.448591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541620+0000","last_change":"2024-04-20T10:21:02.628521+0000","last_active":"2024-04-20T10:21:08.541620+0000","last_peered":"2024-04-20T10:21:08.541620+0000","last_clean":"2024-04-20T10:21:08.541620+0000","last_became_active":"2024-04-20T10:21:02.626414+0000","last_became_peered":"2024-04-20T10:21:02.626414+0000","last_unstale":"2024-04-20T10:21:08.541620+0000","last_undegraded":"2024-04-20T10:21:08.541620+0000","last_fullsized":"2024-04-20T10:21:08.541620+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:01:55.363202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541447+0000","last_change":"2024-04-20T10:21:01.610399+0000","last_active":"2024-04-20T10:21:08.541447+0000","last_peered":"2024-04-20T10:21:08.541447+0000","last_clean":"2024-04-20T10:21:08.541447+0000","last_became_active":"2024-04-20T10:21:01.610116+0000","last_became_peered":"2024-04-20T10:21:01.610116+0000","last_unstale":"2024-04-20T10:21:08.541447+0000","last_undegraded":"2024-04-20T10:21:08.541447+0000","last_fullsized":"2024-04-20T10:21:08.541447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:01:18.578396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.390168+0000","last_change":"2024-04-20T10:20:52.916129+0000","last_active":"2024-04-20T10:21:08.390168+0000","last_peered":"2024-04-20T10:21:08.390168+0000","last_clean":"2024-04-20T10:21:08.390168+0000","last_became_active":"2024-04-20T10:20:52.913922+0000","last_became_peered":"2024-04-20T10:20:52.913922+0000","last_unstale":"2024-04-20T10:21:08.390168+0000","last_undegraded":"2024-04-20T10:21:08.390168+0000","last_fullsized":"2024-04-20T10:21:08.390168+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:20:51.267588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:20:51.267588+0000","last_clean_scrub_stamp":"2024-04-20T10:20:51.267588+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:30:44.465836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.039062+0000","last_change":"2024-04-20T10:21:06.913314+0000","last_active":"2024-04-20T10:21:08.039062+0000","last_peered":"2024-04-20T10:21:08.039062+0000","last_clean":"2024-04-20T10:21:08.039062+0000","last_became_active":"2024-04-20T10:21:06.912896+0000","last_became_peered":"2024-04-20T10:21:06.912896+0000","last_unstale":"2024-04-20T10:21:08.039062+0000","last_undegraded":"2024-04-20T10:21:08.039062+0000","last_fullsized":"2024-04-20T10:21:08.039062+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:16:08.900837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.391138+0000","last_change":"2024-04-20T10:21:02.643414+0000","last_active":"2024-04-20T10:21:08.391138+0000","last_peered":"2024-04-20T10:21:08.391138+0000","last_clean":"2024-04-20T10:21:08.391138+0000","last_became_active":"2024-04-20T10:21:02.641460+0000","last_became_peered":"2024-04-20T10:21:02.641460+0000","last_unstale":"2024-04-20T10:21:08.391138+0000","last_undegraded":"2024-04-20T10:21:08.391138+0000","last_fullsized":"2024-04-20T10:21:08.391138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:24:51.806454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.882750+0000","last_change":"2024-04-20T10:21:01.612256+0000","last_active":"2024-04-20T10:21:07.882750+0000","last_peered":"2024-04-20T10:21:07.882750+0000","last_clean":"2024-04-20T10:21:07.882750+0000","last_became_active":"2024-04-20T10:21:01.611856+0000","last_became_peered":"2024-04-20T10:21:01.611856+0000","last_unstale":"2024-04-20T10:21:07.882750+0000","last_undegraded":"2024-04-20T10:21:07.882750+0000","last_fullsized":"2024-04-20T10:21:07.882750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:59:37.356548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.038723+0000","last_change":"2024-04-20T10:21:06.913797+0000","last_active":"2024-04-20T10:21:08.038723+0000","last_peered":"2024-04-20T10:21:08.038723+0000","last_clean":"2024-04-20T10:21:08.038723+0000","last_became_active":"2024-04-20T10:21:06.912928+0000","last_became_peered":"2024-04-20T10:21:06.912928+0000","last_unstale":"2024-04-20T10:21:08.038723+0000","last_undegraded":"2024-04-20T10:21:08.038723+0000","last_fullsized":"2024-04-20T10:21:08.038723+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:21:29.812831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542069+0000","last_change":"2024-04-20T10:21:02.630629+0000","last_active":"2024-04-20T10:21:08.542069+0000","last_peered":"2024-04-20T10:21:08.542069+0000","last_clean":"2024-04-20T10:21:08.542069+0000","last_became_active":"2024-04-20T10:21:02.623112+0000","last_became_peered":"2024-04-20T10:21:02.623112+0000","last_unstale":"2024-04-20T10:21:08.542069+0000","last_undegraded":"2024-04-20T10:21:08.542069+0000","last_fullsized":"2024-04-20T10:21:08.542069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:37:12.899052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441939+0000","last_change":"2024-04-20T10:21:01.598214+0000","last_active":"2024-04-20T10:21:08.441939+0000","last_peered":"2024-04-20T10:21:08.441939+0000","last_clean":"2024-04-20T10:21:08.441939+0000","last_became_active":"2024-04-20T10:21:01.597260+0000","last_became_peered":"2024-04-20T10:21:01.597260+0000","last_unstale":"2024-04-20T10:21:08.441939+0000","last_undegraded":"2024-04-20T10:21:08.441939+0000","last_fullsized":"2024-04-20T10:21:08.441939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:25:59.939855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.543790+0000","last_change":"2024-04-20T10:21:06.894202+0000","last_active":"2024-04-20T10:21:08.543790+0000","last_peered":"2024-04-20T10:21:08.543790+0000","last_clean":"2024-04-20T10:21:08.543790+0000","last_became_active":"2024-04-20T10:21:06.893620+0000","last_became_peered":"2024-04-20T10:21:06.893620+0000","last_unstale":"2024-04-20T10:21:08.543790+0000","last_undegraded":"2024-04-20T10:21:08.543790+0000","last_fullsized":"2024-04-20T10:21:08.543790+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:31:30.034892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.391863+0000","last_change":"2024-04-20T10:21:02.655710+0000","last_active":"2024-04-20T10:21:08.391863+0000","last_peered":"2024-04-20T10:21:08.391863+0000","last_clean":"2024-04-20T10:21:08.391863+0000","last_became_active":"2024-04-20T10:21:02.655531+0000","last_became_peered":"2024-04-20T10:21:02.655531+0000","last_unstale":"2024-04-20T10:21:08.391863+0000","last_undegraded":"2024-04-20T10:21:08.391863+0000","last_fullsized":"2024-04-20T10:21:08.391863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:27:00.187303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.883115+0000","last_change":"2024-04-20T10:21:01.612100+0000","last_active":"2024-04-20T10:21:07.883115+0000","last_peered":"2024-04-20T10:21:07.883115+0000","last_clean":"2024-04-20T10:21:07.883115+0000","last_became_active":"2024-04-20T10:21:01.610465+0000","last_became_peered":"2024-04-20T10:21:01.610465+0000","last_unstale":"2024-04-20T10:21:07.883115+0000","last_undegraded":"2024-04-20T10:21:07.883115+0000","last_fullsized":"2024-04-20T10:21:07.883115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:01:19.566351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540603+0000","last_change":"2024-04-20T10:21:06.908159+0000","last_active":"2024-04-20T10:21:08.540603+0000","last_peered":"2024-04-20T10:21:08.540603+0000","last_clean":"2024-04-20T10:21:08.540603+0000","last_became_active":"2024-04-20T10:21:06.907687+0000","last_became_peered":"2024-04-20T10:21:06.907687+0000","last_unstale":"2024-04-20T10:21:08.540603+0000","last_undegraded":"2024-04-20T10:21:08.540603+0000","last_fullsized":"2024-04-20T10:21:08.540603+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:17:43.493429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.391503+0000","last_change":"2024-04-20T10:21:02.652617+0000","last_active":"2024-04-20T10:21:08.391503+0000","last_peered":"2024-04-20T10:21:08.391503+0000","last_clean":"2024-04-20T10:21:08.391503+0000","last_became_active":"2024-04-20T10:21:02.652321+0000","last_became_peered":"2024-04-20T10:21:02.652321+0000","last_unstale":"2024-04-20T10:21:08.391503+0000","last_undegraded":"2024-04-20T10:21:08.391503+0000","last_fullsized":"2024-04-20T10:21:08.391503+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:59:24.572150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.887281+0000","last_change":"2024-04-20T10:21:01.608355+0000","last_active":"2024-04-20T10:21:07.887281+0000","last_peered":"2024-04-20T10:21:07.887281+0000","last_clean":"2024-04-20T10:21:07.887281+0000","last_became_active":"2024-04-20T10:21:01.607981+0000","last_became_peered":"2024-04-20T10:21:01.607981+0000","last_unstale":"2024-04-20T10:21:07.887281+0000","last_undegraded":"2024-04-20T10:21:07.887281+0000","last_fullsized":"2024-04-20T10:21:07.887281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:10:57.859781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.885574+0000","last_change":"2024-04-20T10:21:06.900202+0000","last_active":"2024-04-20T10:21:07.885574+0000","last_peered":"2024-04-20T10:21:07.885574+0000","last_clean":"2024-04-20T10:21:07.885574+0000","last_became_active":"2024-04-20T10:21:06.899795+0000","last_became_peered":"2024-04-20T10:21:06.899795+0000","last_unstale":"2024-04-20T10:21:07.885574+0000","last_undegraded":"2024-04-20T10:21:07.885574+0000","last_fullsized":"2024-04-20T10:21:07.885574+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:01:59.638230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.390393+0000","last_change":"2024-04-20T10:21:02.646213+0000","last_active":"2024-04-20T10:21:08.390393+0000","last_peered":"2024-04-20T10:21:08.390393+0000","last_clean":"2024-04-20T10:21:08.390393+0000","last_became_active":"2024-04-20T10:21:02.645881+0000","last_became_peered":"2024-04-20T10:21:02.645881+0000","last_unstale":"2024-04-20T10:21:08.390393+0000","last_undegraded":"2024-04-20T10:21:08.390393+0000","last_fullsized":"2024-04-20T10:21:08.390393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:21:25.537803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.881093+0000","last_change":"2024-04-20T10:21:01.601030+0000","last_active":"2024-04-20T10:21:07.881093+0000","last_peered":"2024-04-20T10:21:07.881093+0000","last_clean":"2024-04-20T10:21:07.881093+0000","last_became_active":"2024-04-20T10:21:01.600702+0000","last_became_peered":"2024-04-20T10:21:01.600702+0000","last_unstale":"2024-04-20T10:21:07.881093+0000","last_undegraded":"2024-04-20T10:21:07.881093+0000","last_fullsized":"2024-04-20T10:21:07.881093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:36:15.966290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.543987+0000","last_change":"2024-04-20T10:21:06.904141+0000","last_active":"2024-04-20T10:21:08.543987+0000","last_peered":"2024-04-20T10:21:08.543987+0000","last_clean":"2024-04-20T10:21:08.543987+0000","last_became_active":"2024-04-20T10:21:06.903212+0000","last_became_peered":"2024-04-20T10:21:06.903212+0000","last_unstale":"2024-04-20T10:21:08.543987+0000","last_undegraded":"2024-04-20T10:21:08.543987+0000","last_fullsized":"2024-04-20T10:21:08.543987+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:59:28.847178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.038508+0000","last_change":"2024-04-20T10:21:02.644636+0000","last_active":"2024-04-20T10:21:08.038508+0000","last_peered":"2024-04-20T10:21:08.038508+0000","last_clean":"2024-04-20T10:21:08.038508+0000","last_became_active":"2024-04-20T10:21:02.644320+0000","last_became_peered":"2024-04-20T10:21:02.644320+0000","last_unstale":"2024-04-20T10:21:08.038508+0000","last_undegraded":"2024-04-20T10:21:08.038508+0000","last_fullsized":"2024-04-20T10:21:08.038508+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:06:59.919488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.038609+0000","last_change":"2024-04-20T10:21:01.616010+0000","last_active":"2024-04-20T10:21:08.038609+0000","last_peered":"2024-04-20T10:21:08.038609+0000","last_clean":"2024-04-20T10:21:08.038609+0000","last_became_active":"2024-04-20T10:21:01.615701+0000","last_became_peered":"2024-04-20T10:21:01.615701+0000","last_unstale":"2024-04-20T10:21:08.038609+0000","last_undegraded":"2024-04-20T10:21:08.038609+0000","last_fullsized":"2024-04-20T10:21:08.038609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:32:02.981747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542308+0000","last_change":"2024-04-20T10:21:06.909393+0000","last_active":"2024-04-20T10:21:08.542308+0000","last_peered":"2024-04-20T10:21:08.542308+0000","last_clean":"2024-04-20T10:21:08.542308+0000","last_became_active":"2024-04-20T10:21:06.907616+0000","last_became_peered":"2024-04-20T10:21:06.907616+0000","last_unstale":"2024-04-20T10:21:08.542308+0000","last_undegraded":"2024-04-20T10:21:08.542308+0000","last_fullsized":"2024-04-20T10:21:08.542308+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:07:32.815495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.390789+0000","last_change":"2024-04-20T10:21:02.654823+0000","last_active":"2024-04-20T10:21:08.390789+0000","last_peered":"2024-04-20T10:21:08.390789+0000","last_clean":"2024-04-20T10:21:08.390789+0000","last_became_active":"2024-04-20T10:21:02.654556+0000","last_became_peered":"2024-04-20T10:21:02.654556+0000","last_unstale":"2024-04-20T10:21:08.390789+0000","last_undegraded":"2024-04-20T10:21:08.390789+0000","last_fullsized":"2024-04-20T10:21:08.390789+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:52:24.023900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541418+0000","last_change":"2024-04-20T10:21:01.618588+0000","last_active":"2024-04-20T10:21:08.541418+0000","last_peered":"2024-04-20T10:21:08.541418+0000","last_clean":"2024-04-20T10:21:08.541418+0000","last_became_active":"2024-04-20T10:21:01.618294+0000","last_became_peered":"2024-04-20T10:21:01.618294+0000","last_unstale":"2024-04-20T10:21:08.541418+0000","last_undegraded":"2024-04-20T10:21:08.541418+0000","last_fullsized":"2024-04-20T10:21:08.541418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:34:27.492302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542602+0000","last_change":"2024-04-20T10:21:06.897913+0000","last_active":"2024-04-20T10:21:08.542602+0000","last_peered":"2024-04-20T10:21:08.542602+0000","last_clean":"2024-04-20T10:21:08.542602+0000","last_became_active":"2024-04-20T10:21:06.897536+0000","last_became_peered":"2024-04-20T10:21:06.897536+0000","last_unstale":"2024-04-20T10:21:08.542602+0000","last_undegraded":"2024-04-20T10:21:08.542602+0000","last_fullsized":"2024-04-20T10:21:08.542602+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:27:49.300931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.543513+0000","last_change":"2024-04-20T10:21:02.644302+0000","last_active":"2024-04-20T10:21:08.543513+0000","last_peered":"2024-04-20T10:21:08.543513+0000","last_clean":"2024-04-20T10:21:08.543513+0000","last_became_active":"2024-04-20T10:21:02.643236+0000","last_became_peered":"2024-04-20T10:21:02.643236+0000","last_unstale":"2024-04-20T10:21:08.543513+0000","last_undegraded":"2024-04-20T10:21:08.543513+0000","last_fullsized":"2024-04-20T10:21:08.543513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:06:59.919488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.544270+0000","last_change":"2024-04-20T10:21:01.604692+0000","last_active":"2024-04-20T10:21:08.544270+0000","last_peered":"2024-04-20T10:21:08.544270+0000","last_clean":"2024-04-20T10:21:08.544270+0000","last_became_active":"2024-04-20T10:21:01.604345+0000","last_became_peered":"2024-04-20T10:21:01.604345+0000","last_unstale":"2024-04-20T10:21:08.544270+0000","last_undegraded":"2024-04-20T10:21:08.544270+0000","last_fullsized":"2024-04-20T10:21:08.544270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:29:47.733236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542165+0000","last_change":"2024-04-20T10:21:06.901480+0000","last_active":"2024-04-20T10:21:08.542165+0000","last_peered":"2024-04-20T10:21:08.542165+0000","last_clean":"2024-04-20T10:21:08.542165+0000","last_became_active":"2024-04-20T10:21:06.900820+0000","last_became_peered":"2024-04-20T10:21:06.900820+0000","last_unstale":"2024-04-20T10:21:08.542165+0000","last_undegraded":"2024-04-20T10:21:08.542165+0000","last_fullsized":"2024-04-20T10:21:08.542165+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:15:30.919284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.391163+0000","last_change":"2024-04-20T10:21:02.642723+0000","last_active":"2024-04-20T10:21:08.391163+0000","last_peered":"2024-04-20T10:21:08.391163+0000","last_clean":"2024-04-20T10:21:08.391163+0000","last_became_active":"2024-04-20T10:21:02.641828+0000","last_became_peered":"2024-04-20T10:21:02.641828+0000","last_unstale":"2024-04-20T10:21:08.391163+0000","last_undegraded":"2024-04-20T10:21:08.391163+0000","last_fullsized":"2024-04-20T10:21:08.391163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:31:25.759865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:21.609415+0000","last_change":"2024-04-20T10:21:01.619387+0000","last_active":"2024-04-20T10:21:21.609415+0000","last_peered":"2024-04-20T10:21:21.609415+0000","last_clean":"2024-04-20T10:21:21.609415+0000","last_became_active":"2024-04-20T10:21:01.619119+0000","last_became_peered":"2024-04-20T10:21:01.619119+0000","last_unstale":"2024-04-20T10:21:21.609415+0000","last_undegraded":"2024-04-20T10:21:21.609415+0000","last_fullsized":"2024-04-20T10:21:21.609415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:55:41.208744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441019+0000","last_change":"2024-04-20T10:21:06.911043+0000","last_active":"2024-04-20T10:21:08.441019+0000","last_peered":"2024-04-20T10:21:08.441019+0000","last_clean":"2024-04-20T10:21:08.441019+0000","last_became_active":"2024-04-20T10:21:06.909896+0000","last_became_peered":"2024-04-20T10:21:06.909896+0000","last_unstale":"2024-04-20T10:21:08.441019+0000","last_undegraded":"2024-04-20T10:21:08.441019+0000","last_fullsized":"2024-04-20T10:21:08.441019+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:24:56.081482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.887153+0000","last_change":"2024-04-20T10:21:02.624575+0000","last_active":"2024-04-20T10:21:07.887153+0000","last_peered":"2024-04-20T10:21:07.887153+0000","last_clean":"2024-04-20T10:21:07.887153+0000","last_became_active":"2024-04-20T10:21:02.624230+0000","last_became_peered":"2024-04-20T10:21:02.624230+0000","last_unstale":"2024-04-20T10:21:07.887153+0000","last_undegraded":"2024-04-20T10:21:07.887153+0000","last_fullsized":"2024-04-20T10:21:07.887153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:07:51.297896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441862+0000","last_change":"2024-04-20T10:21:01.619231+0000","last_active":"2024-04-20T10:21:08.441862+0000","last_peered":"2024-04-20T10:21:08.441862+0000","last_clean":"2024-04-20T10:21:08.441862+0000","last_became_active":"2024-04-20T10:21:01.618872+0000","last_became_peered":"2024-04-20T10:21:01.618872+0000","last_unstale":"2024-04-20T10:21:08.441862+0000","last_undegraded":"2024-04-20T10:21:08.441862+0000","last_fullsized":"2024-04-20T10:21:08.441862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:38:10.686751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.886989+0000","last_change":"2024-04-20T10:21:06.908460+0000","last_active":"2024-04-20T10:21:07.886989+0000","last_peered":"2024-04-20T10:21:07.886989+0000","last_clean":"2024-04-20T10:21:07.886989+0000","last_became_active":"2024-04-20T10:21:06.907722+0000","last_became_peered":"2024-04-20T10:21:06.907722+0000","last_unstale":"2024-04-20T10:21:07.886989+0000","last_undegraded":"2024-04-20T10:21:07.886989+0000","last_fullsized":"2024-04-20T10:21:07.886989+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:06:15.702589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.544085+0000","last_change":"2024-04-20T10:21:02.646092+0000","last_active":"2024-04-20T10:21:08.544085+0000","last_peered":"2024-04-20T10:21:08.544085+0000","last_clean":"2024-04-20T10:21:08.544085+0000","last_became_active":"2024-04-20T10:21:02.645788+0000","last_became_peered":"2024-04-20T10:21:02.645788+0000","last_unstale":"2024-04-20T10:21:08.544085+0000","last_undegraded":"2024-04-20T10:21:08.544085+0000","last_fullsized":"2024-04-20T10:21:08.544085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:22:36.605724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542309+0000","last_change":"2024-04-20T10:21:01.623128+0000","last_active":"2024-04-20T10:21:08.542309+0000","last_peered":"2024-04-20T10:21:08.542309+0000","last_clean":"2024-04-20T10:21:08.542309+0000","last_became_active":"2024-04-20T10:21:01.622867+0000","last_became_peered":"2024-04-20T10:21:01.622867+0000","last_unstale":"2024-04-20T10:21:08.542309+0000","last_undegraded":"2024-04-20T10:21:08.542309+0000","last_fullsized":"2024-04-20T10:21:08.542309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:50:23.957089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541079+0000","last_change":"2024-04-20T10:21:06.915147+0000","last_active":"2024-04-20T10:21:08.541079+0000","last_peered":"2024-04-20T10:21:08.541079+0000","last_clean":"2024-04-20T10:21:08.541079+0000","last_became_active":"2024-04-20T10:21:06.913832+0000","last_became_peered":"2024-04-20T10:21:06.913832+0000","last_unstale":"2024-04-20T10:21:08.541079+0000","last_undegraded":"2024-04-20T10:21:08.541079+0000","last_fullsized":"2024-04-20T10:21:08.541079+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:10:39.166791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542260+0000","last_change":"2024-04-20T10:21:02.638354+0000","last_active":"2024-04-20T10:21:08.542260+0000","last_peered":"2024-04-20T10:21:08.542260+0000","last_clean":"2024-04-20T10:21:08.542260+0000","last_became_active":"2024-04-20T10:21:02.638065+0000","last_became_peered":"2024-04-20T10:21:02.638065+0000","last_unstale":"2024-04-20T10:21:08.542260+0000","last_undegraded":"2024-04-20T10:21:08.542260+0000","last_fullsized":"2024-04-20T10:21:08.542260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:12:12.144216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.039425+0000","last_change":"2024-04-20T10:21:01.621766+0000","last_active":"2024-04-20T10:21:08.039425+0000","last_peered":"2024-04-20T10:21:08.039425+0000","last_clean":"2024-04-20T10:21:08.039425+0000","last_became_active":"2024-04-20T10:21:01.621546+0000","last_became_peered":"2024-04-20T10:21:01.621546+0000","last_unstale":"2024-04-20T10:21:08.039425+0000","last_undegraded":"2024-04-20T10:21:08.039425+0000","last_fullsized":"2024-04-20T10:21:08.039425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:01:19.566351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540739+0000","last_change":"2024-04-20T10:21:06.910220+0000","last_active":"2024-04-20T10:21:08.540739+0000","last_peered":"2024-04-20T10:21:08.540739+0000","last_clean":"2024-04-20T10:21:08.540739+0000","last_became_active":"2024-04-20T10:21:06.909876+0000","last_became_peered":"2024-04-20T10:21:06.909876+0000","last_unstale":"2024-04-20T10:21:08.540739+0000","last_undegraded":"2024-04-20T10:21:08.540739+0000","last_fullsized":"2024-04-20T10:21:08.540739+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:27:04.462330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.543797+0000","last_change":"2024-04-20T10:21:06.903823+0000","last_active":"2024-04-20T10:21:08.543797+0000","last_peered":"2024-04-20T10:21:08.543797+0000","last_clean":"2024-04-20T10:21:08.543797+0000","last_became_active":"2024-04-20T10:21:06.903100+0000","last_became_peered":"2024-04-20T10:21:06.903100+0000","last_unstale":"2024-04-20T10:21:08.543797+0000","last_undegraded":"2024-04-20T10:21:08.543797+0000","last_fullsized":"2024-04-20T10:21:08.543797+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:24:56.081482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.440526+0000","last_change":"2024-04-20T10:21:02.636194+0000","last_active":"2024-04-20T10:21:08.440526+0000","last_peered":"2024-04-20T10:21:08.440526+0000","last_clean":"2024-04-20T10:21:08.440526+0000","last_became_active":"2024-04-20T10:21:02.635907+0000","last_became_peered":"2024-04-20T10:21:02.635907+0000","last_unstale":"2024-04-20T10:21:08.440526+0000","last_undegraded":"2024-04-20T10:21:08.440526+0000","last_fullsized":"2024-04-20T10:21:08.440526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:06:59.919488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540844+0000","last_change":"2024-04-20T10:21:01.617415+0000","last_active":"2024-04-20T10:21:08.540844+0000","last_peered":"2024-04-20T10:21:08.540844+0000","last_clean":"2024-04-20T10:21:08.540844+0000","last_became_active":"2024-04-20T10:21:01.617082+0000","last_became_peered":"2024-04-20T10:21:01.617082+0000","last_unstale":"2024-04-20T10:21:08.540844+0000","last_undegraded":"2024-04-20T10:21:08.540844+0000","last_fullsized":"2024-04-20T10:21:08.540844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:56:45.348182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.039217+0000","last_change":"2024-04-20T10:21:06.915304+0000","last_active":"2024-04-20T10:21:08.039217+0000","last_peered":"2024-04-20T10:21:08.039217+0000","last_clean":"2024-04-20T10:21:08.039217+0000","last_became_active":"2024-04-20T10:21:06.914915+0000","last_became_peered":"2024-04-20T10:21:06.914915+0000","last_unstale":"2024-04-20T10:21:08.039217+0000","last_undegraded":"2024-04-20T10:21:08.039217+0000","last_fullsized":"2024-04-20T10:21:08.039217+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:52:28.298927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540918+0000","last_change":"2024-04-20T10:21:02.646474+0000","last_active":"2024-04-20T10:21:08.540918+0000","last_peered":"2024-04-20T10:21:08.540918+0000","last_clean":"2024-04-20T10:21:08.540918+0000","last_became_active":"2024-04-20T10:21:02.646219+0000","last_became_peered":"2024-04-20T10:21:02.646219+0000","last_unstale":"2024-04-20T10:21:08.540918+0000","last_undegraded":"2024-04-20T10:21:08.540918+0000","last_fullsized":"2024-04-20T10:21:08.540918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:06:59.919488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541015+0000","last_change":"2024-04-20T10:21:01.609406+0000","last_active":"2024-04-20T10:21:08.541015+0000","last_peered":"2024-04-20T10:21:08.541015+0000","last_clean":"2024-04-20T10:21:08.541015+0000","last_became_active":"2024-04-20T10:21:01.609064+0000","last_became_peered":"2024-04-20T10:21:01.609064+0000","last_unstale":"2024-04-20T10:21:08.541015+0000","last_undegraded":"2024-04-20T10:21:08.541015+0000","last_fullsized":"2024-04-20T10:21:08.541015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:28:01.100842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.039237+0000","last_change":"2024-04-20T10:21:01.615891+0000","last_active":"2024-04-20T10:21:08.039237+0000","last_peered":"2024-04-20T10:21:08.039237+0000","last_clean":"2024-04-20T10:21:08.039237+0000","last_became_active":"2024-04-20T10:21:01.615584+0000","last_became_peered":"2024-04-20T10:21:01.615584+0000","last_unstale":"2024-04-20T10:21:08.039237+0000","last_undegraded":"2024-04-20T10:21:08.039237+0000","last_fullsized":"2024-04-20T10:21:08.039237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:59:37.356548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.544232+0000","last_change":"2024-04-20T10:21:02.643605+0000","last_active":"2024-04-20T10:21:08.544232+0000","last_peered":"2024-04-20T10:21:08.544232+0000","last_clean":"2024-04-20T10:21:08.544232+0000","last_became_active":"2024-04-20T10:21:02.643327+0000","last_became_peered":"2024-04-20T10:21:02.643327+0000","last_unstale":"2024-04-20T10:21:08.544232+0000","last_undegraded":"2024-04-20T10:21:08.544232+0000","last_fullsized":"2024-04-20T10:21:08.544232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:01:55.363202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.882164+0000","last_change":"2024-04-20T10:21:06.892955+0000","last_active":"2024-04-20T10:21:07.882164+0000","last_peered":"2024-04-20T10:21:07.882164+0000","last_clean":"2024-04-20T10:21:07.882164+0000","last_became_active":"2024-04-20T10:21:06.892534+0000","last_became_peered":"2024-04-20T10:21:06.892534+0000","last_unstale":"2024-04-20T10:21:07.882164+0000","last_undegraded":"2024-04-20T10:21:07.882164+0000","last_fullsized":"2024-04-20T10:21:07.882164+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:17:28.840720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541245+0000","last_change":"2024-04-20T10:21:01.618877+0000","last_active":"2024-04-20T10:21:08.541245+0000","last_peered":"2024-04-20T10:21:08.541245+0000","last_clean":"2024-04-20T10:21:08.541245+0000","last_became_active":"2024-04-20T10:21:01.618549+0000","last_became_peered":"2024-04-20T10:21:01.618549+0000","last_unstale":"2024-04-20T10:21:08.541245+0000","last_undegraded":"2024-04-20T10:21:08.541245+0000","last_fullsized":"2024-04-20T10:21:08.541245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:36:15.966290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.886244+0000","last_change":"2024-04-20T10:21:02.645042+0000","last_active":"2024-04-20T10:21:07.886244+0000","last_peered":"2024-04-20T10:21:07.886244+0000","last_clean":"2024-04-20T10:21:07.886244+0000","last_became_active":"2024-04-20T10:21:02.644630+0000","last_became_peered":"2024-04-20T10:21:02.644630+0000","last_unstale":"2024-04-20T10:21:07.886244+0000","last_undegraded":"2024-04-20T10:21:07.886244+0000","last_fullsized":"2024-04-20T10:21:07.886244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:59:38.370072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542570+0000","last_change":"2024-04-20T10:21:06.888033+0000","last_active":"2024-04-20T10:21:08.542570+0000","last_peered":"2024-04-20T10:21:08.542570+0000","last_clean":"2024-04-20T10:21:08.542570+0000","last_became_active":"2024-04-20T10:21:06.887672+0000","last_became_peered":"2024-04-20T10:21:06.887672+0000","last_unstale":"2024-04-20T10:21:08.542570+0000","last_undegraded":"2024-04-20T10:21:08.542570+0000","last_fullsized":"2024-04-20T10:21:08.542570+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:13:54.194111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.038167+0000","last_change":"2024-04-20T10:21:02.644672+0000","last_active":"2024-04-20T10:21:08.038167+0000","last_peered":"2024-04-20T10:21:08.038167+0000","last_clean":"2024-04-20T10:21:08.038167+0000","last_became_active":"2024-04-20T10:21:02.644327+0000","last_became_peered":"2024-04-20T10:21:02.644327+0000","last_unstale":"2024-04-20T10:21:08.038167+0000","last_undegraded":"2024-04-20T10:21:08.038167+0000","last_fullsized":"2024-04-20T10:21:08.038167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:28:54.408773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.881607+0000","last_change":"2024-04-20T10:21:02.624910+0000","last_active":"2024-04-20T10:21:07.881607+0000","last_peered":"2024-04-20T10:21:07.881607+0000","last_clean":"2024-04-20T10:21:07.881607+0000","last_became_active":"2024-04-20T10:21:02.624555+0000","last_became_peered":"2024-04-20T10:21:02.624555+0000","last_unstale":"2024-04-20T10:21:07.881607+0000","last_undegraded":"2024-04-20T10:21:07.881607+0000","last_fullsized":"2024-04-20T10:21:07.881607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:07:51.297896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541176+0000","last_change":"2024-04-20T10:21:02.636958+0000","last_active":"2024-04-20T10:21:08.541176+0000","last_peered":"2024-04-20T10:21:08.541176+0000","last_clean":"2024-04-20T10:21:08.541176+0000","last_became_active":"2024-04-20T10:21:02.636594+0000","last_became_peered":"2024-04-20T10:21:02.636594+0000","last_unstale":"2024-04-20T10:21:08.541176+0000","last_undegraded":"2024-04-20T10:21:08.541176+0000","last_fullsized":"2024-04-20T10:21:08.541176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:12:07.469367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.543932+0000","last_change":"2024-04-20T10:21:02.649761+0000","last_active":"2024-04-20T10:21:08.543932+0000","last_peered":"2024-04-20T10:21:08.543932+0000","last_clean":"2024-04-20T10:21:08.543932+0000","last_became_active":"2024-04-20T10:21:02.649471+0000","last_became_peered":"2024-04-20T10:21:02.649471+0000","last_unstale":"2024-04-20T10:21:08.543932+0000","last_undegraded":"2024-04-20T10:21:08.543932+0000","last_fullsized":"2024-04-20T10:21:08.543932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:14:16.934710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.543183+0000","last_change":"2024-04-20T10:21:02.644223+0000","last_active":"2024-04-20T10:21:08.543183+0000","last_peered":"2024-04-20T10:21:08.543183+0000","last_clean":"2024-04-20T10:21:08.543183+0000","last_became_active":"2024-04-20T10:21:02.643893+0000","last_became_peered":"2024-04-20T10:21:02.643893+0000","last_unstale":"2024-04-20T10:21:08.543183+0000","last_undegraded":"2024-04-20T10:21:08.543183+0000","last_fullsized":"2024-04-20T10:21:08.543183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:09:41.966188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441369+0000","last_change":"2024-04-20T10:21:02.636144+0000","last_active":"2024-04-20T10:21:08.441369+0000","last_peered":"2024-04-20T10:21:08.441369+0000","last_clean":"2024-04-20T10:21:08.441369+0000","last_became_active":"2024-04-20T10:21:02.635834+0000","last_became_peered":"2024-04-20T10:21:02.635834+0000","last_unstale":"2024-04-20T10:21:08.441369+0000","last_undegraded":"2024-04-20T10:21:08.441369+0000","last_fullsized":"2024-04-20T10:21:08.441369+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:10:15.206686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.039750+0000","last_change":"2024-04-20T10:21:02.643146+0000","last_active":"2024-04-20T10:21:08.039750+0000","last_peered":"2024-04-20T10:21:08.039750+0000","last_clean":"2024-04-20T10:21:08.039750+0000","last_became_active":"2024-04-20T10:21:02.639521+0000","last_became_peered":"2024-04-20T10:21:02.639521+0000","last_unstale":"2024-04-20T10:21:08.039750+0000","last_undegraded":"2024-04-20T10:21:08.039750+0000","last_fullsized":"2024-04-20T10:21:08.039750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:01:55.363202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542473+0000","last_change":"2024-04-20T10:21:02.631553+0000","last_active":"2024-04-20T10:21:08.542473+0000","last_peered":"2024-04-20T10:21:08.542473+0000","last_clean":"2024-04-20T10:21:08.542473+0000","last_became_active":"2024-04-20T10:21:02.626210+0000","last_became_peered":"2024-04-20T10:21:02.626210+0000","last_unstale":"2024-04-20T10:21:08.542473+0000","last_undegraded":"2024-04-20T10:21:08.542473+0000","last_fullsized":"2024-04-20T10:21:08.542473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:02:26.716730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540787+0000","last_change":"2024-04-20T10:21:02.625457+0000","last_active":"2024-04-20T10:21:08.540787+0000","last_peered":"2024-04-20T10:21:08.540787+0000","last_clean":"2024-04-20T10:21:08.540787+0000","last_became_active":"2024-04-20T10:21:02.624987+0000","last_became_peered":"2024-04-20T10:21:02.624987+0000","last_unstale":"2024-04-20T10:21:08.540787+0000","last_undegraded":"2024-04-20T10:21:08.540787+0000","last_fullsized":"2024-04-20T10:21:08.540787+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:13:44.633309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540862+0000","last_change":"2024-04-20T10:21:02.628208+0000","last_active":"2024-04-20T10:21:08.540862+0000","last_peered":"2024-04-20T10:21:08.540862+0000","last_clean":"2024-04-20T10:21:08.540862+0000","last_became_active":"2024-04-20T10:21:02.625255+0000","last_became_peered":"2024-04-20T10:21:02.625255+0000","last_unstale":"2024-04-20T10:21:08.540862+0000","last_undegraded":"2024-04-20T10:21:08.540862+0000","last_fullsized":"2024-04-20T10:21:08.540862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:06:35.349863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541152+0000","last_change":"2024-04-20T10:21:02.629528+0000","last_active":"2024-04-20T10:21:08.541152+0000","last_peered":"2024-04-20T10:21:08.541152+0000","last_clean":"2024-04-20T10:21:08.541152+0000","last_became_active":"2024-04-20T10:21:02.626300+0000","last_became_peered":"2024-04-20T10:21:02.626300+0000","last_unstale":"2024-04-20T10:21:08.541152+0000","last_undegraded":"2024-04-20T10:21:08.541152+0000","last_fullsized":"2024-04-20T10:21:08.541152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:17:15.177678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.543489+0000","last_change":"2024-04-20T10:21:02.657102+0000","last_active":"2024-04-20T10:21:08.543489+0000","last_peered":"2024-04-20T10:21:08.543489+0000","last_clean":"2024-04-20T10:21:08.543489+0000","last_became_active":"2024-04-20T10:21:02.656810+0000","last_became_peered":"2024-04-20T10:21:02.656810+0000","last_unstale":"2024-04-20T10:21:08.543489+0000","last_undegraded":"2024-04-20T10:21:08.543489+0000","last_fullsized":"2024-04-20T10:21:08.543489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:04:04.238410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.039198+0000","last_change":"2024-04-20T10:21:02.639499+0000","last_active":"2024-04-20T10:21:08.039198+0000","last_peered":"2024-04-20T10:21:08.039198+0000","last_clean":"2024-04-20T10:21:08.039198+0000","last_became_active":"2024-04-20T10:21:02.638713+0000","last_became_peered":"2024-04-20T10:21:02.638713+0000","last_unstale":"2024-04-20T10:21:08.039198+0000","last_undegraded":"2024-04-20T10:21:08.039198+0000","last_fullsized":"2024-04-20T10:21:08.039198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:31:53.442042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.391493+0000","last_change":"2024-04-20T10:21:02.639481+0000","last_active":"2024-04-20T10:21:08.391493+0000","last_peered":"2024-04-20T10:21:08.391493+0000","last_clean":"2024-04-20T10:21:08.391493+0000","last_became_active":"2024-04-20T10:21:02.639198+0000","last_became_peered":"2024-04-20T10:21:02.639198+0000","last_unstale":"2024-04-20T10:21:08.391493+0000","last_undegraded":"2024-04-20T10:21:08.391493+0000","last_fullsized":"2024-04-20T10:21:08.391493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:35:54.235610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.883015+0000","last_change":"2024-04-20T10:21:02.640221+0000","last_active":"2024-04-20T10:21:07.883015+0000","last_peered":"2024-04-20T10:21:07.883015+0000","last_clean":"2024-04-20T10:21:07.883015+0000","last_became_active":"2024-04-20T10:21:02.639833+0000","last_became_peered":"2024-04-20T10:21:02.639833+0000","last_unstale":"2024-04-20T10:21:07.883015+0000","last_undegraded":"2024-04-20T10:21:07.883015+0000","last_fullsized":"2024-04-20T10:21:07.883015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:38:11.700275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542925+0000","last_change":"2024-04-20T10:21:02.640233+0000","last_active":"2024-04-20T10:21:08.542925+0000","last_peered":"2024-04-20T10:21:08.542925+0000","last_clean":"2024-04-20T10:21:08.542925+0000","last_became_active":"2024-04-20T10:21:02.639913+0000","last_became_peered":"2024-04-20T10:21:02.639913+0000","last_unstale":"2024-04-20T10:21:08.542925+0000","last_undegraded":"2024-04-20T10:21:08.542925+0000","last_fullsized":"2024-04-20T10:21:08.542925+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:45:33.753385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540514+0000","last_change":"2024-04-20T10:21:02.643714+0000","last_active":"2024-04-20T10:21:08.540514+0000","last_peered":"2024-04-20T10:21:08.540514+0000","last_clean":"2024-04-20T10:21:08.540514+0000","last_became_active":"2024-04-20T10:21:02.643403+0000","last_became_peered":"2024-04-20T10:21:02.643403+0000","last_unstale":"2024-04-20T10:21:08.540514+0000","last_undegraded":"2024-04-20T10:21:08.540514+0000","last_fullsized":"2024-04-20T10:21:08.540514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:27:44.403086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.440671+0000","last_change":"2024-04-20T10:21:02.643476+0000","last_active":"2024-04-20T10:21:08.440671+0000","last_peered":"2024-04-20T10:21:08.440671+0000","last_clean":"2024-04-20T10:21:08.440671+0000","last_became_active":"2024-04-20T10:21:02.643180+0000","last_became_peered":"2024-04-20T10:21:02.643180+0000","last_unstale":"2024-04-20T10:21:08.440671+0000","last_undegraded":"2024-04-20T10:21:08.440671+0000","last_fullsized":"2024-04-20T10:21:08.440671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:31:53.442042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.543269+0000","last_change":"2024-04-20T10:21:02.646081+0000","last_active":"2024-04-20T10:21:08.543269+0000","last_peered":"2024-04-20T10:21:08.543269+0000","last_clean":"2024-04-20T10:21:08.543269+0000","last_became_active":"2024-04-20T10:21:02.645798+0000","last_became_peered":"2024-04-20T10:21:02.645798+0000","last_unstale":"2024-04-20T10:21:08.543269+0000","last_undegraded":"2024-04-20T10:21:08.543269+0000","last_fullsized":"2024-04-20T10:21:08.543269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:45:52.178877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540989+0000","last_change":"2024-04-20T10:21:02.639147+0000","last_active":"2024-04-20T10:21:08.540989+0000","last_peered":"2024-04-20T10:21:08.540989+0000","last_clean":"2024-04-20T10:21:08.540989+0000","last_became_active":"2024-04-20T10:21:02.638861+0000","last_became_peered":"2024-04-20T10:21:02.638861+0000","last_unstale":"2024-04-20T10:21:08.540989+0000","last_undegraded":"2024-04-20T10:21:08.540989+0000","last_fullsized":"2024-04-20T10:21:08.540989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:22:36.605724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.887047+0000","last_change":"2024-04-20T10:21:02.638429+0000","last_active":"2024-04-20T10:21:07.887047+0000","last_peered":"2024-04-20T10:21:07.887047+0000","last_clean":"2024-04-20T10:21:07.887047+0000","last_became_active":"2024-04-20T10:21:02.638131+0000","last_became_peered":"2024-04-20T10:21:02.638131+0000","last_unstale":"2024-04-20T10:21:07.887047+0000","last_undegraded":"2024-04-20T10:21:07.887047+0000","last_fullsized":"2024-04-20T10:21:07.887047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:50:28.730495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.039192+0000","last_change":"2024-04-20T10:21:02.646725+0000","last_active":"2024-04-20T10:21:08.039192+0000","last_peered":"2024-04-20T10:21:08.039192+0000","last_clean":"2024-04-20T10:21:08.039192+0000","last_became_active":"2024-04-20T10:21:02.646439+0000","last_became_peered":"2024-04-20T10:21:02.646439+0000","last_unstale":"2024-04-20T10:21:08.039192+0000","last_undegraded":"2024-04-20T10:21:08.039192+0000","last_fullsized":"2024-04-20T10:21:08.039192+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:06:35.349863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.039464+0000","last_change":"2024-04-20T10:21:02.641401+0000","last_active":"2024-04-20T10:21:08.039464+0000","last_peered":"2024-04-20T10:21:08.039464+0000","last_clean":"2024-04-20T10:21:08.039464+0000","last_became_active":"2024-04-20T10:21:02.640962+0000","last_became_peered":"2024-04-20T10:21:02.640962+0000","last_unstale":"2024-04-20T10:21:08.039464+0000","last_undegraded":"2024-04-20T10:21:08.039464+0000","last_fullsized":"2024-04-20T10:21:08.039464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:22:36.605724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.391828+0000","last_change":"2024-04-20T10:21:02.654762+0000","last_active":"2024-04-20T10:21:08.391828+0000","last_peered":"2024-04-20T10:21:08.391828+0000","last_clean":"2024-04-20T10:21:08.391828+0000","last_became_active":"2024-04-20T10:21:02.654497+0000","last_became_peered":"2024-04-20T10:21:02.654497+0000","last_unstale":"2024-04-20T10:21:08.391828+0000","last_undegraded":"2024-04-20T10:21:08.391828+0000","last_fullsized":"2024-04-20T10:21:08.391828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:46:41.173563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540653+0000","last_change":"2024-04-20T10:21:02.630507+0000","last_active":"2024-04-20T10:21:08.540653+0000","last_peered":"2024-04-20T10:21:08.540653+0000","last_clean":"2024-04-20T10:21:08.540653+0000","last_became_active":"2024-04-20T10:21:02.626482+0000","last_became_peered":"2024-04-20T10:21:02.626482+0000","last_unstale":"2024-04-20T10:21:08.540653+0000","last_undegraded":"2024-04-20T10:21:08.540653+0000","last_fullsized":"2024-04-20T10:21:08.540653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:28:54.408773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.390187+0000","last_change":"2024-04-20T10:21:02.649796+0000","last_active":"2024-04-20T10:21:08.390187+0000","last_peered":"2024-04-20T10:21:08.390187+0000","last_clean":"2024-04-20T10:21:08.390187+0000","last_became_active":"2024-04-20T10:21:02.649541+0000","last_became_peered":"2024-04-20T10:21:02.649541+0000","last_unstale":"2024-04-20T10:21:08.390187+0000","last_undegraded":"2024-04-20T10:21:08.390187+0000","last_fullsized":"2024-04-20T10:21:08.390187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:02:26.716730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540951+0000","last_change":"2024-04-20T10:21:02.653552+0000","last_active":"2024-04-20T10:21:08.540951+0000","last_peered":"2024-04-20T10:21:08.540951+0000","last_clean":"2024-04-20T10:21:08.540951+0000","last_became_active":"2024-04-20T10:21:02.653288+0000","last_became_peered":"2024-04-20T10:21:02.653288+0000","last_unstale":"2024-04-20T10:21:08.540951+0000","last_undegraded":"2024-04-20T10:21:08.540951+0000","last_fullsized":"2024-04-20T10:21:08.540951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:49:26.986908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.390549+0000","last_change":"2024-04-20T10:21:02.633119+0000","last_active":"2024-04-20T10:21:08.390549+0000","last_peered":"2024-04-20T10:21:08.390549+0000","last_clean":"2024-04-20T10:21:08.390549+0000","last_became_active":"2024-04-20T10:21:02.632816+0000","last_became_peered":"2024-04-20T10:21:02.632816+0000","last_unstale":"2024-04-20T10:21:08.390549+0000","last_undegraded":"2024-04-20T10:21:08.390549+0000","last_fullsized":"2024-04-20T10:21:08.390549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:16:04.625809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541721+0000","last_change":"2024-04-20T10:21:02.638226+0000","last_active":"2024-04-20T10:21:08.541721+0000","last_peered":"2024-04-20T10:21:08.541721+0000","last_clean":"2024-04-20T10:21:08.541721+0000","last_became_active":"2024-04-20T10:21:02.637640+0000","last_became_peered":"2024-04-20T10:21:02.637640+0000","last_unstale":"2024-04-20T10:21:08.541721+0000","last_undegraded":"2024-04-20T10:21:08.541721+0000","last_fullsized":"2024-04-20T10:21:08.541721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:44:55.208482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.544254+0000","last_change":"2024-04-20T10:21:02.641981+0000","last_active":"2024-04-20T10:21:08.544254+0000","last_peered":"2024-04-20T10:21:08.544254+0000","last_clean":"2024-04-20T10:21:08.544254+0000","last_became_active":"2024-04-20T10:21:02.640073+0000","last_became_peered":"2024-04-20T10:21:02.640073+0000","last_unstale":"2024-04-20T10:21:08.544254+0000","last_undegraded":"2024-04-20T10:21:08.544254+0000","last_fullsized":"2024-04-20T10:21:08.544254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:28:54.408773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441547+0000","last_change":"2024-04-20T10:21:02.646636+0000","last_active":"2024-04-20T10:21:08.441547+0000","last_peered":"2024-04-20T10:21:08.441547+0000","last_clean":"2024-04-20T10:21:08.441547+0000","last_became_active":"2024-04-20T10:21:02.646359+0000","last_became_peered":"2024-04-20T10:21:02.646359+0000","last_unstale":"2024-04-20T10:21:08.441547+0000","last_undegraded":"2024-04-20T10:21:08.441547+0000","last_fullsized":"2024-04-20T10:21:08.441547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:22:36.605724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.391583+0000","last_change":"2024-04-20T10:21:02.652784+0000","last_active":"2024-04-20T10:21:08.391583+0000","last_peered":"2024-04-20T10:21:08.391583+0000","last_clean":"2024-04-20T10:21:08.391583+0000","last_became_active":"2024-04-20T10:21:02.652454+0000","last_became_peered":"2024-04-20T10:21:02.652454+0000","last_unstale":"2024-04-20T10:21:08.391583+0000","last_undegraded":"2024-04-20T10:21:08.391583+0000","last_fullsized":"2024-04-20T10:21:08.391583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:38:01.376108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542153,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":694509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542154,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":107401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542153,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":103649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542153,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":103685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542153,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":107401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542153,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":103559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542155,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":103719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542155,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":868352,"data_stored":694565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T10:21:31.694 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T10:21:31.937 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.937+0000 7fc75e200640 1 Processor -- start 2024-04-20T10:21:31.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.938+0000 7fc75e200640 1 -- start start 2024-04-20T10:21:31.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.939+0000 7fc75e200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc758156c20 0x7fc75806b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:31.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.939+0000 7fc75e200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc758141c60 con 0x7fc75815ca40 2024-04-20T10:21:31.939 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.939+0000 7fc75e200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7580adab0 con 0x7fc7580d0360 2024-04-20T10:21:31.939 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.939+0000 7fc75e200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fc75815cf90 con 0x7fc758156c20 2024-04-20T10:21:31.939 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.939+0000 7fc757400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc758156c20 0x7fc75806b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:31.939 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.940+0000 7fc757400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc758156c20 0x7fc75806b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54160/0 (socket says 172.21.15.121:54160) 2024-04-20T10:21:31.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.940+0000 7fc756a00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fc75815ca40 0x7fc7580ad3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:49982/0 (socket says 172.21.15.121:49982) 2024-04-20T10:21:31.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.940+0000 7fc757400640 1 -- 172.21.15.121:0/2914438715 learned_addr learned my addr 172.21.15.121:0/2914438715 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:31.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.940+0000 7fc757400640 1 -- 172.21.15.121:0/2914438715 >> v1:172.21.15.123:6789/0 conn(0x7fc7580d0360 legacy=0x7fc758142140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:31.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.940+0000 7fc757400640 1 -- 172.21.15.121:0/2914438715 >> v1:172.21.15.121:6789/0 conn(0x7fc75815ca40 legacy=0x7fc7580ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:31.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.940+0000 7fc757400640 1 -- 172.21.15.121:0/2914438715 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc75815d270 con 0x7fc758156c20 2024-04-20T10:21:31.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.941+0000 7fc757400640 1 --2- 172.21.15.121:0/2914438715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc758156c20 0x7fc75806b900 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc748009e50 tx=0x7fc74802bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22106b603fb5672 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:31.941 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.941+0000 7fc756000640 1 -- 172.21.15.121:0/2914438715 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74802cb90 con 0x7fc758156c20 2024-04-20T10:21:31.941 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.941+0000 7fc756000640 1 -- 172.21.15.121:0/2914438715 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc74802cd30 con 0x7fc758156c20 2024-04-20T10:21:31.941 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.941+0000 7fc756000640 1 -- 172.21.15.121:0/2914438715 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc748034610 con 0x7fc758156c20 2024-04-20T10:21:31.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.942+0000 7fc75e200640 1 -- 172.21.15.121:0/2914438715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc758156c20 msgr2=0x7fc75806b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:31.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.942+0000 7fc75e200640 1 --2- 172.21.15.121:0/2914438715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc758156c20 0x7fc75806b900 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc748009e50 tx=0x7fc74802bff0 comp rx=0 tx=0).stop 2024-04-20T10:21:31.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.942+0000 7fc75e200640 1 -- 172.21.15.121:0/2914438715 shutdown_connections 2024-04-20T10:21:31.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.942+0000 7fc75e200640 1 --2- 172.21.15.121:0/2914438715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc758156c20 0x7fc75806b900 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:31.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.942+0000 7fc75e200640 1 -- 172.21.15.121:0/2914438715 >> 172.21.15.121:0/2914438715 conn(0x7fc7580b3350 msgr2=0x7fc7580b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:31.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.942+0000 7fc75e200640 1 -- 172.21.15.121:0/2914438715 shutdown_connections 2024-04-20T10:21:31.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.942+0000 7fc75e200640 1 -- 172.21.15.121:0/2914438715 wait complete. 2024-04-20T10:21:31.944 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.944+0000 7fc75e200640 1 Processor -- start 2024-04-20T10:21:31.944 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.945+0000 7fc75e200640 1 -- start start 2024-04-20T10:21:31.946 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.946+0000 7fc75e200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7580d0360 0x7fc7581f4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:31.946 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.946+0000 7fc75e200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc758156c20 0x7fc7580cd540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:31.946 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.946+0000 7fc75e200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc75815ca40 0x7fc7580ce390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:31.946 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.946+0000 7fc75e200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc75806be40 con 0x7fc7580d0360 2024-04-20T10:21:31.946 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.946+0000 7fc75e200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc75815d830 con 0x7fc758156c20 2024-04-20T10:21:31.946 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.946+0000 7fc75e200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fc75806bfc0 con 0x7fc75815ca40 2024-04-20T10:21:31.946 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.946+0000 7fc757400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7580d0360 0x7fc7581f4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:31.947 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.946+0000 7fc757400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7580d0360 0x7fc7581f4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54170/0 (socket says 172.21.15.121:54170) 2024-04-20T10:21:31.947 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.946+0000 7fc757400640 1 -- 172.21.15.121:0/2012584694 learned_addr learned my addr 172.21.15.121:0/2012584694 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:31.947 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.946+0000 7fc757400640 1 -- 172.21.15.121:0/2012584694 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc75815ca40 msgr2=0x7fc7580ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:31.947 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.946+0000 7fc757400640 1 --2- 172.21.15.121:0/2012584694 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc75815ca40 0x7fc7580ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:31.947 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.946+0000 7fc757400640 1 -- 172.21.15.121:0/2012584694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc758156c20 msgr2=0x7fc7580cd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:31.947 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.946+0000 7fc757400640 1 --2- 172.21.15.121:0/2012584694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc758156c20 0x7fc7580cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:31.947 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.946+0000 7fc757400640 1 -- 172.21.15.121:0/2012584694 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7580cdd90 con 0x7fc7580d0360 2024-04-20T10:21:31.947 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.947+0000 7fc757400640 1 --2- 172.21.15.121:0/2012584694 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7580d0360 0x7fc7581f4ab0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc748000f80 tx=0x7fc748004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:31.948 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.947+0000 7fc74f400640 1 -- 172.21.15.121:0/2012584694 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc748043020 con 0x7fc7580d0360 2024-04-20T10:21:31.948 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.947+0000 7fc74f400640 1 -- 172.21.15.121:0/2012584694 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc74803e450 con 0x7fc7580d0360 2024-04-20T10:21:31.948 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.947+0000 7fc74f400640 1 -- 172.21.15.121:0/2012584694 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc74803e840 con 0x7fc7580d0360 2024-04-20T10:21:31.948 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.947+0000 7fc75e200640 1 -- 172.21.15.121:0/2012584694 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc75806dc50 con 0x7fc7580d0360 2024-04-20T10:21:31.948 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.947+0000 7fc75e200640 1 -- 172.21.15.121:0/2012584694 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc75806e210 con 0x7fc7580d0360 2024-04-20T10:21:31.948 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.949+0000 7fc75e200640 1 -- 172.21.15.121:0/2012584694 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc758068980 con 0x7fc7580d0360 2024-04-20T10:21:31.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.950+0000 7fc74f400640 1 -- 172.21.15.121:0/2012584694 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc748046020 con 0x7fc7580d0360 2024-04-20T10:21:31.951 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.951+0000 7fc74f400640 1 --2- 172.21.15.121:0/2012584694 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fc73406c8f0 0x7fc73406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:31.951 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.951+0000 7fc74f400640 1 -- 172.21.15.121:0/2012584694 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc7480ae6f0 con 0x7fc7580d0360 2024-04-20T10:21:31.951 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.951+0000 7fc756a00640 1 --2- 172.21.15.121:0/2012584694 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fc73406c8f0 0x7fc73406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:31.951 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.952+0000 7fc756a00640 1 --2- 172.21.15.121:0/2012584694 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fc73406c8f0 0x7fc73406edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc7581f8560 tx=0x7fc7400079c0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:31.955 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:31.955+0000 7fc74f400640 1 -- 172.21.15.121:0/2012584694 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc74804d030 con 0x7fc7580d0360 2024-04-20T10:21:32.155 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.155+0000 7fc75e200640 1 -- 172.21.15.121:0/2012584694 --> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc75815def0 con 0x7fc73406c8f0 2024-04-20T10:21:32.177 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.178+0000 7fc74f400640 1 -- 172.21.15.121:0/2012584694 <== mgr.4103 v2:172.21.15.121:6832/1408233272 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332538 (secure 0 0 0) 0x7fc75815def0 con 0x7fc73406c8f0 2024-04-20T10:21:32.182 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:21:32.183 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-04-20T10:21:32.185 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.184+0000 7fc75e200640 1 -- 172.21.15.121:0/2012584694 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fc73406c8f0 msgr2=0x7fc73406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:32.185 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.184+0000 7fc75e200640 1 --2- 172.21.15.121:0/2012584694 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fc73406c8f0 0x7fc73406edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc7581f8560 tx=0x7fc7400079c0 comp rx=0 tx=0).stop 2024-04-20T10:21:32.185 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.184+0000 7fc75e200640 1 -- 172.21.15.121:0/2012584694 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7580d0360 msgr2=0x7fc7581f4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:32.185 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.184+0000 7fc75e200640 1 --2- 172.21.15.121:0/2012584694 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7580d0360 0x7fc7581f4ab0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc748000f80 tx=0x7fc748004a40 comp rx=0 tx=0).stop 2024-04-20T10:21:32.185 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.184+0000 7fc75e200640 1 -- 172.21.15.121:0/2012584694 shutdown_connections 2024-04-20T10:21:32.185 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.184+0000 7fc75e200640 1 --2- 172.21.15.121:0/2012584694 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc75815ca40 0x7fc7580ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:32.185 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.184+0000 7fc75e200640 1 --2- 172.21.15.121:0/2012584694 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fc73406c8f0 0x7fc73406edb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:32.186 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.184+0000 7fc75e200640 1 --2- 172.21.15.121:0/2012584694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc758156c20 0x7fc7580cd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:32.186 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.184+0000 7fc75e200640 1 --2- 172.21.15.121:0/2012584694 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc7580d0360 0x7fc7581f4ab0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:32.186 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.184+0000 7fc75e200640 1 -- 172.21.15.121:0/2012584694 >> 172.21.15.121:0/2012584694 conn(0x7fc7580b3350 msgr2=0x7fc7580af1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:32.186 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.185+0000 7fc75e200640 1 -- 172.21.15.121:0/2012584694 shutdown_connections 2024-04-20T10:21:32.186 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.185+0000 7fc75e200640 1 -- 172.21.15.121:0/2012584694 wait complete. 2024-04-20T10:21:32.203 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-04-20T10:21:31.118017+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219436,"kb_used_data":3272,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755284,"statfs":{"total":773094113280,"available":772869410816,"internally_reserved":0,"allocated":3350528,"data_stored":2018488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004173"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.544361+0000","last_change":"2024-04-20T10:21:01.620269+0000","last_active":"2024-04-20T10:21:08.544361+0000","last_peered":"2024-04-20T10:21:08.544361+0000","last_clean":"2024-04-20T10:21:08.544361+0000","last_became_active":"2024-04-20T10:21:01.619751+0000","last_became_peered":"2024-04-20T10:21:01.619751+0000","last_unstale":"2024-04-20T10:21:08.544361+0000","last_undegraded":"2024-04-20T10:21:08.544361+0000","last_fullsized":"2024-04-20T10:21:08.544361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:28:01.100842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.544942+0000","last_change":"2024-04-20T10:21:02.642147+0000","last_active":"2024-04-20T10:21:08.544942+0000","last_peered":"2024-04-20T10:21:08.544942+0000","last_clean":"2024-04-20T10:21:08.544942+0000","last_became_active":"2024-04-20T10:21:02.641636+0000","last_became_peered":"2024-04-20T10:21:02.641636+0000","last_unstale":"2024-04-20T10:21:08.544942+0000","last_undegraded":"2024-04-20T10:21:08.544942+0000","last_fullsized":"2024-04-20T10:21:08.544942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:53:57.418208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540582+0000","last_change":"2024-04-20T10:21:06.909318+0000","last_active":"2024-04-20T10:21:08.540582+0000","last_peered":"2024-04-20T10:21:08.540582+0000","last_clean":"2024-04-20T10:21:08.540582+0000","last_became_active":"2024-04-20T10:21:06.907900+0000","last_became_peered":"2024-04-20T10:21:06.907900+0000","last_unstale":"2024-04-20T10:21:08.540582+0000","last_undegraded":"2024-04-20T10:21:08.540582+0000","last_fullsized":"2024-04-20T10:21:08.540582+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:35:58.510637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.545148+0000","last_change":"2024-04-20T10:21:01.609344+0000","last_active":"2024-04-20T10:21:08.545148+0000","last_peered":"2024-04-20T10:21:08.545148+0000","last_clean":"2024-04-20T10:21:08.545148+0000","last_became_active":"2024-04-20T10:21:01.608863+0000","last_became_peered":"2024-04-20T10:21:01.608863+0000","last_unstale":"2024-04-20T10:21:08.545148+0000","last_undegraded":"2024-04-20T10:21:08.545148+0000","last_fullsized":"2024-04-20T10:21:08.545148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:32:02.981747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.543880+0000","last_change":"2024-04-20T10:21:02.649607+0000","last_active":"2024-04-20T10:21:08.543880+0000","last_peered":"2024-04-20T10:21:08.543880+0000","last_clean":"2024-04-20T10:21:08.543880+0000","last_became_active":"2024-04-20T10:21:02.649283+0000","last_became_peered":"2024-04-20T10:21:02.649283+0000","last_unstale":"2024-04-20T10:21:08.543880+0000","last_undegraded":"2024-04-20T10:21:08.543880+0000","last_fullsized":"2024-04-20T10:21:08.543880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:06:35.349863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.885288+0000","last_change":"2024-04-20T10:21:06.910300+0000","last_active":"2024-04-20T10:21:07.885288+0000","last_peered":"2024-04-20T10:21:07.885288+0000","last_clean":"2024-04-20T10:21:07.885288+0000","last_became_active":"2024-04-20T10:21:06.909341+0000","last_became_peered":"2024-04-20T10:21:06.909341+0000","last_unstale":"2024-04-20T10:21:07.885288+0000","last_undegraded":"2024-04-20T10:21:07.885288+0000","last_fullsized":"2024-04-20T10:21:07.885288+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:06:39.624891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.391510+0000","last_change":"2024-04-20T10:21:01.623714+0000","last_active":"2024-04-20T10:21:08.391510+0000","last_peered":"2024-04-20T10:21:08.391510+0000","last_clean":"2024-04-20T10:21:08.391510+0000","last_became_active":"2024-04-20T10:21:01.623427+0000","last_became_peered":"2024-04-20T10:21:01.623427+0000","last_unstale":"2024-04-20T10:21:08.391510+0000","last_undegraded":"2024-04-20T10:21:08.391510+0000","last_fullsized":"2024-04-20T10:21:08.391510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:25:59.939855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541788+0000","last_change":"2024-04-20T10:21:02.636522+0000","last_active":"2024-04-20T10:21:08.541788+0000","last_peered":"2024-04-20T10:21:08.541788+0000","last_clean":"2024-04-20T10:21:08.541788+0000","last_became_active":"2024-04-20T10:21:02.633235+0000","last_became_peered":"2024-04-20T10:21:02.633235+0000","last_unstale":"2024-04-20T10:21:08.541788+0000","last_undegraded":"2024-04-20T10:21:08.541788+0000","last_fullsized":"2024-04-20T10:21:08.541788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:55:00.905038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.886695+0000","last_change":"2024-04-20T10:21:06.911285+0000","last_active":"2024-04-20T10:21:07.886695+0000","last_peered":"2024-04-20T10:21:07.886695+0000","last_clean":"2024-04-20T10:21:07.886695+0000","last_became_active":"2024-04-20T10:21:06.908548+0000","last_became_peered":"2024-04-20T10:21:06.908548+0000","last_unstale":"2024-04-20T10:21:07.886695+0000","last_undegraded":"2024-04-20T10:21:07.886695+0000","last_fullsized":"2024-04-20T10:21:07.886695+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:09:46.241216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.391163+0000","last_change":"2024-04-20T10:21:01.626685+0000","last_active":"2024-04-20T10:21:08.391163+0000","last_peered":"2024-04-20T10:21:08.391163+0000","last_clean":"2024-04-20T10:21:08.391163+0000","last_became_active":"2024-04-20T10:21:01.626354+0000","last_became_peered":"2024-04-20T10:21:01.626354+0000","last_unstale":"2024-04-20T10:21:08.391163+0000","last_undegraded":"2024-04-20T10:21:08.391163+0000","last_fullsized":"2024-04-20T10:21:08.391163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:07:50.284372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542097+0000","last_change":"2024-04-20T10:21:02.636800+0000","last_active":"2024-04-20T10:21:08.542097+0000","last_peered":"2024-04-20T10:21:08.542097+0000","last_clean":"2024-04-20T10:21:08.542097+0000","last_became_active":"2024-04-20T10:21:02.636523+0000","last_became_peered":"2024-04-20T10:21:02.636523+0000","last_unstale":"2024-04-20T10:21:08.542097+0000","last_undegraded":"2024-04-20T10:21:08.542097+0000","last_fullsized":"2024-04-20T10:21:08.542097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:31:07.363373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.885879+0000","last_change":"2024-04-20T10:21:06.893126+0000","last_active":"2024-04-20T10:21:07.885879+0000","last_peered":"2024-04-20T10:21:07.885879+0000","last_clean":"2024-04-20T10:21:07.885879+0000","last_became_active":"2024-04-20T10:21:06.892722+0000","last_became_peered":"2024-04-20T10:21:06.892722+0000","last_unstale":"2024-04-20T10:21:07.885879+0000","last_undegraded":"2024-04-20T10:21:07.885879+0000","last_fullsized":"2024-04-20T10:21:07.885879+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:10:19.481713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.038387+0000","last_change":"2024-04-20T10:21:02.633548+0000","last_active":"2024-04-20T10:21:08.038387+0000","last_peered":"2024-04-20T10:21:08.038387+0000","last_clean":"2024-04-20T10:21:08.038387+0000","last_became_active":"2024-04-20T10:21:02.631142+0000","last_became_peered":"2024-04-20T10:21:02.631142+0000","last_unstale":"2024-04-20T10:21:08.038387+0000","last_undegraded":"2024-04-20T10:21:08.038387+0000","last_fullsized":"2024-04-20T10:21:08.038387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:14:16.934710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.390972+0000","last_change":"2024-04-20T10:21:01.622399+0000","last_active":"2024-04-20T10:21:08.390972+0000","last_peered":"2024-04-20T10:21:08.390972+0000","last_clean":"2024-04-20T10:21:08.390972+0000","last_became_active":"2024-04-20T10:21:01.622072+0000","last_became_peered":"2024-04-20T10:21:01.622072+0000","last_unstale":"2024-04-20T10:21:08.390972+0000","last_undegraded":"2024-04-20T10:21:08.390972+0000","last_fullsized":"2024-04-20T10:21:08.390972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:35:02.394948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.887355+0000","last_change":"2024-04-20T10:21:06.897024+0000","last_active":"2024-04-20T10:21:07.887355+0000","last_peered":"2024-04-20T10:21:07.887355+0000","last_clean":"2024-04-20T10:21:07.887355+0000","last_became_active":"2024-04-20T10:21:06.896315+0000","last_became_peered":"2024-04-20T10:21:06.896315+0000","last_unstale":"2024-04-20T10:21:07.887355+0000","last_undegraded":"2024-04-20T10:21:07.887355+0000","last_fullsized":"2024-04-20T10:21:07.887355+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:21:29.812831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.544646+0000","last_change":"2024-04-20T10:21:02.646217+0000","last_active":"2024-04-20T10:21:08.544646+0000","last_peered":"2024-04-20T10:21:08.544646+0000","last_clean":"2024-04-20T10:21:08.544646+0000","last_became_active":"2024-04-20T10:21:02.645926+0000","last_became_peered":"2024-04-20T10:21:02.645926+0000","last_unstale":"2024-04-20T10:21:08.544646+0000","last_undegraded":"2024-04-20T10:21:08.544646+0000","last_fullsized":"2024-04-20T10:21:08.544646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:10:15.206686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441134+0000","last_change":"2024-04-20T10:21:01.617948+0000","last_active":"2024-04-20T10:21:08.441134+0000","last_peered":"2024-04-20T10:21:08.441134+0000","last_clean":"2024-04-20T10:21:08.441134+0000","last_became_active":"2024-04-20T10:21:01.617628+0000","last_became_peered":"2024-04-20T10:21:01.617628+0000","last_unstale":"2024-04-20T10:21:08.441134+0000","last_undegraded":"2024-04-20T10:21:08.441134+0000","last_fullsized":"2024-04-20T10:21:08.441134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:55:41.208744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.886801+0000","last_change":"2024-04-20T10:21:06.900164+0000","last_active":"2024-04-20T10:21:07.886801+0000","last_peered":"2024-04-20T10:21:07.886801+0000","last_clean":"2024-04-20T10:21:07.886801+0000","last_became_active":"2024-04-20T10:21:06.899783+0000","last_became_peered":"2024-04-20T10:21:06.899783+0000","last_unstale":"2024-04-20T10:21:07.886801+0000","last_undegraded":"2024-04-20T10:21:07.886801+0000","last_fullsized":"2024-04-20T10:21:07.886801+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:54:01.693236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541211+0000","last_change":"2024-04-20T10:21:02.629982+0000","last_active":"2024-04-20T10:21:08.541211+0000","last_peered":"2024-04-20T10:21:08.541211+0000","last_clean":"2024-04-20T10:21:08.541211+0000","last_became_active":"2024-04-20T10:21:02.629729+0000","last_became_peered":"2024-04-20T10:21:02.629729+0000","last_unstale":"2024-04-20T10:21:08.541211+0000","last_undegraded":"2024-04-20T10:21:08.541211+0000","last_fullsized":"2024-04-20T10:21:08.541211+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:31:53.442042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.440779+0000","last_change":"2024-04-20T10:21:01.604313+0000","last_active":"2024-04-20T10:21:08.440779+0000","last_peered":"2024-04-20T10:21:08.440779+0000","last_clean":"2024-04-20T10:21:08.440779+0000","last_became_active":"2024-04-20T10:21:01.603629+0000","last_became_peered":"2024-04-20T10:21:01.603629+0000","last_unstale":"2024-04-20T10:21:08.440779+0000","last_undegraded":"2024-04-20T10:21:08.440779+0000","last_fullsized":"2024-04-20T10:21:08.440779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:01:19.566351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441622+0000","last_change":"2024-04-20T10:21:06.903477+0000","last_active":"2024-04-20T10:21:08.441622+0000","last_peered":"2024-04-20T10:21:08.441622+0000","last_clean":"2024-04-20T10:21:08.441622+0000","last_became_active":"2024-04-20T10:21:06.903010+0000","last_became_peered":"2024-04-20T10:21:06.903010+0000","last_unstale":"2024-04-20T10:21:08.441622+0000","last_undegraded":"2024-04-20T10:21:08.441622+0000","last_fullsized":"2024-04-20T10:21:08.441622+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:46:45.448591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.885935+0000","last_change":"2024-04-20T10:21:02.614476+0000","last_active":"2024-04-20T10:21:07.885935+0000","last_peered":"2024-04-20T10:21:07.885935+0000","last_clean":"2024-04-20T10:21:07.885935+0000","last_became_active":"2024-04-20T10:21:02.614140+0000","last_became_peered":"2024-04-20T10:21:02.614140+0000","last_unstale":"2024-04-20T10:21:07.885935+0000","last_undegraded":"2024-04-20T10:21:07.885935+0000","last_fullsized":"2024-04-20T10:21:07.885935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:01:20.579875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.389864+0000","last_change":"2024-04-20T10:21:01.619958+0000","last_active":"2024-04-20T10:21:08.389864+0000","last_peered":"2024-04-20T10:21:08.389864+0000","last_clean":"2024-04-20T10:21:08.389864+0000","last_became_active":"2024-04-20T10:21:01.619688+0000","last_became_peered":"2024-04-20T10:21:01.619688+0000","last_unstale":"2024-04-20T10:21:08.389864+0000","last_undegraded":"2024-04-20T10:21:08.389864+0000","last_fullsized":"2024-04-20T10:21:08.389864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:45:51.165353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.887140+0000","last_change":"2024-04-20T10:21:06.908789+0000","last_active":"2024-04-20T10:21:07.887140+0000","last_peered":"2024-04-20T10:21:07.887140+0000","last_clean":"2024-04-20T10:21:07.887140+0000","last_became_active":"2024-04-20T10:21:06.908405+0000","last_became_peered":"2024-04-20T10:21:06.908405+0000","last_unstale":"2024-04-20T10:21:07.887140+0000","last_undegraded":"2024-04-20T10:21:07.887140+0000","last_fullsized":"2024-04-20T10:21:07.887140+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:02:30.991758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.544406+0000","last_change":"2024-04-20T10:21:02.649177+0000","last_active":"2024-04-20T10:21:08.544406+0000","last_peered":"2024-04-20T10:21:08.544406+0000","last_clean":"2024-04-20T10:21:08.544406+0000","last_became_active":"2024-04-20T10:21:02.648861+0000","last_became_peered":"2024-04-20T10:21:02.648861+0000","last_unstale":"2024-04-20T10:21:08.544406+0000","last_undegraded":"2024-04-20T10:21:08.544406+0000","last_fullsized":"2024-04-20T10:21:08.544406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:31:53.442042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.039602+0000","last_change":"2024-04-20T10:21:01.621228+0000","last_active":"2024-04-20T10:21:08.039602+0000","last_peered":"2024-04-20T10:21:08.039602+0000","last_clean":"2024-04-20T10:21:08.039602+0000","last_became_active":"2024-04-20T10:21:01.620525+0000","last_became_peered":"2024-04-20T10:21:01.620525+0000","last_unstale":"2024-04-20T10:21:08.039602+0000","last_undegraded":"2024-04-20T10:21:08.039602+0000","last_fullsized":"2024-04-20T10:21:08.039602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:50:27.716971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.885713+0000","last_change":"2024-04-20T10:21:06.903693+0000","last_active":"2024-04-20T10:21:07.885713+0000","last_peered":"2024-04-20T10:21:07.885713+0000","last_clean":"2024-04-20T10:21:07.885713+0000","last_became_active":"2024-04-20T10:21:06.903087+0000","last_became_peered":"2024-04-20T10:21:06.903087+0000","last_unstale":"2024-04-20T10:21:07.885713+0000","last_undegraded":"2024-04-20T10:21:07.885713+0000","last_fullsized":"2024-04-20T10:21:07.885713+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:22:40.880752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542691+0000","last_change":"2024-04-20T10:21:02.632360+0000","last_active":"2024-04-20T10:21:08.542691+0000","last_peered":"2024-04-20T10:21:08.542691+0000","last_clean":"2024-04-20T10:21:08.542691+0000","last_became_active":"2024-04-20T10:21:02.632062+0000","last_became_peered":"2024-04-20T10:21:02.632062+0000","last_unstale":"2024-04-20T10:21:08.542691+0000","last_undegraded":"2024-04-20T10:21:08.542691+0000","last_fullsized":"2024-04-20T10:21:08.542691+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:06:11.427561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441791+0000","last_change":"2024-04-20T10:21:01.622049+0000","last_active":"2024-04-20T10:21:08.441791+0000","last_peered":"2024-04-20T10:21:08.441791+0000","last_clean":"2024-04-20T10:21:08.441791+0000","last_became_active":"2024-04-20T10:21:01.621779+0000","last_became_peered":"2024-04-20T10:21:01.621779+0000","last_unstale":"2024-04-20T10:21:08.441791+0000","last_undegraded":"2024-04-20T10:21:08.441791+0000","last_fullsized":"2024-04-20T10:21:08.441791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:17:23.552169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540642+0000","last_change":"2024-04-20T10:21:06.908686+0000","last_active":"2024-04-20T10:21:08.540642+0000","last_peered":"2024-04-20T10:21:08.540642+0000","last_clean":"2024-04-20T10:21:08.540642+0000","last_became_active":"2024-04-20T10:21:06.907525+0000","last_became_peered":"2024-04-20T10:21:06.907525+0000","last_unstale":"2024-04-20T10:21:08.540642+0000","last_undegraded":"2024-04-20T10:21:08.540642+0000","last_fullsized":"2024-04-20T10:21:08.540642+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:38:05.651135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441452+0000","last_change":"2024-04-20T10:21:02.633193+0000","last_active":"2024-04-20T10:21:08.441452+0000","last_peered":"2024-04-20T10:21:08.441452+0000","last_clean":"2024-04-20T10:21:08.441452+0000","last_became_active":"2024-04-20T10:21:02.632896+0000","last_became_peered":"2024-04-20T10:21:02.632896+0000","last_unstale":"2024-04-20T10:21:08.441452+0000","last_undegraded":"2024-04-20T10:21:08.441452+0000","last_fullsized":"2024-04-20T10:21:08.441452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:06:35.349863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.391636+0000","last_change":"2024-04-20T10:21:01.624712+0000","last_active":"2024-04-20T10:21:08.391636+0000","last_peered":"2024-04-20T10:21:08.391636+0000","last_clean":"2024-04-20T10:21:08.391636+0000","last_became_active":"2024-04-20T10:21:01.624381+0000","last_became_peered":"2024-04-20T10:21:01.624381+0000","last_unstale":"2024-04-20T10:21:08.391636+0000","last_undegraded":"2024-04-20T10:21:08.391636+0000","last_fullsized":"2024-04-20T10:21:08.391636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:17:23.552169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.543354+0000","last_change":"2024-04-20T10:21:06.903399+0000","last_active":"2024-04-20T10:21:08.543354+0000","last_peered":"2024-04-20T10:21:08.543354+0000","last_clean":"2024-04-20T10:21:08.543354+0000","last_became_active":"2024-04-20T10:21:06.902529+0000","last_became_peered":"2024-04-20T10:21:06.902529+0000","last_unstale":"2024-04-20T10:21:08.543354+0000","last_undegraded":"2024-04-20T10:21:08.543354+0000","last_fullsized":"2024-04-20T10:21:08.543354+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:52:28.298927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542324+0000","last_change":"2024-04-20T10:21:02.637139+0000","last_active":"2024-04-20T10:21:08.542324+0000","last_peered":"2024-04-20T10:21:08.542324+0000","last_clean":"2024-04-20T10:21:08.542324+0000","last_became_active":"2024-04-20T10:21:02.636849+0000","last_became_peered":"2024-04-20T10:21:02.636849+0000","last_unstale":"2024-04-20T10:21:08.542324+0000","last_undegraded":"2024-04-20T10:21:08.542324+0000","last_fullsized":"2024-04-20T10:21:08.542324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:44:52.759993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441617+0000","last_change":"2024-04-20T10:21:01.606832+0000","last_active":"2024-04-20T10:21:08.441617+0000","last_peered":"2024-04-20T10:21:08.441617+0000","last_clean":"2024-04-20T10:21:08.441617+0000","last_became_active":"2024-04-20T10:21:01.606562+0000","last_became_peered":"2024-04-20T10:21:01.606562+0000","last_unstale":"2024-04-20T10:21:08.441617+0000","last_undegraded":"2024-04-20T10:21:08.441617+0000","last_fullsized":"2024-04-20T10:21:08.441617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:50:27.716971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.440686+0000","last_change":"2024-04-20T10:21:06.894216+0000","last_active":"2024-04-20T10:21:08.440686+0000","last_peered":"2024-04-20T10:21:08.440686+0000","last_clean":"2024-04-20T10:21:08.440686+0000","last_became_active":"2024-04-20T10:21:06.893762+0000","last_became_peered":"2024-04-20T10:21:06.893762+0000","last_unstale":"2024-04-20T10:21:08.440686+0000","last_undegraded":"2024-04-20T10:21:08.440686+0000","last_fullsized":"2024-04-20T10:21:08.440686+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:31:30.034892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.887359+0000","last_change":"2024-04-20T10:21:02.648574+0000","last_active":"2024-04-20T10:21:07.887359+0000","last_peered":"2024-04-20T10:21:07.887359+0000","last_clean":"2024-04-20T10:21:07.887359+0000","last_became_active":"2024-04-20T10:21:02.648275+0000","last_became_peered":"2024-04-20T10:21:02.648275+0000","last_unstale":"2024-04-20T10:21:07.887359+0000","last_undegraded":"2024-04-20T10:21:07.887359+0000","last_fullsized":"2024-04-20T10:21:07.887359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:55:42.222268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541444+0000","last_change":"2024-04-20T10:21:01.615460+0000","last_active":"2024-04-20T10:21:08.541444+0000","last_peered":"2024-04-20T10:21:08.541444+0000","last_clean":"2024-04-20T10:21:08.541444+0000","last_became_active":"2024-04-20T10:21:01.614405+0000","last_became_peered":"2024-04-20T10:21:01.614405+0000","last_unstale":"2024-04-20T10:21:08.541444+0000","last_undegraded":"2024-04-20T10:21:08.541444+0000","last_fullsized":"2024-04-20T10:21:08.541444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:51:46.790176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441597+0000","last_change":"2024-04-20T10:21:06.899961+0000","last_active":"2024-04-20T10:21:08.441597+0000","last_peered":"2024-04-20T10:21:08.441597+0000","last_clean":"2024-04-20T10:21:08.441597+0000","last_became_active":"2024-04-20T10:21:06.899574+0000","last_became_peered":"2024-04-20T10:21:06.899574+0000","last_unstale":"2024-04-20T10:21:08.441597+0000","last_undegraded":"2024-04-20T10:21:08.441597+0000","last_fullsized":"2024-04-20T10:21:08.441597+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:59:28.847178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.440949+0000","last_change":"2024-04-20T10:21:02.627320+0000","last_active":"2024-04-20T10:21:08.440949+0000","last_peered":"2024-04-20T10:21:08.440949+0000","last_clean":"2024-04-20T10:21:08.440949+0000","last_became_active":"2024-04-20T10:21:02.627008+0000","last_became_peered":"2024-04-20T10:21:02.627008+0000","last_unstale":"2024-04-20T10:21:08.440949+0000","last_undegraded":"2024-04-20T10:21:08.440949+0000","last_fullsized":"2024-04-20T10:21:08.440949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:01:55.363202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.881645+0000","last_change":"2024-04-20T10:21:01.611725+0000","last_active":"2024-04-20T10:21:07.881645+0000","last_peered":"2024-04-20T10:21:07.881645+0000","last_clean":"2024-04-20T10:21:07.881645+0000","last_became_active":"2024-04-20T10:21:01.609982+0000","last_became_peered":"2024-04-20T10:21:01.609982+0000","last_unstale":"2024-04-20T10:21:07.881645+0000","last_undegraded":"2024-04-20T10:21:07.881645+0000","last_fullsized":"2024-04-20T10:21:07.881645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:32:02.981747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441859+0000","last_change":"2024-04-20T10:21:06.897904+0000","last_active":"2024-04-20T10:21:08.441859+0000","last_peered":"2024-04-20T10:21:08.441859+0000","last_clean":"2024-04-20T10:21:08.441859+0000","last_became_active":"2024-04-20T10:21:06.896628+0000","last_became_peered":"2024-04-20T10:21:06.896628+0000","last_unstale":"2024-04-20T10:21:08.441859+0000","last_undegraded":"2024-04-20T10:21:08.441859+0000","last_fullsized":"2024-04-20T10:21:08.441859+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:35:58.510637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542234+0000","last_change":"2024-04-20T10:21:02.651679+0000","last_active":"2024-04-20T10:21:08.542234+0000","last_peered":"2024-04-20T10:21:08.542234+0000","last_clean":"2024-04-20T10:21:08.542234+0000","last_became_active":"2024-04-20T10:21:02.651336+0000","last_became_peered":"2024-04-20T10:21:02.651336+0000","last_unstale":"2024-04-20T10:21:08.542234+0000","last_undegraded":"2024-04-20T10:21:08.542234+0000","last_fullsized":"2024-04-20T10:21:08.542234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:17:24.565693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.389935+0000","last_change":"2024-04-20T10:21:01.620091+0000","last_active":"2024-04-20T10:21:08.389935+0000","last_peered":"2024-04-20T10:21:08.389935+0000","last_clean":"2024-04-20T10:21:08.389935+0000","last_became_active":"2024-04-20T10:21:01.619747+0000","last_became_peered":"2024-04-20T10:21:01.619747+0000","last_unstale":"2024-04-20T10:21:08.389935+0000","last_undegraded":"2024-04-20T10:21:08.389935+0000","last_fullsized":"2024-04-20T10:21:08.389935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:38:10.686751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.544184+0000","last_change":"2024-04-20T10:21:06.902502+0000","last_active":"2024-04-20T10:21:08.544184+0000","last_peered":"2024-04-20T10:21:08.544184+0000","last_clean":"2024-04-20T10:21:08.544184+0000","last_became_active":"2024-04-20T10:21:06.899513+0000","last_became_peered":"2024-04-20T10:21:06.899513+0000","last_unstale":"2024-04-20T10:21:08.544184+0000","last_undegraded":"2024-04-20T10:21:08.544184+0000","last_fullsized":"2024-04-20T10:21:08.544184+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:46:45.448591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541620+0000","last_change":"2024-04-20T10:21:02.628521+0000","last_active":"2024-04-20T10:21:08.541620+0000","last_peered":"2024-04-20T10:21:08.541620+0000","last_clean":"2024-04-20T10:21:08.541620+0000","last_became_active":"2024-04-20T10:21:02.626414+0000","last_became_peered":"2024-04-20T10:21:02.626414+0000","last_unstale":"2024-04-20T10:21:08.541620+0000","last_undegraded":"2024-04-20T10:21:08.541620+0000","last_fullsized":"2024-04-20T10:21:08.541620+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:01:55.363202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541447+0000","last_change":"2024-04-20T10:21:01.610399+0000","last_active":"2024-04-20T10:21:08.541447+0000","last_peered":"2024-04-20T10:21:08.541447+0000","last_clean":"2024-04-20T10:21:08.541447+0000","last_became_active":"2024-04-20T10:21:01.610116+0000","last_became_peered":"2024-04-20T10:21:01.610116+0000","last_unstale":"2024-04-20T10:21:08.541447+0000","last_undegraded":"2024-04-20T10:21:08.541447+0000","last_fullsized":"2024-04-20T10:21:08.541447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:01:18.578396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.390168+0000","last_change":"2024-04-20T10:20:52.916129+0000","last_active":"2024-04-20T10:21:08.390168+0000","last_peered":"2024-04-20T10:21:08.390168+0000","last_clean":"2024-04-20T10:21:08.390168+0000","last_became_active":"2024-04-20T10:20:52.913922+0000","last_became_peered":"2024-04-20T10:20:52.913922+0000","last_unstale":"2024-04-20T10:21:08.390168+0000","last_undegraded":"2024-04-20T10:21:08.390168+0000","last_fullsized":"2024-04-20T10:21:08.390168+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:20:51.267588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:20:51.267588+0000","last_clean_scrub_stamp":"2024-04-20T10:20:51.267588+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:30:44.465836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.039062+0000","last_change":"2024-04-20T10:21:06.913314+0000","last_active":"2024-04-20T10:21:08.039062+0000","last_peered":"2024-04-20T10:21:08.039062+0000","last_clean":"2024-04-20T10:21:08.039062+0000","last_became_active":"2024-04-20T10:21:06.912896+0000","last_became_peered":"2024-04-20T10:21:06.912896+0000","last_unstale":"2024-04-20T10:21:08.039062+0000","last_undegraded":"2024-04-20T10:21:08.039062+0000","last_fullsized":"2024-04-20T10:21:08.039062+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:16:08.900837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.391138+0000","last_change":"2024-04-20T10:21:02.643414+0000","last_active":"2024-04-20T10:21:08.391138+0000","last_peered":"2024-04-20T10:21:08.391138+0000","last_clean":"2024-04-20T10:21:08.391138+0000","last_became_active":"2024-04-20T10:21:02.641460+0000","last_became_peered":"2024-04-20T10:21:02.641460+0000","last_unstale":"2024-04-20T10:21:08.391138+0000","last_undegraded":"2024-04-20T10:21:08.391138+0000","last_fullsized":"2024-04-20T10:21:08.391138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:24:51.806454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.882750+0000","last_change":"2024-04-20T10:21:01.612256+0000","last_active":"2024-04-20T10:21:07.882750+0000","last_peered":"2024-04-20T10:21:07.882750+0000","last_clean":"2024-04-20T10:21:07.882750+0000","last_became_active":"2024-04-20T10:21:01.611856+0000","last_became_peered":"2024-04-20T10:21:01.611856+0000","last_unstale":"2024-04-20T10:21:07.882750+0000","last_undegraded":"2024-04-20T10:21:07.882750+0000","last_fullsized":"2024-04-20T10:21:07.882750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:59:37.356548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.038723+0000","last_change":"2024-04-20T10:21:06.913797+0000","last_active":"2024-04-20T10:21:08.038723+0000","last_peered":"2024-04-20T10:21:08.038723+0000","last_clean":"2024-04-20T10:21:08.038723+0000","last_became_active":"2024-04-20T10:21:06.912928+0000","last_became_peered":"2024-04-20T10:21:06.912928+0000","last_unstale":"2024-04-20T10:21:08.038723+0000","last_undegraded":"2024-04-20T10:21:08.038723+0000","last_fullsized":"2024-04-20T10:21:08.038723+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:21:29.812831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542069+0000","last_change":"2024-04-20T10:21:02.630629+0000","last_active":"2024-04-20T10:21:08.542069+0000","last_peered":"2024-04-20T10:21:08.542069+0000","last_clean":"2024-04-20T10:21:08.542069+0000","last_became_active":"2024-04-20T10:21:02.623112+0000","last_became_peered":"2024-04-20T10:21:02.623112+0000","last_unstale":"2024-04-20T10:21:08.542069+0000","last_undegraded":"2024-04-20T10:21:08.542069+0000","last_fullsized":"2024-04-20T10:21:08.542069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:37:12.899052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441939+0000","last_change":"2024-04-20T10:21:01.598214+0000","last_active":"2024-04-20T10:21:08.441939+0000","last_peered":"2024-04-20T10:21:08.441939+0000","last_clean":"2024-04-20T10:21:08.441939+0000","last_became_active":"2024-04-20T10:21:01.597260+0000","last_became_peered":"2024-04-20T10:21:01.597260+0000","last_unstale":"2024-04-20T10:21:08.441939+0000","last_undegraded":"2024-04-20T10:21:08.441939+0000","last_fullsized":"2024-04-20T10:21:08.441939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:25:59.939855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.543790+0000","last_change":"2024-04-20T10:21:06.894202+0000","last_active":"2024-04-20T10:21:08.543790+0000","last_peered":"2024-04-20T10:21:08.543790+0000","last_clean":"2024-04-20T10:21:08.543790+0000","last_became_active":"2024-04-20T10:21:06.893620+0000","last_became_peered":"2024-04-20T10:21:06.893620+0000","last_unstale":"2024-04-20T10:21:08.543790+0000","last_undegraded":"2024-04-20T10:21:08.543790+0000","last_fullsized":"2024-04-20T10:21:08.543790+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:31:30.034892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.391863+0000","last_change":"2024-04-20T10:21:02.655710+0000","last_active":"2024-04-20T10:21:08.391863+0000","last_peered":"2024-04-20T10:21:08.391863+0000","last_clean":"2024-04-20T10:21:08.391863+0000","last_became_active":"2024-04-20T10:21:02.655531+0000","last_became_peered":"2024-04-20T10:21:02.655531+0000","last_unstale":"2024-04-20T10:21:08.391863+0000","last_undegraded":"2024-04-20T10:21:08.391863+0000","last_fullsized":"2024-04-20T10:21:08.391863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:27:00.187303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.883115+0000","last_change":"2024-04-20T10:21:01.612100+0000","last_active":"2024-04-20T10:21:07.883115+0000","last_peered":"2024-04-20T10:21:07.883115+0000","last_clean":"2024-04-20T10:21:07.883115+0000","last_became_active":"2024-04-20T10:21:01.610465+0000","last_became_peered":"2024-04-20T10:21:01.610465+0000","last_unstale":"2024-04-20T10:21:07.883115+0000","last_undegraded":"2024-04-20T10:21:07.883115+0000","last_fullsized":"2024-04-20T10:21:07.883115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:01:19.566351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540603+0000","last_change":"2024-04-20T10:21:06.908159+0000","last_active":"2024-04-20T10:21:08.540603+0000","last_peered":"2024-04-20T10:21:08.540603+0000","last_clean":"2024-04-20T10:21:08.540603+0000","last_became_active":"2024-04-20T10:21:06.907687+0000","last_became_peered":"2024-04-20T10:21:06.907687+0000","last_unstale":"2024-04-20T10:21:08.540603+0000","last_undegraded":"2024-04-20T10:21:08.540603+0000","last_fullsized":"2024-04-20T10:21:08.540603+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:17:43.493429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.391503+0000","last_change":"2024-04-20T10:21:02.652617+0000","last_active":"2024-04-20T10:21:08.391503+0000","last_peered":"2024-04-20T10:21:08.391503+0000","last_clean":"2024-04-20T10:21:08.391503+0000","last_became_active":"2024-04-20T10:21:02.652321+0000","last_became_peered":"2024-04-20T10:21:02.652321+0000","last_unstale":"2024-04-20T10:21:08.391503+0000","last_undegraded":"2024-04-20T10:21:08.391503+0000","last_fullsized":"2024-04-20T10:21:08.391503+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:59:24.572150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.887281+0000","last_change":"2024-04-20T10:21:01.608355+0000","last_active":"2024-04-20T10:21:07.887281+0000","last_peered":"2024-04-20T10:21:07.887281+0000","last_clean":"2024-04-20T10:21:07.887281+0000","last_became_active":"2024-04-20T10:21:01.607981+0000","last_became_peered":"2024-04-20T10:21:01.607981+0000","last_unstale":"2024-04-20T10:21:07.887281+0000","last_undegraded":"2024-04-20T10:21:07.887281+0000","last_fullsized":"2024-04-20T10:21:07.887281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:10:57.859781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.885574+0000","last_change":"2024-04-20T10:21:06.900202+0000","last_active":"2024-04-20T10:21:07.885574+0000","last_peered":"2024-04-20T10:21:07.885574+0000","last_clean":"2024-04-20T10:21:07.885574+0000","last_became_active":"2024-04-20T10:21:06.899795+0000","last_became_peered":"2024-04-20T10:21:06.899795+0000","last_unstale":"2024-04-20T10:21:07.885574+0000","last_undegraded":"2024-04-20T10:21:07.885574+0000","last_fullsized":"2024-04-20T10:21:07.885574+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:01:59.638230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.390393+0000","last_change":"2024-04-20T10:21:02.646213+0000","last_active":"2024-04-20T10:21:08.390393+0000","last_peered":"2024-04-20T10:21:08.390393+0000","last_clean":"2024-04-20T10:21:08.390393+0000","last_became_active":"2024-04-20T10:21:02.645881+0000","last_became_peered":"2024-04-20T10:21:02.645881+0000","last_unstale":"2024-04-20T10:21:08.390393+0000","last_undegraded":"2024-04-20T10:21:08.390393+0000","last_fullsized":"2024-04-20T10:21:08.390393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:21:25.537803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.881093+0000","last_change":"2024-04-20T10:21:01.601030+0000","last_active":"2024-04-20T10:21:07.881093+0000","last_peered":"2024-04-20T10:21:07.881093+0000","last_clean":"2024-04-20T10:21:07.881093+0000","last_became_active":"2024-04-20T10:21:01.600702+0000","last_became_peered":"2024-04-20T10:21:01.600702+0000","last_unstale":"2024-04-20T10:21:07.881093+0000","last_undegraded":"2024-04-20T10:21:07.881093+0000","last_fullsized":"2024-04-20T10:21:07.881093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:36:15.966290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.543987+0000","last_change":"2024-04-20T10:21:06.904141+0000","last_active":"2024-04-20T10:21:08.543987+0000","last_peered":"2024-04-20T10:21:08.543987+0000","last_clean":"2024-04-20T10:21:08.543987+0000","last_became_active":"2024-04-20T10:21:06.903212+0000","last_became_peered":"2024-04-20T10:21:06.903212+0000","last_unstale":"2024-04-20T10:21:08.543987+0000","last_undegraded":"2024-04-20T10:21:08.543987+0000","last_fullsized":"2024-04-20T10:21:08.543987+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:59:28.847178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.038508+0000","last_change":"2024-04-20T10:21:02.644636+0000","last_active":"2024-04-20T10:21:08.038508+0000","last_peered":"2024-04-20T10:21:08.038508+0000","last_clean":"2024-04-20T10:21:08.038508+0000","last_became_active":"2024-04-20T10:21:02.644320+0000","last_became_peered":"2024-04-20T10:21:02.644320+0000","last_unstale":"2024-04-20T10:21:08.038508+0000","last_undegraded":"2024-04-20T10:21:08.038508+0000","last_fullsized":"2024-04-20T10:21:08.038508+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:06:59.919488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.038609+0000","last_change":"2024-04-20T10:21:01.616010+0000","last_active":"2024-04-20T10:21:08.038609+0000","last_peered":"2024-04-20T10:21:08.038609+0000","last_clean":"2024-04-20T10:21:08.038609+0000","last_became_active":"2024-04-20T10:21:01.615701+0000","last_became_peered":"2024-04-20T10:21:01.615701+0000","last_unstale":"2024-04-20T10:21:08.038609+0000","last_undegraded":"2024-04-20T10:21:08.038609+0000","last_fullsized":"2024-04-20T10:21:08.038609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:32:02.981747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542308+0000","last_change":"2024-04-20T10:21:06.909393+0000","last_active":"2024-04-20T10:21:08.542308+0000","last_peered":"2024-04-20T10:21:08.542308+0000","last_clean":"2024-04-20T10:21:08.542308+0000","last_became_active":"2024-04-20T10:21:06.907616+0000","last_became_peered":"2024-04-20T10:21:06.907616+0000","last_unstale":"2024-04-20T10:21:08.542308+0000","last_undegraded":"2024-04-20T10:21:08.542308+0000","last_fullsized":"2024-04-20T10:21:08.542308+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:07:32.815495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.390789+0000","last_change":"2024-04-20T10:21:02.654823+0000","last_active":"2024-04-20T10:21:08.390789+0000","last_peered":"2024-04-20T10:21:08.390789+0000","last_clean":"2024-04-20T10:21:08.390789+0000","last_became_active":"2024-04-20T10:21:02.654556+0000","last_became_peered":"2024-04-20T10:21:02.654556+0000","last_unstale":"2024-04-20T10:21:08.390789+0000","last_undegraded":"2024-04-20T10:21:08.390789+0000","last_fullsized":"2024-04-20T10:21:08.390789+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:52:24.023900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541418+0000","last_change":"2024-04-20T10:21:01.618588+0000","last_active":"2024-04-20T10:21:08.541418+0000","last_peered":"2024-04-20T10:21:08.541418+0000","last_clean":"2024-04-20T10:21:08.541418+0000","last_became_active":"2024-04-20T10:21:01.618294+0000","last_became_peered":"2024-04-20T10:21:01.618294+0000","last_unstale":"2024-04-20T10:21:08.541418+0000","last_undegraded":"2024-04-20T10:21:08.541418+0000","last_fullsized":"2024-04-20T10:21:08.541418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:34:27.492302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542602+0000","last_change":"2024-04-20T10:21:06.897913+0000","last_active":"2024-04-20T10:21:08.542602+0000","last_peered":"2024-04-20T10:21:08.542602+0000","last_clean":"2024-04-20T10:21:08.542602+0000","last_became_active":"2024-04-20T10:21:06.897536+0000","last_became_peered":"2024-04-20T10:21:06.897536+0000","last_unstale":"2024-04-20T10:21:08.542602+0000","last_undegraded":"2024-04-20T10:21:08.542602+0000","last_fullsized":"2024-04-20T10:21:08.542602+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:27:49.300931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.543513+0000","last_change":"2024-04-20T10:21:02.644302+0000","last_active":"2024-04-20T10:21:08.543513+0000","last_peered":"2024-04-20T10:21:08.543513+0000","last_clean":"2024-04-20T10:21:08.543513+0000","last_became_active":"2024-04-20T10:21:02.643236+0000","last_became_peered":"2024-04-20T10:21:02.643236+0000","last_unstale":"2024-04-20T10:21:08.543513+0000","last_undegraded":"2024-04-20T10:21:08.543513+0000","last_fullsized":"2024-04-20T10:21:08.543513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:06:59.919488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.544270+0000","last_change":"2024-04-20T10:21:01.604692+0000","last_active":"2024-04-20T10:21:08.544270+0000","last_peered":"2024-04-20T10:21:08.544270+0000","last_clean":"2024-04-20T10:21:08.544270+0000","last_became_active":"2024-04-20T10:21:01.604345+0000","last_became_peered":"2024-04-20T10:21:01.604345+0000","last_unstale":"2024-04-20T10:21:08.544270+0000","last_undegraded":"2024-04-20T10:21:08.544270+0000","last_fullsized":"2024-04-20T10:21:08.544270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:29:47.733236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542165+0000","last_change":"2024-04-20T10:21:06.901480+0000","last_active":"2024-04-20T10:21:08.542165+0000","last_peered":"2024-04-20T10:21:08.542165+0000","last_clean":"2024-04-20T10:21:08.542165+0000","last_became_active":"2024-04-20T10:21:06.900820+0000","last_became_peered":"2024-04-20T10:21:06.900820+0000","last_unstale":"2024-04-20T10:21:08.542165+0000","last_undegraded":"2024-04-20T10:21:08.542165+0000","last_fullsized":"2024-04-20T10:21:08.542165+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:15:30.919284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.391163+0000","last_change":"2024-04-20T10:21:02.642723+0000","last_active":"2024-04-20T10:21:08.391163+0000","last_peered":"2024-04-20T10:21:08.391163+0000","last_clean":"2024-04-20T10:21:08.391163+0000","last_became_active":"2024-04-20T10:21:02.641828+0000","last_became_peered":"2024-04-20T10:21:02.641828+0000","last_unstale":"2024-04-20T10:21:08.391163+0000","last_undegraded":"2024-04-20T10:21:08.391163+0000","last_fullsized":"2024-04-20T10:21:08.391163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:31:25.759865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:21.609415+0000","last_change":"2024-04-20T10:21:01.619387+0000","last_active":"2024-04-20T10:21:21.609415+0000","last_peered":"2024-04-20T10:21:21.609415+0000","last_clean":"2024-04-20T10:21:21.609415+0000","last_became_active":"2024-04-20T10:21:01.619119+0000","last_became_peered":"2024-04-20T10:21:01.619119+0000","last_unstale":"2024-04-20T10:21:21.609415+0000","last_undegraded":"2024-04-20T10:21:21.609415+0000","last_fullsized":"2024-04-20T10:21:21.609415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:55:41.208744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441019+0000","last_change":"2024-04-20T10:21:06.911043+0000","last_active":"2024-04-20T10:21:08.441019+0000","last_peered":"2024-04-20T10:21:08.441019+0000","last_clean":"2024-04-20T10:21:08.441019+0000","last_became_active":"2024-04-20T10:21:06.909896+0000","last_became_peered":"2024-04-20T10:21:06.909896+0000","last_unstale":"2024-04-20T10:21:08.441019+0000","last_undegraded":"2024-04-20T10:21:08.441019+0000","last_fullsized":"2024-04-20T10:21:08.441019+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:24:56.081482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.887153+0000","last_change":"2024-04-20T10:21:02.624575+0000","last_active":"2024-04-20T10:21:07.887153+0000","last_peered":"2024-04-20T10:21:07.887153+0000","last_clean":"2024-04-20T10:21:07.887153+0000","last_became_active":"2024-04-20T10:21:02.624230+0000","last_became_peered":"2024-04-20T10:21:02.624230+0000","last_unstale":"2024-04-20T10:21:07.887153+0000","last_undegraded":"2024-04-20T10:21:07.887153+0000","last_fullsized":"2024-04-20T10:21:07.887153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:07:51.297896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441862+0000","last_change":"2024-04-20T10:21:01.619231+0000","last_active":"2024-04-20T10:21:08.441862+0000","last_peered":"2024-04-20T10:21:08.441862+0000","last_clean":"2024-04-20T10:21:08.441862+0000","last_became_active":"2024-04-20T10:21:01.618872+0000","last_became_peered":"2024-04-20T10:21:01.618872+0000","last_unstale":"2024-04-20T10:21:08.441862+0000","last_undegraded":"2024-04-20T10:21:08.441862+0000","last_fullsized":"2024-04-20T10:21:08.441862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:38:10.686751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.886989+0000","last_change":"2024-04-20T10:21:06.908460+0000","last_active":"2024-04-20T10:21:07.886989+0000","last_peered":"2024-04-20T10:21:07.886989+0000","last_clean":"2024-04-20T10:21:07.886989+0000","last_became_active":"2024-04-20T10:21:06.907722+0000","last_became_peered":"2024-04-20T10:21:06.907722+0000","last_unstale":"2024-04-20T10:21:07.886989+0000","last_undegraded":"2024-04-20T10:21:07.886989+0000","last_fullsized":"2024-04-20T10:21:07.886989+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:06:15.702589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.544085+0000","last_change":"2024-04-20T10:21:02.646092+0000","last_active":"2024-04-20T10:21:08.544085+0000","last_peered":"2024-04-20T10:21:08.544085+0000","last_clean":"2024-04-20T10:21:08.544085+0000","last_became_active":"2024-04-20T10:21:02.645788+0000","last_became_peered":"2024-04-20T10:21:02.645788+0000","last_unstale":"2024-04-20T10:21:08.544085+0000","last_undegraded":"2024-04-20T10:21:08.544085+0000","last_fullsized":"2024-04-20T10:21:08.544085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:22:36.605724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542309+0000","last_change":"2024-04-20T10:21:01.623128+0000","last_active":"2024-04-20T10:21:08.542309+0000","last_peered":"2024-04-20T10:21:08.542309+0000","last_clean":"2024-04-20T10:21:08.542309+0000","last_became_active":"2024-04-20T10:21:01.622867+0000","last_became_peered":"2024-04-20T10:21:01.622867+0000","last_unstale":"2024-04-20T10:21:08.542309+0000","last_undegraded":"2024-04-20T10:21:08.542309+0000","last_fullsized":"2024-04-20T10:21:08.542309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:50:23.957089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541079+0000","last_change":"2024-04-20T10:21:06.915147+0000","last_active":"2024-04-20T10:21:08.541079+0000","last_peered":"2024-04-20T10:21:08.541079+0000","last_clean":"2024-04-20T10:21:08.541079+0000","last_became_active":"2024-04-20T10:21:06.913832+0000","last_became_peered":"2024-04-20T10:21:06.913832+0000","last_unstale":"2024-04-20T10:21:08.541079+0000","last_undegraded":"2024-04-20T10:21:08.541079+0000","last_fullsized":"2024-04-20T10:21:08.541079+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:10:39.166791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542260+0000","last_change":"2024-04-20T10:21:02.638354+0000","last_active":"2024-04-20T10:21:08.542260+0000","last_peered":"2024-04-20T10:21:08.542260+0000","last_clean":"2024-04-20T10:21:08.542260+0000","last_became_active":"2024-04-20T10:21:02.638065+0000","last_became_peered":"2024-04-20T10:21:02.638065+0000","last_unstale":"2024-04-20T10:21:08.542260+0000","last_undegraded":"2024-04-20T10:21:08.542260+0000","last_fullsized":"2024-04-20T10:21:08.542260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:12:12.144216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.039425+0000","last_change":"2024-04-20T10:21:01.621766+0000","last_active":"2024-04-20T10:21:08.039425+0000","last_peered":"2024-04-20T10:21:08.039425+0000","last_clean":"2024-04-20T10:21:08.039425+0000","last_became_active":"2024-04-20T10:21:01.621546+0000","last_became_peered":"2024-04-20T10:21:01.621546+0000","last_unstale":"2024-04-20T10:21:08.039425+0000","last_undegraded":"2024-04-20T10:21:08.039425+0000","last_fullsized":"2024-04-20T10:21:08.039425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:01:19.566351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540739+0000","last_change":"2024-04-20T10:21:06.910220+0000","last_active":"2024-04-20T10:21:08.540739+0000","last_peered":"2024-04-20T10:21:08.540739+0000","last_clean":"2024-04-20T10:21:08.540739+0000","last_became_active":"2024-04-20T10:21:06.909876+0000","last_became_peered":"2024-04-20T10:21:06.909876+0000","last_unstale":"2024-04-20T10:21:08.540739+0000","last_undegraded":"2024-04-20T10:21:08.540739+0000","last_fullsized":"2024-04-20T10:21:08.540739+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:27:04.462330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.543797+0000","last_change":"2024-04-20T10:21:06.903823+0000","last_active":"2024-04-20T10:21:08.543797+0000","last_peered":"2024-04-20T10:21:08.543797+0000","last_clean":"2024-04-20T10:21:08.543797+0000","last_became_active":"2024-04-20T10:21:06.903100+0000","last_became_peered":"2024-04-20T10:21:06.903100+0000","last_unstale":"2024-04-20T10:21:08.543797+0000","last_undegraded":"2024-04-20T10:21:08.543797+0000","last_fullsized":"2024-04-20T10:21:08.543797+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:24:56.081482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.440526+0000","last_change":"2024-04-20T10:21:02.636194+0000","last_active":"2024-04-20T10:21:08.440526+0000","last_peered":"2024-04-20T10:21:08.440526+0000","last_clean":"2024-04-20T10:21:08.440526+0000","last_became_active":"2024-04-20T10:21:02.635907+0000","last_became_peered":"2024-04-20T10:21:02.635907+0000","last_unstale":"2024-04-20T10:21:08.440526+0000","last_undegraded":"2024-04-20T10:21:08.440526+0000","last_fullsized":"2024-04-20T10:21:08.440526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:06:59.919488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540844+0000","last_change":"2024-04-20T10:21:01.617415+0000","last_active":"2024-04-20T10:21:08.540844+0000","last_peered":"2024-04-20T10:21:08.540844+0000","last_clean":"2024-04-20T10:21:08.540844+0000","last_became_active":"2024-04-20T10:21:01.617082+0000","last_became_peered":"2024-04-20T10:21:01.617082+0000","last_unstale":"2024-04-20T10:21:08.540844+0000","last_undegraded":"2024-04-20T10:21:08.540844+0000","last_fullsized":"2024-04-20T10:21:08.540844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:56:45.348182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.039217+0000","last_change":"2024-04-20T10:21:06.915304+0000","last_active":"2024-04-20T10:21:08.039217+0000","last_peered":"2024-04-20T10:21:08.039217+0000","last_clean":"2024-04-20T10:21:08.039217+0000","last_became_active":"2024-04-20T10:21:06.914915+0000","last_became_peered":"2024-04-20T10:21:06.914915+0000","last_unstale":"2024-04-20T10:21:08.039217+0000","last_undegraded":"2024-04-20T10:21:08.039217+0000","last_fullsized":"2024-04-20T10:21:08.039217+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:52:28.298927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540918+0000","last_change":"2024-04-20T10:21:02.646474+0000","last_active":"2024-04-20T10:21:08.540918+0000","last_peered":"2024-04-20T10:21:08.540918+0000","last_clean":"2024-04-20T10:21:08.540918+0000","last_became_active":"2024-04-20T10:21:02.646219+0000","last_became_peered":"2024-04-20T10:21:02.646219+0000","last_unstale":"2024-04-20T10:21:08.540918+0000","last_undegraded":"2024-04-20T10:21:08.540918+0000","last_fullsized":"2024-04-20T10:21:08.540918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:06:59.919488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541015+0000","last_change":"2024-04-20T10:21:01.609406+0000","last_active":"2024-04-20T10:21:08.541015+0000","last_peered":"2024-04-20T10:21:08.541015+0000","last_clean":"2024-04-20T10:21:08.541015+0000","last_became_active":"2024-04-20T10:21:01.609064+0000","last_became_peered":"2024-04-20T10:21:01.609064+0000","last_unstale":"2024-04-20T10:21:08.541015+0000","last_undegraded":"2024-04-20T10:21:08.541015+0000","last_fullsized":"2024-04-20T10:21:08.541015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:28:01.100842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.039237+0000","last_change":"2024-04-20T10:21:01.615891+0000","last_active":"2024-04-20T10:21:08.039237+0000","last_peered":"2024-04-20T10:21:08.039237+0000","last_clean":"2024-04-20T10:21:08.039237+0000","last_became_active":"2024-04-20T10:21:01.615584+0000","last_became_peered":"2024-04-20T10:21:01.615584+0000","last_unstale":"2024-04-20T10:21:08.039237+0000","last_undegraded":"2024-04-20T10:21:08.039237+0000","last_fullsized":"2024-04-20T10:21:08.039237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:59:37.356548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.544232+0000","last_change":"2024-04-20T10:21:02.643605+0000","last_active":"2024-04-20T10:21:08.544232+0000","last_peered":"2024-04-20T10:21:08.544232+0000","last_clean":"2024-04-20T10:21:08.544232+0000","last_became_active":"2024-04-20T10:21:02.643327+0000","last_became_peered":"2024-04-20T10:21:02.643327+0000","last_unstale":"2024-04-20T10:21:08.544232+0000","last_undegraded":"2024-04-20T10:21:08.544232+0000","last_fullsized":"2024-04-20T10:21:08.544232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:01:55.363202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.882164+0000","last_change":"2024-04-20T10:21:06.892955+0000","last_active":"2024-04-20T10:21:07.882164+0000","last_peered":"2024-04-20T10:21:07.882164+0000","last_clean":"2024-04-20T10:21:07.882164+0000","last_became_active":"2024-04-20T10:21:06.892534+0000","last_became_peered":"2024-04-20T10:21:06.892534+0000","last_unstale":"2024-04-20T10:21:07.882164+0000","last_undegraded":"2024-04-20T10:21:07.882164+0000","last_fullsized":"2024-04-20T10:21:07.882164+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:17:28.840720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541245+0000","last_change":"2024-04-20T10:21:01.618877+0000","last_active":"2024-04-20T10:21:08.541245+0000","last_peered":"2024-04-20T10:21:08.541245+0000","last_clean":"2024-04-20T10:21:08.541245+0000","last_became_active":"2024-04-20T10:21:01.618549+0000","last_became_peered":"2024-04-20T10:21:01.618549+0000","last_unstale":"2024-04-20T10:21:08.541245+0000","last_undegraded":"2024-04-20T10:21:08.541245+0000","last_fullsized":"2024-04-20T10:21:08.541245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:36:15.966290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.886244+0000","last_change":"2024-04-20T10:21:02.645042+0000","last_active":"2024-04-20T10:21:07.886244+0000","last_peered":"2024-04-20T10:21:07.886244+0000","last_clean":"2024-04-20T10:21:07.886244+0000","last_became_active":"2024-04-20T10:21:02.644630+0000","last_became_peered":"2024-04-20T10:21:02.644630+0000","last_unstale":"2024-04-20T10:21:07.886244+0000","last_undegraded":"2024-04-20T10:21:07.886244+0000","last_fullsized":"2024-04-20T10:21:07.886244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:59:38.370072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542570+0000","last_change":"2024-04-20T10:21:06.888033+0000","last_active":"2024-04-20T10:21:08.542570+0000","last_peered":"2024-04-20T10:21:08.542570+0000","last_clean":"2024-04-20T10:21:08.542570+0000","last_became_active":"2024-04-20T10:21:06.887672+0000","last_became_peered":"2024-04-20T10:21:06.887672+0000","last_unstale":"2024-04-20T10:21:08.542570+0000","last_undegraded":"2024-04-20T10:21:08.542570+0000","last_fullsized":"2024-04-20T10:21:08.542570+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:13:54.194111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.038167+0000","last_change":"2024-04-20T10:21:02.644672+0000","last_active":"2024-04-20T10:21:08.038167+0000","last_peered":"2024-04-20T10:21:08.038167+0000","last_clean":"2024-04-20T10:21:08.038167+0000","last_became_active":"2024-04-20T10:21:02.644327+0000","last_became_peered":"2024-04-20T10:21:02.644327+0000","last_unstale":"2024-04-20T10:21:08.038167+0000","last_undegraded":"2024-04-20T10:21:08.038167+0000","last_fullsized":"2024-04-20T10:21:08.038167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:28:54.408773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.881607+0000","last_change":"2024-04-20T10:21:02.624910+0000","last_active":"2024-04-20T10:21:07.881607+0000","last_peered":"2024-04-20T10:21:07.881607+0000","last_clean":"2024-04-20T10:21:07.881607+0000","last_became_active":"2024-04-20T10:21:02.624555+0000","last_became_peered":"2024-04-20T10:21:02.624555+0000","last_unstale":"2024-04-20T10:21:07.881607+0000","last_undegraded":"2024-04-20T10:21:07.881607+0000","last_fullsized":"2024-04-20T10:21:07.881607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:07:51.297896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541176+0000","last_change":"2024-04-20T10:21:02.636958+0000","last_active":"2024-04-20T10:21:08.541176+0000","last_peered":"2024-04-20T10:21:08.541176+0000","last_clean":"2024-04-20T10:21:08.541176+0000","last_became_active":"2024-04-20T10:21:02.636594+0000","last_became_peered":"2024-04-20T10:21:02.636594+0000","last_unstale":"2024-04-20T10:21:08.541176+0000","last_undegraded":"2024-04-20T10:21:08.541176+0000","last_fullsized":"2024-04-20T10:21:08.541176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:12:07.469367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.543932+0000","last_change":"2024-04-20T10:21:02.649761+0000","last_active":"2024-04-20T10:21:08.543932+0000","last_peered":"2024-04-20T10:21:08.543932+0000","last_clean":"2024-04-20T10:21:08.543932+0000","last_became_active":"2024-04-20T10:21:02.649471+0000","last_became_peered":"2024-04-20T10:21:02.649471+0000","last_unstale":"2024-04-20T10:21:08.543932+0000","last_undegraded":"2024-04-20T10:21:08.543932+0000","last_fullsized":"2024-04-20T10:21:08.543932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:14:16.934710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.543183+0000","last_change":"2024-04-20T10:21:02.644223+0000","last_active":"2024-04-20T10:21:08.543183+0000","last_peered":"2024-04-20T10:21:08.543183+0000","last_clean":"2024-04-20T10:21:08.543183+0000","last_became_active":"2024-04-20T10:21:02.643893+0000","last_became_peered":"2024-04-20T10:21:02.643893+0000","last_unstale":"2024-04-20T10:21:08.543183+0000","last_undegraded":"2024-04-20T10:21:08.543183+0000","last_fullsized":"2024-04-20T10:21:08.543183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:09:41.966188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441369+0000","last_change":"2024-04-20T10:21:02.636144+0000","last_active":"2024-04-20T10:21:08.441369+0000","last_peered":"2024-04-20T10:21:08.441369+0000","last_clean":"2024-04-20T10:21:08.441369+0000","last_became_active":"2024-04-20T10:21:02.635834+0000","last_became_peered":"2024-04-20T10:21:02.635834+0000","last_unstale":"2024-04-20T10:21:08.441369+0000","last_undegraded":"2024-04-20T10:21:08.441369+0000","last_fullsized":"2024-04-20T10:21:08.441369+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:10:15.206686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.039750+0000","last_change":"2024-04-20T10:21:02.643146+0000","last_active":"2024-04-20T10:21:08.039750+0000","last_peered":"2024-04-20T10:21:08.039750+0000","last_clean":"2024-04-20T10:21:08.039750+0000","last_became_active":"2024-04-20T10:21:02.639521+0000","last_became_peered":"2024-04-20T10:21:02.639521+0000","last_unstale":"2024-04-20T10:21:08.039750+0000","last_undegraded":"2024-04-20T10:21:08.039750+0000","last_fullsized":"2024-04-20T10:21:08.039750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:01:55.363202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542473+0000","last_change":"2024-04-20T10:21:02.631553+0000","last_active":"2024-04-20T10:21:08.542473+0000","last_peered":"2024-04-20T10:21:08.542473+0000","last_clean":"2024-04-20T10:21:08.542473+0000","last_became_active":"2024-04-20T10:21:02.626210+0000","last_became_peered":"2024-04-20T10:21:02.626210+0000","last_unstale":"2024-04-20T10:21:08.542473+0000","last_undegraded":"2024-04-20T10:21:08.542473+0000","last_fullsized":"2024-04-20T10:21:08.542473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:02:26.716730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540787+0000","last_change":"2024-04-20T10:21:02.625457+0000","last_active":"2024-04-20T10:21:08.540787+0000","last_peered":"2024-04-20T10:21:08.540787+0000","last_clean":"2024-04-20T10:21:08.540787+0000","last_became_active":"2024-04-20T10:21:02.624987+0000","last_became_peered":"2024-04-20T10:21:02.624987+0000","last_unstale":"2024-04-20T10:21:08.540787+0000","last_undegraded":"2024-04-20T10:21:08.540787+0000","last_fullsized":"2024-04-20T10:21:08.540787+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:13:44.633309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540862+0000","last_change":"2024-04-20T10:21:02.628208+0000","last_active":"2024-04-20T10:21:08.540862+0000","last_peered":"2024-04-20T10:21:08.540862+0000","last_clean":"2024-04-20T10:21:08.540862+0000","last_became_active":"2024-04-20T10:21:02.625255+0000","last_became_peered":"2024-04-20T10:21:02.625255+0000","last_unstale":"2024-04-20T10:21:08.540862+0000","last_undegraded":"2024-04-20T10:21:08.540862+0000","last_fullsized":"2024-04-20T10:21:08.540862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:06:35.349863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541152+0000","last_change":"2024-04-20T10:21:02.629528+0000","last_active":"2024-04-20T10:21:08.541152+0000","last_peered":"2024-04-20T10:21:08.541152+0000","last_clean":"2024-04-20T10:21:08.541152+0000","last_became_active":"2024-04-20T10:21:02.626300+0000","last_became_peered":"2024-04-20T10:21:02.626300+0000","last_unstale":"2024-04-20T10:21:08.541152+0000","last_undegraded":"2024-04-20T10:21:08.541152+0000","last_fullsized":"2024-04-20T10:21:08.541152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:17:15.177678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.543489+0000","last_change":"2024-04-20T10:21:02.657102+0000","last_active":"2024-04-20T10:21:08.543489+0000","last_peered":"2024-04-20T10:21:08.543489+0000","last_clean":"2024-04-20T10:21:08.543489+0000","last_became_active":"2024-04-20T10:21:02.656810+0000","last_became_peered":"2024-04-20T10:21:02.656810+0000","last_unstale":"2024-04-20T10:21:08.543489+0000","last_undegraded":"2024-04-20T10:21:08.543489+0000","last_fullsized":"2024-04-20T10:21:08.543489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:04:04.238410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.039198+0000","last_change":"2024-04-20T10:21:02.639499+0000","last_active":"2024-04-20T10:21:08.039198+0000","last_peered":"2024-04-20T10:21:08.039198+0000","last_clean":"2024-04-20T10:21:08.039198+0000","last_became_active":"2024-04-20T10:21:02.638713+0000","last_became_peered":"2024-04-20T10:21:02.638713+0000","last_unstale":"2024-04-20T10:21:08.039198+0000","last_undegraded":"2024-04-20T10:21:08.039198+0000","last_fullsized":"2024-04-20T10:21:08.039198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:31:53.442042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.391493+0000","last_change":"2024-04-20T10:21:02.639481+0000","last_active":"2024-04-20T10:21:08.391493+0000","last_peered":"2024-04-20T10:21:08.391493+0000","last_clean":"2024-04-20T10:21:08.391493+0000","last_became_active":"2024-04-20T10:21:02.639198+0000","last_became_peered":"2024-04-20T10:21:02.639198+0000","last_unstale":"2024-04-20T10:21:08.391493+0000","last_undegraded":"2024-04-20T10:21:08.391493+0000","last_fullsized":"2024-04-20T10:21:08.391493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:35:54.235610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.883015+0000","last_change":"2024-04-20T10:21:02.640221+0000","last_active":"2024-04-20T10:21:07.883015+0000","last_peered":"2024-04-20T10:21:07.883015+0000","last_clean":"2024-04-20T10:21:07.883015+0000","last_became_active":"2024-04-20T10:21:02.639833+0000","last_became_peered":"2024-04-20T10:21:02.639833+0000","last_unstale":"2024-04-20T10:21:07.883015+0000","last_undegraded":"2024-04-20T10:21:07.883015+0000","last_fullsized":"2024-04-20T10:21:07.883015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:38:11.700275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542925+0000","last_change":"2024-04-20T10:21:02.640233+0000","last_active":"2024-04-20T10:21:08.542925+0000","last_peered":"2024-04-20T10:21:08.542925+0000","last_clean":"2024-04-20T10:21:08.542925+0000","last_became_active":"2024-04-20T10:21:02.639913+0000","last_became_peered":"2024-04-20T10:21:02.639913+0000","last_unstale":"2024-04-20T10:21:08.542925+0000","last_undegraded":"2024-04-20T10:21:08.542925+0000","last_fullsized":"2024-04-20T10:21:08.542925+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:45:33.753385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540514+0000","last_change":"2024-04-20T10:21:02.643714+0000","last_active":"2024-04-20T10:21:08.540514+0000","last_peered":"2024-04-20T10:21:08.540514+0000","last_clean":"2024-04-20T10:21:08.540514+0000","last_became_active":"2024-04-20T10:21:02.643403+0000","last_became_peered":"2024-04-20T10:21:02.643403+0000","last_unstale":"2024-04-20T10:21:08.540514+0000","last_undegraded":"2024-04-20T10:21:08.540514+0000","last_fullsized":"2024-04-20T10:21:08.540514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:27:44.403086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.440671+0000","last_change":"2024-04-20T10:21:02.643476+0000","last_active":"2024-04-20T10:21:08.440671+0000","last_peered":"2024-04-20T10:21:08.440671+0000","last_clean":"2024-04-20T10:21:08.440671+0000","last_became_active":"2024-04-20T10:21:02.643180+0000","last_became_peered":"2024-04-20T10:21:02.643180+0000","last_unstale":"2024-04-20T10:21:08.440671+0000","last_undegraded":"2024-04-20T10:21:08.440671+0000","last_fullsized":"2024-04-20T10:21:08.440671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:31:53.442042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.543269+0000","last_change":"2024-04-20T10:21:02.646081+0000","last_active":"2024-04-20T10:21:08.543269+0000","last_peered":"2024-04-20T10:21:08.543269+0000","last_clean":"2024-04-20T10:21:08.543269+0000","last_became_active":"2024-04-20T10:21:02.645798+0000","last_became_peered":"2024-04-20T10:21:02.645798+0000","last_unstale":"2024-04-20T10:21:08.543269+0000","last_undegraded":"2024-04-20T10:21:08.543269+0000","last_fullsized":"2024-04-20T10:21:08.543269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:45:52.178877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540989+0000","last_change":"2024-04-20T10:21:02.639147+0000","last_active":"2024-04-20T10:21:08.540989+0000","last_peered":"2024-04-20T10:21:08.540989+0000","last_clean":"2024-04-20T10:21:08.540989+0000","last_became_active":"2024-04-20T10:21:02.638861+0000","last_became_peered":"2024-04-20T10:21:02.638861+0000","last_unstale":"2024-04-20T10:21:08.540989+0000","last_undegraded":"2024-04-20T10:21:08.540989+0000","last_fullsized":"2024-04-20T10:21:08.540989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:22:36.605724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.887047+0000","last_change":"2024-04-20T10:21:02.638429+0000","last_active":"2024-04-20T10:21:07.887047+0000","last_peered":"2024-04-20T10:21:07.887047+0000","last_clean":"2024-04-20T10:21:07.887047+0000","last_became_active":"2024-04-20T10:21:02.638131+0000","last_became_peered":"2024-04-20T10:21:02.638131+0000","last_unstale":"2024-04-20T10:21:07.887047+0000","last_undegraded":"2024-04-20T10:21:07.887047+0000","last_fullsized":"2024-04-20T10:21:07.887047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:50:28.730495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.039192+0000","last_change":"2024-04-20T10:21:02.646725+0000","last_active":"2024-04-20T10:21:08.039192+0000","last_peered":"2024-04-20T10:21:08.039192+0000","last_clean":"2024-04-20T10:21:08.039192+0000","last_became_active":"2024-04-20T10:21:02.646439+0000","last_became_peered":"2024-04-20T10:21:02.646439+0000","last_unstale":"2024-04-20T10:21:08.039192+0000","last_undegraded":"2024-04-20T10:21:08.039192+0000","last_fullsized":"2024-04-20T10:21:08.039192+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:06:35.349863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.039464+0000","last_change":"2024-04-20T10:21:02.641401+0000","last_active":"2024-04-20T10:21:08.039464+0000","last_peered":"2024-04-20T10:21:08.039464+0000","last_clean":"2024-04-20T10:21:08.039464+0000","last_became_active":"2024-04-20T10:21:02.640962+0000","last_became_peered":"2024-04-20T10:21:02.640962+0000","last_unstale":"2024-04-20T10:21:08.039464+0000","last_undegraded":"2024-04-20T10:21:08.039464+0000","last_fullsized":"2024-04-20T10:21:08.039464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:22:36.605724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.391828+0000","last_change":"2024-04-20T10:21:02.654762+0000","last_active":"2024-04-20T10:21:08.391828+0000","last_peered":"2024-04-20T10:21:08.391828+0000","last_clean":"2024-04-20T10:21:08.391828+0000","last_became_active":"2024-04-20T10:21:02.654497+0000","last_became_peered":"2024-04-20T10:21:02.654497+0000","last_unstale":"2024-04-20T10:21:08.391828+0000","last_undegraded":"2024-04-20T10:21:08.391828+0000","last_fullsized":"2024-04-20T10:21:08.391828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:46:41.173563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540653+0000","last_change":"2024-04-20T10:21:02.630507+0000","last_active":"2024-04-20T10:21:08.540653+0000","last_peered":"2024-04-20T10:21:08.540653+0000","last_clean":"2024-04-20T10:21:08.540653+0000","last_became_active":"2024-04-20T10:21:02.626482+0000","last_became_peered":"2024-04-20T10:21:02.626482+0000","last_unstale":"2024-04-20T10:21:08.540653+0000","last_undegraded":"2024-04-20T10:21:08.540653+0000","last_fullsized":"2024-04-20T10:21:08.540653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:28:54.408773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.390187+0000","last_change":"2024-04-20T10:21:02.649796+0000","last_active":"2024-04-20T10:21:08.390187+0000","last_peered":"2024-04-20T10:21:08.390187+0000","last_clean":"2024-04-20T10:21:08.390187+0000","last_became_active":"2024-04-20T10:21:02.649541+0000","last_became_peered":"2024-04-20T10:21:02.649541+0000","last_unstale":"2024-04-20T10:21:08.390187+0000","last_undegraded":"2024-04-20T10:21:08.390187+0000","last_fullsized":"2024-04-20T10:21:08.390187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:02:26.716730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540951+0000","last_change":"2024-04-20T10:21:02.653552+0000","last_active":"2024-04-20T10:21:08.540951+0000","last_peered":"2024-04-20T10:21:08.540951+0000","last_clean":"2024-04-20T10:21:08.540951+0000","last_became_active":"2024-04-20T10:21:02.653288+0000","last_became_peered":"2024-04-20T10:21:02.653288+0000","last_unstale":"2024-04-20T10:21:08.540951+0000","last_undegraded":"2024-04-20T10:21:08.540951+0000","last_fullsized":"2024-04-20T10:21:08.540951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:49:26.986908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.390549+0000","last_change":"2024-04-20T10:21:02.633119+0000","last_active":"2024-04-20T10:21:08.390549+0000","last_peered":"2024-04-20T10:21:08.390549+0000","last_clean":"2024-04-20T10:21:08.390549+0000","last_became_active":"2024-04-20T10:21:02.632816+0000","last_became_peered":"2024-04-20T10:21:02.632816+0000","last_unstale":"2024-04-20T10:21:08.390549+0000","last_undegraded":"2024-04-20T10:21:08.390549+0000","last_fullsized":"2024-04-20T10:21:08.390549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:16:04.625809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541721+0000","last_change":"2024-04-20T10:21:02.638226+0000","last_active":"2024-04-20T10:21:08.541721+0000","last_peered":"2024-04-20T10:21:08.541721+0000","last_clean":"2024-04-20T10:21:08.541721+0000","last_became_active":"2024-04-20T10:21:02.637640+0000","last_became_peered":"2024-04-20T10:21:02.637640+0000","last_unstale":"2024-04-20T10:21:08.541721+0000","last_undegraded":"2024-04-20T10:21:08.541721+0000","last_fullsized":"2024-04-20T10:21:08.541721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:44:55.208482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.544254+0000","last_change":"2024-04-20T10:21:02.641981+0000","last_active":"2024-04-20T10:21:08.544254+0000","last_peered":"2024-04-20T10:21:08.544254+0000","last_clean":"2024-04-20T10:21:08.544254+0000","last_became_active":"2024-04-20T10:21:02.640073+0000","last_became_peered":"2024-04-20T10:21:02.640073+0000","last_unstale":"2024-04-20T10:21:08.544254+0000","last_undegraded":"2024-04-20T10:21:08.544254+0000","last_fullsized":"2024-04-20T10:21:08.544254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:28:54.408773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441547+0000","last_change":"2024-04-20T10:21:02.646636+0000","last_active":"2024-04-20T10:21:08.441547+0000","last_peered":"2024-04-20T10:21:08.441547+0000","last_clean":"2024-04-20T10:21:08.441547+0000","last_became_active":"2024-04-20T10:21:02.646359+0000","last_became_peered":"2024-04-20T10:21:02.646359+0000","last_unstale":"2024-04-20T10:21:08.441547+0000","last_undegraded":"2024-04-20T10:21:08.441547+0000","last_fullsized":"2024-04-20T10:21:08.441547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:22:36.605724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.391583+0000","last_change":"2024-04-20T10:21:02.652784+0000","last_active":"2024-04-20T10:21:08.391583+0000","last_peered":"2024-04-20T10:21:08.391583+0000","last_clean":"2024-04-20T10:21:08.391583+0000","last_became_active":"2024-04-20T10:21:02.652454+0000","last_became_peered":"2024-04-20T10:21:02.652454+0000","last_unstale":"2024-04-20T10:21:08.391583+0000","last_undegraded":"2024-04-20T10:21:08.391583+0000","last_fullsized":"2024-04-20T10:21:08.391583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:38:01.376108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542153,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":694509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542154,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":107401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542153,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":103649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542153,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":103685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542153,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":107401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542153,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":103559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542155,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":103719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542155,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":868352,"data_stored":694565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T10:21:32.208 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-20T10:21:32.209 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-20T10:21:32.209 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-20T10:21:32.209 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-20T10:21:32.209 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-20T10:21:32.453 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.453+0000 7f356aa00640 1 Processor -- start 2024-04-20T10:21:32.453 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.454+0000 7f356aa00640 1 -- start start 2024-04-20T10:21:32.455 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.455+0000 7f356aa00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f356406cde0 0x7f35640d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:32.455 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.455+0000 7f356aa00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f35640b3330 0x7f35640b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:32.455 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.455+0000 7f356aa00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f356406c190 con 0x7f3564161d50 2024-04-20T10:21:32.455 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.455+0000 7f356aa00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f3564162130 con 0x7f35640b3330 2024-04-20T10:21:32.455 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.455+0000 7f356aa00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f356406c470 con 0x7f356406cde0 2024-04-20T10:21:32.455 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.455+0000 7f356a000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f35640b3330 0x7f35640b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:32.455 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.456+0000 7f3569600640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f3564161d50 0x7f356406c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:49996/0 (socket says 172.21.15.121:49996) 2024-04-20T10:21:32.455 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.456+0000 7f3569600640 1 -- 172.21.15.121:0/2100010244 learned_addr learned my addr 172.21.15.121:0/2100010244 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:32.455 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.456+0000 7f3568c00640 1 --2- 172.21.15.121:0/2100010244 >> v2:172.21.15.123:3300/0 conn(0x7f356406cde0 0x7f35640d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:32.456 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.456+0000 7f356a000640 1 -- 172.21.15.121:0/2100010244 >> v2:172.21.15.123:3300/0 conn(0x7f356406cde0 msgr2=0x7f35640d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:32.456 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.456+0000 7f356a000640 1 --2- 172.21.15.121:0/2100010244 >> v2:172.21.15.123:3300/0 conn(0x7f356406cde0 0x7f35640d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:32.456 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.456+0000 7f356a000640 1 -- 172.21.15.121:0/2100010244 >> v1:172.21.15.121:6789/0 conn(0x7f3564161d50 legacy=0x7f356406c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:32.456 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.456+0000 7f356a000640 1 -- 172.21.15.121:0/2100010244 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35640d2e00 con 0x7f35640b3330 2024-04-20T10:21:32.456 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.456+0000 7f3568c00640 1 --2- 172.21.15.121:0/2100010244 >> v2:172.21.15.123:3300/0 conn(0x7f356406cde0 0x7f35640d38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:21:32.456 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.456+0000 7f3563e00640 1 -- 172.21.15.121:0/2100010244 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2460066722 0 0) 0x7f356406c190 con 0x7f3564161d50 2024-04-20T10:21:32.457 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.458+0000 7f356a000640 1 --2- 172.21.15.121:0/2100010244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35640b3330 0x7f35640b3880 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f35640d1f00 tx=0x7f355c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=501201405bdc891 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:32.457 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.458+0000 7f3563e00640 1 -- 172.21.15.121:0/2100010244 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355c039070 con 0x7f35640b3330 2024-04-20T10:21:32.457 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.458+0000 7f3563e00640 1 -- 172.21.15.121:0/2100010244 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f355c02cc80 con 0x7f35640b3330 2024-04-20T10:21:32.457 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.458+0000 7f3563e00640 1 -- 172.21.15.121:0/2100010244 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355c034660 con 0x7f35640b3330 2024-04-20T10:21:32.458 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.459+0000 7f356aa00640 1 -- 172.21.15.121:0/2100010244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35640b3330 msgr2=0x7f35640b3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:32.458 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.459+0000 7f356aa00640 1 --2- 172.21.15.121:0/2100010244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35640b3330 0x7f35640b3880 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f35640d1f00 tx=0x7f355c02bed0 comp rx=0 tx=0).stop 2024-04-20T10:21:32.458 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.459+0000 7f356aa00640 1 -- 172.21.15.121:0/2100010244 shutdown_connections 2024-04-20T10:21:32.459 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.459+0000 7f356aa00640 1 --2- 172.21.15.121:0/2100010244 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35640b3330 0x7f35640b3880 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:32.459 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.459+0000 7f356aa00640 1 --2- 172.21.15.121:0/2100010244 >> v2:172.21.15.123:3300/0 conn(0x7f356406cde0 0x7f35640d38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:32.459 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.459+0000 7f356aa00640 1 -- 172.21.15.121:0/2100010244 >> 172.21.15.121:0/2100010244 conn(0x7f35640d4450 msgr2=0x7f35640d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:32.459 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.459+0000 7f356aa00640 1 -- 172.21.15.121:0/2100010244 shutdown_connections 2024-04-20T10:21:32.459 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.459+0000 7f356aa00640 1 -- 172.21.15.121:0/2100010244 wait complete. 2024-04-20T10:21:32.460 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.461+0000 7f356aa00640 1 Processor -- start 2024-04-20T10:21:32.460 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.461+0000 7f356aa00640 1 -- start start 2024-04-20T10:21:32.461 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.462+0000 7f356aa00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f356406cde0 0x7f356414f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:32.461 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.462+0000 7f356aa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35640b3330 0x7f35640b8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:32.461 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.462+0000 7f356aa00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3564161d50 0x7f35640b9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:32.462 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.462+0000 7f356aa00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f35640b7660 con 0x7f3564161d50 2024-04-20T10:21:32.462 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.462+0000 7f356aa00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f35640d33c0 con 0x7f35640b3330 2024-04-20T10:21:32.462 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.462+0000 7f356aa00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f35640b77e0 con 0x7f356406cde0 2024-04-20T10:21:32.462 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.462+0000 7f356a000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3564161d50 0x7f35640b9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:32.462 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.462+0000 7f356a000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3564161d50 0x7f35640b9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54190/0 (socket says 172.21.15.121:54190) 2024-04-20T10:21:32.462 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.462+0000 7f356a000640 1 -- 172.21.15.121:0/2666373365 learned_addr learned my addr 172.21.15.121:0/2666373365 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:32.463 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.462+0000 7f3569600640 1 --2- 172.21.15.121:0/2666373365 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f356406cde0 0x7f356414f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:32.463 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.462+0000 7f356a000640 1 -- 172.21.15.121:0/2666373365 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f356406cde0 msgr2=0x7f356414f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:32.463 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.462+0000 7f3568c00640 1 --2- 172.21.15.121:0/2666373365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35640b3330 0x7f35640b8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:32.464 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.462+0000 7f356a000640 1 --2- 172.21.15.121:0/2666373365 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f356406cde0 0x7f356414f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:32.464 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.462+0000 7f356a000640 1 -- 172.21.15.121:0/2666373365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35640b3330 msgr2=0x7f35640b8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:32.464 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.462+0000 7f356a000640 1 --2- 172.21.15.121:0/2666373365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35640b3330 0x7f35640b8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:32.464 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.462+0000 7f356a000640 1 -- 172.21.15.121:0/2666373365 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35640cc3e0 con 0x7f3564161d50 2024-04-20T10:21:32.464 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.462+0000 7f3569600640 1 --2- 172.21.15.121:0/2666373365 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f356406cde0 0x7f356414f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:32.464 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.462+0000 7f3568c00640 1 --2- 172.21.15.121:0/2666373365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35640b3330 0x7f35640b8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:32.464 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.463+0000 7f356a000640 1 --2- 172.21.15.121:0/2666373365 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3564161d50 0x7f35640b9050 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f355c009e80 tx=0x7f355c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:32.464 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.463+0000 7f3561600640 1 -- 172.21.15.121:0/2666373365 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355c039050 con 0x7f3564161d50 2024-04-20T10:21:32.464 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.463+0000 7f3561600640 1 -- 172.21.15.121:0/2666373365 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f355c045070 con 0x7f3564161d50 2024-04-20T10:21:32.464 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.463+0000 7f3561600640 1 -- 172.21.15.121:0/2666373365 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355c03e830 con 0x7f3564161d50 2024-04-20T10:21:32.464 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.463+0000 7f356aa00640 1 -- 172.21.15.121:0/2666373365 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f356414fda0 con 0x7f3564161d50 2024-04-20T10:21:32.464 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.463+0000 7f356aa00640 1 -- 172.21.15.121:0/2666373365 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35640cccb0 con 0x7f3564161d50 2024-04-20T10:21:32.465 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.464+0000 7f356aa00640 1 -- 172.21.15.121:0/2666373365 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3564075b60 con 0x7f3564161d50 2024-04-20T10:21:32.465 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.465+0000 7f3561600640 1 -- 172.21.15.121:0/2666373365 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f355c03e9d0 con 0x7f3564161d50 2024-04-20T10:21:32.466 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.466+0000 7f3561600640 1 --2- 172.21.15.121:0/2666373365 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f354806c920 0x7f354806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:32.466 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.466+0000 7f3561600640 1 -- 172.21.15.121:0/2666373365 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f355c0aecb0 con 0x7f3564161d50 2024-04-20T10:21:32.466 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.467+0000 7f3569600640 1 --2- 172.21.15.121:0/2666373365 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f354806c920 0x7f354806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:32.466 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.467+0000 7f3569600640 1 --2- 172.21.15.121:0/2666373365 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f354806c920 0x7f354806ede0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3554000f60 tx=0x7f3554009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:32.470 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.471+0000 7f3561600640 1 -- 172.21.15.121:0/2666373365 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f355c032e20 con 0x7f3564161d50 2024-04-20T10:21:32.723 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.724+0000 7f356aa00640 1 -- 172.21.15.121:0/2666373365 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f35640cb2e0 con 0x7f3564161d50 2024-04-20T10:21:32.729 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.729+0000 7f3561600640 1 -- 172.21.15.121:0/2666373365 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174143 (secure 0 0 0) 0x7f355c07cd00 con 0x7f3564161d50 2024-04-20T10:21:32.732 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:21:32.733 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.734+0000 7f356aa00640 1 -- 172.21.15.121:0/2666373365 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f354806c920 msgr2=0x7f354806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:32.733 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.734+0000 7f356aa00640 1 --2- 172.21.15.121:0/2666373365 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f354806c920 0x7f354806ede0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3554000f60 tx=0x7f3554009040 comp rx=0 tx=0).stop 2024-04-20T10:21:32.733 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.734+0000 7f356aa00640 1 -- 172.21.15.121:0/2666373365 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3564161d50 msgr2=0x7f35640b9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:32.733 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.734+0000 7f356aa00640 1 --2- 172.21.15.121:0/2666373365 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3564161d50 0x7f35640b9050 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f355c009e80 tx=0x7f355c004a40 comp rx=0 tx=0).stop 2024-04-20T10:21:32.734 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.734+0000 7f356aa00640 1 -- 172.21.15.121:0/2666373365 shutdown_connections 2024-04-20T10:21:32.734 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.734+0000 7f356aa00640 1 --2- 172.21.15.121:0/2666373365 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3564161d50 0x7f35640b9050 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:32.734 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.734+0000 7f356aa00640 1 --2- 172.21.15.121:0/2666373365 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f354806c920 0x7f354806ede0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:32.734 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.734+0000 7f356aa00640 1 --2- 172.21.15.121:0/2666373365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35640b3330 0x7f35640b8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:32.734 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.734+0000 7f356aa00640 1 --2- 172.21.15.121:0/2666373365 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f356406cde0 0x7f356414f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:32.734 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.734+0000 7f356aa00640 1 -- 172.21.15.121:0/2666373365 >> 172.21.15.121:0/2666373365 conn(0x7f35640d4450 msgr2=0x7f35640d2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:32.734 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.734+0000 7f356aa00640 1 -- 172.21.15.121:0/2666373365 shutdown_connections 2024-04-20T10:21:32.734 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.734+0000 7f356aa00640 1 -- 172.21.15.121:0/2666373365 wait complete. 2024-04-20T10:21:32.752 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":5,"flags":0,"active_gid":4103,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":1408233272},{"type":"v1","addr":"172.21.15.121:6833","nonce":1408233272}]},"active_addr":"172.21.15.121:6833/1408233272","active_change":"2024-04-20T10:20:43.086864+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4101,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.121:0","nonce":3567884452}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.121:0","nonce":2060179780}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.121:0","nonce":555958733}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.121:0","nonce":1535193339}]}]} 2024-04-20T10:21:32.756 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-20T10:21:32.756 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-20T10:21:32.756 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:21:32.958 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.958+0000 7f86f4e00640 1 Processor -- start 2024-04-20T10:21:32.958 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.958+0000 7f86f4e00640 1 -- start start 2024-04-20T10:21:32.960 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.960+0000 7f86f4e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f86f0156c20 0x7f86f006b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:32.960 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.960+0000 7f86f4e00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86f00d0360 0x7f86f00d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:32.960 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.960+0000 7f86f4e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86f0141c60 con 0x7f86f015ca40 2024-04-20T10:21:32.960 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.960+0000 7f86f4e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f86f00adab0 con 0x7f86f0156c20 2024-04-20T10:21:32.960 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.960+0000 7f86f4e00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f86f015cf90 con 0x7f86f00d0360 2024-04-20T10:21:32.960 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.960+0000 7f86ede00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f86f0156c20 0x7f86f006b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:32.960 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.960+0000 7f86ede00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f86f0156c20 0x7f86f006b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54206/0 (socket says 172.21.15.121:54206) 2024-04-20T10:21:32.960 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.960+0000 7f86ede00640 1 -- 172.21.15.121:0/48091764 learned_addr learned my addr 172.21.15.121:0/48091764 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:32.960 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.961+0000 7f86ee800640 1 --2- 172.21.15.121:0/48091764 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86f00d0360 0x7f86f00d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:32.961 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.961+0000 7f86ede00640 1 -- 172.21.15.121:0/48091764 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86f00d0360 msgr2=0x7f86f00d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:32.961 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.961+0000 7f86ede00640 1 --2- 172.21.15.121:0/48091764 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86f00d0360 0x7f86f00d07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:32.961 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.961+0000 7f86ede00640 1 -- 172.21.15.121:0/48091764 >> v1:172.21.15.123:6789/0 conn(0x7f86f015ca40 legacy=0x7f86f00ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:32.961 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.961+0000 7f86ede00640 1 -- 172.21.15.121:0/48091764 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f015d270 con 0x7f86f0156c20 2024-04-20T10:21:32.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.962+0000 7f86ede00640 1 --2- 172.21.15.121:0/48091764 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86f0156c20 0x7f86f006b900 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f86e0009870 tx=0x7f86e002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e0fd17c28d2660 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:32.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.963+0000 7f86ed400640 1 -- 172.21.15.121:0/48091764 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e0034070 con 0x7f86f0156c20 2024-04-20T10:21:32.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.963+0000 7f86ed400640 1 -- 172.21.15.121:0/48091764 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86e0030840 con 0x7f86f0156c20 2024-04-20T10:21:32.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.963+0000 7f86ed400640 1 -- 172.21.15.121:0/48091764 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e0030c10 con 0x7f86f0156c20 2024-04-20T10:21:32.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.963+0000 7f86f4e00640 1 -- 172.21.15.121:0/48091764 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86f0156c20 msgr2=0x7f86f006b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:32.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.963+0000 7f86f4e00640 1 --2- 172.21.15.121:0/48091764 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86f0156c20 0x7f86f006b900 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f86e0009870 tx=0x7f86e002b980 comp rx=0 tx=0).stop 2024-04-20T10:21:32.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.963+0000 7f86f4e00640 1 -- 172.21.15.121:0/48091764 shutdown_connections 2024-04-20T10:21:32.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.963+0000 7f86f4e00640 1 --2- 172.21.15.121:0/48091764 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86f00d0360 0x7f86f00d07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:32.964 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.963+0000 7f86f4e00640 1 --2- 172.21.15.121:0/48091764 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86f0156c20 0x7f86f006b900 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:32.964 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.963+0000 7f86f4e00640 1 -- 172.21.15.121:0/48091764 >> 172.21.15.121:0/48091764 conn(0x7f86f00b3350 msgr2=0x7f86f00b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:32.964 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.964+0000 7f86f4e00640 1 -- 172.21.15.121:0/48091764 shutdown_connections 2024-04-20T10:21:32.964 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.964+0000 7f86f4e00640 1 -- 172.21.15.121:0/48091764 wait complete. 2024-04-20T10:21:32.964 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.965+0000 7f86f4e00640 1 Processor -- start 2024-04-20T10:21:32.965 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.965+0000 7f86f4e00640 1 -- start start 2024-04-20T10:21:32.966 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.966+0000 7f86f4e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86f00d0360 0x7f86f0073080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:32.966 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.966+0000 7f86f4e00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86f0156c20 0x7f86f01f45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:32.966 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.966+0000 7f86f4e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86f015ca40 0x7f86f015b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:32.966 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.966+0000 7f86f4e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f86f006be40 con 0x7f86f015ca40 2024-04-20T10:21:32.966 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.966+0000 7f86f4e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f86f015d830 con 0x7f86f00d0360 2024-04-20T10:21:32.966 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.966+0000 7f86f4e00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f86f006bfc0 con 0x7f86f0156c20 2024-04-20T10:21:32.966 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.967+0000 7f86ee800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86f015ca40 0x7f86f015b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:32.966 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.967+0000 7f86ee800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86f015ca40 0x7f86f015b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54214/0 (socket says 172.21.15.121:54214) 2024-04-20T10:21:32.966 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.967+0000 7f86ee800640 1 -- 172.21.15.121:0/4125070500 learned_addr learned my addr 172.21.15.121:0/4125070500 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:32.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.967+0000 7f86e7e00640 1 --2- 172.21.15.121:0/4125070500 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86f0156c20 0x7f86f01f45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:32.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.967+0000 7f86ede00640 1 --2- 172.21.15.121:0/4125070500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86f00d0360 0x7f86f0073080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:32.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.967+0000 7f86ee800640 1 -- 172.21.15.121:0/4125070500 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86f0156c20 msgr2=0x7f86f01f45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:32.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.967+0000 7f86ee800640 1 --2- 172.21.15.121:0/4125070500 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86f0156c20 0x7f86f01f45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:32.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.967+0000 7f86ee800640 1 -- 172.21.15.121:0/4125070500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86f00d0360 msgr2=0x7f86f0073080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:32.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.967+0000 7f86ee800640 1 --2- 172.21.15.121:0/4125070500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86f00d0360 0x7f86f0073080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:32.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.967+0000 7f86ee800640 1 -- 172.21.15.121:0/4125070500 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86f0153730 con 0x7f86f015ca40 2024-04-20T10:21:32.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.967+0000 7f86ee800640 1 --2- 172.21.15.121:0/4125070500 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86f015ca40 0x7f86f015b410 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f86e8007d20 tx=0x7f86e800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:32.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.968+0000 7f86e6000640 1 -- 172.21.15.121:0/4125070500 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e8014420 con 0x7f86f015ca40 2024-04-20T10:21:32.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.968+0000 7f86e6000640 1 -- 172.21.15.121:0/4125070500 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86e8019070 con 0x7f86f015ca40 2024-04-20T10:21:32.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.968+0000 7f86e6000640 1 -- 172.21.15.121:0/4125070500 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86e8014820 con 0x7f86f015ca40 2024-04-20T10:21:32.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.968+0000 7f86f4e00640 1 -- 172.21.15.121:0/4125070500 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86f0155430 con 0x7f86f015ca40 2024-04-20T10:21:32.969 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.968+0000 7f86f4e00640 1 -- 172.21.15.121:0/4125070500 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86f01539d0 con 0x7f86f015ca40 2024-04-20T10:21:32.969 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.969+0000 7f86f4e00640 1 -- 172.21.15.121:0/4125070500 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86f0068980 con 0x7f86f015ca40 2024-04-20T10:21:32.969 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.970+0000 7f86e6000640 1 -- 172.21.15.121:0/4125070500 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f86e80149c0 con 0x7f86f015ca40 2024-04-20T10:21:32.970 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.971+0000 7f86e6000640 1 --2- 172.21.15.121:0/4125070500 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f86cc06c8f0 0x7f86cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:32.971 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.971+0000 7f86e6000640 1 -- 172.21.15.121:0/4125070500 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f86e808c710 con 0x7f86f015ca40 2024-04-20T10:21:32.971 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.971+0000 7f86ede00640 1 --2- 172.21.15.121:0/4125070500 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f86cc06c8f0 0x7f86cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:32.971 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.972+0000 7f86ede00640 1 --2- 172.21.15.121:0/4125070500 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f86cc06c8f0 0x7f86cc06edb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f86e0036000 tx=0x7f86e003b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:32.976 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:32.976+0000 7f86e6000640 1 -- 172.21.15.121:0/4125070500 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f86e805a7f0 con 0x7f86f015ca40 2024-04-20T10:21:33.178 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.178+0000 7f86f4e00640 1 -- 172.21.15.121:0/4125070500 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f86f015f3a0 con 0x7f86f015ca40 2024-04-20T10:21:33.181 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.182+0000 7f86e6000640 1 -- 172.21.15.121:0/4125070500 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7f86e805a0f0 con 0x7f86f015ca40 2024-04-20T10:21:33.182 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:21:33.182 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":25,"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","created":"2024-04-20T10:20:38.076076+0000","modified":"2024-04-20T10:21:07.336542+0000","last_up_change":"2024-04-20T10:20:50.162700+0000","last_in_change":"2024-04-20T10:20:41.510382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:20:51.141365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:21:00.436937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:21:01.026957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:21:04.862693+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"629a7b8c-a40a-4a2a-a3cc-d9f357774310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6809","nonce":1044588835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6811","nonce":1044588835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6815","nonce":1044588835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6813","nonce":1044588835}]},"public_addr":"172.21.15.121:6809/1044588835","cluster_addr":"172.21.15.121:6811/1044588835","heartbeat_back_addr":"172.21.15.121:6815/1044588835","heartbeat_front_addr":"172.21.15.121:6813/1044588835","state":["exists","up"]},{"osd":1,"uuid":"f2570399-aa50-4ff3-bf5d-3d09207290db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6801","nonce":3505199624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6803","nonce":3505199624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6807","nonce":3505199624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6805","nonce":3505199624}]},"public_addr":"172.21.15.121:6801/3505199624","cluster_addr":"172.21.15.121:6803/3505199624","heartbeat_back_addr":"172.21.15.121:6807/3505199624","heartbeat_front_addr":"172.21.15.121:6805/3505199624","state":["exists","up"]},{"osd":2,"uuid":"50893807-dc30-49f4-b41f-739366626658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6817","nonce":3656724207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6819","nonce":3656724207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6827","nonce":3656724207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6822","nonce":3656724207}]},"public_addr":"172.21.15.121:6817/3656724207","cluster_addr":"172.21.15.121:6819/3656724207","heartbeat_back_addr":"172.21.15.121:6827/3656724207","heartbeat_front_addr":"172.21.15.121:6822/3656724207","state":["exists","up"]},{"osd":3,"uuid":"dadb01bc-5491-4ffc-8824-d48a057240b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6823","nonce":2232342832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6826","nonce":2232342832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6831","nonce":2232342832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6829","nonce":2232342832}]},"public_addr":"172.21.15.121:6823/2232342832","cluster_addr":"172.21.15.121:6826/2232342832","heartbeat_back_addr":"172.21.15.121:6831/2232342832","heartbeat_front_addr":"172.21.15.121:6829/2232342832","state":["exists","up"]},{"osd":4,"uuid":"b3f3b4a1-b375-4526-b69e-344ab24c2da1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6801","nonce":99353111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6803","nonce":99353111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6807","nonce":99353111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6805","nonce":99353111}]},"public_addr":"172.21.15.123:6801/99353111","cluster_addr":"172.21.15.123:6803/99353111","heartbeat_back_addr":"172.21.15.123:6807/99353111","heartbeat_front_addr":"172.21.15.123:6805/99353111","state":["exists","up"]},{"osd":5,"uuid":"2ee24a08-1624-4961-9642-d7b510282484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6818","nonce":2504374688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6822","nonce":2504374688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6830","nonce":2504374688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6826","nonce":2504374688}]},"public_addr":"172.21.15.123:6818/2504374688","cluster_addr":"172.21.15.123:6822/2504374688","heartbeat_back_addr":"172.21.15.123:6830/2504374688","heartbeat_front_addr":"172.21.15.123:6826/2504374688","state":["exists","up"]},{"osd":6,"uuid":"186118cc-6128-49de-acca-55260945c1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6819","nonce":4139299599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6823","nonce":4139299599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6829","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6831","nonce":4139299599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6827","nonce":4139299599}]},"public_addr":"172.21.15.123:6819/4139299599","cluster_addr":"172.21.15.123:6823/4139299599","heartbeat_back_addr":"172.21.15.123:6831/4139299599","heartbeat_front_addr":"172.21.15.123:6827/4139299599","state":["exists","up"]},{"osd":7,"uuid":"a00aac6e-4bd9-415a-a93d-ae85210865ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6809","nonce":4043981069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6811","nonce":4043981069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6815","nonce":4043981069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6813","nonce":4043981069}]},"public_addr":"172.21.15.123:6809/4043981069","cluster_addr":"172.21.15.123:6811/4043981069","heartbeat_back_addr":"172.21.15.123:6815/4043981069","heartbeat_front_addr":"172.21.15.123:6813/4043981069","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.904540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.958585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.616367+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.885244+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.103127+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.167248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.884348+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.862356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:21:33.184 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.184+0000 7f86f4e00640 1 -- 172.21.15.121:0/4125070500 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f86cc06c8f0 msgr2=0x7f86cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:33.184 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.184+0000 7f86f4e00640 1 --2- 172.21.15.121:0/4125070500 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f86cc06c8f0 0x7f86cc06edb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f86e0036000 tx=0x7f86e003b040 comp rx=0 tx=0).stop 2024-04-20T10:21:33.184 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.184+0000 7f86f4e00640 1 -- 172.21.15.121:0/4125070500 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86f015ca40 msgr2=0x7f86f015b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:33.184 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.184+0000 7f86f4e00640 1 --2- 172.21.15.121:0/4125070500 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86f015ca40 0x7f86f015b410 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f86e8007d20 tx=0x7f86e800e4b0 comp rx=0 tx=0).stop 2024-04-20T10:21:33.185 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.185+0000 7f86f4e00640 1 -- 172.21.15.121:0/4125070500 shutdown_connections 2024-04-20T10:21:33.185 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.185+0000 7f86f4e00640 1 --2- 172.21.15.121:0/4125070500 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86f015ca40 0x7f86f015b410 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:33.185 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.185+0000 7f86f4e00640 1 --2- 172.21.15.121:0/4125070500 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86f0156c20 0x7f86f01f45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:33.185 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.185+0000 7f86f4e00640 1 --2- 172.21.15.121:0/4125070500 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f86cc06c8f0 0x7f86cc06edb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:33.185 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.185+0000 7f86f4e00640 1 --2- 172.21.15.121:0/4125070500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86f00d0360 0x7f86f0073080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:33.185 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.185+0000 7f86f4e00640 1 -- 172.21.15.121:0/4125070500 >> 172.21.15.121:0/4125070500 conn(0x7f86f00b3350 msgr2=0x7f86f00997c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:33.185 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.185+0000 7f86f4e00640 1 -- 172.21.15.121:0/4125070500 shutdown_connections 2024-04-20T10:21:33.185 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.185+0000 7f86f4e00640 1 -- 172.21.15.121:0/4125070500 wait complete. 2024-04-20T10:21:33.205 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-20T10:21:33.205 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:21:33.403 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.403+0000 7f354a800640 1 Processor -- start 2024-04-20T10:21:33.403 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.403+0000 7f354a800640 1 -- start start 2024-04-20T10:21:33.405 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.405+0000 7f354a800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f354415e8a0 0x7f354406e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:33.405 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.405+0000 7f354a800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f354406dbb0 0x7f35440d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:33.405 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.405+0000 7f354a800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f35440d08a0 0x7f354406b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:33.405 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.405+0000 7f354a800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f354406f060 con 0x7f35440d08a0 2024-04-20T10:21:33.405 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.405+0000 7f354a800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f354406e7c0 con 0x7f354406dbb0 2024-04-20T10:21:33.405 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.405+0000 7f354a800640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f354406eaa0 con 0x7f354415e8a0 2024-04-20T10:21:33.406 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.405+0000 7f3543e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f35440d08a0 0x7f354406b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:33.406 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.405+0000 7f3543e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f35440d08a0 0x7f354406b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54216/0 (socket says 172.21.15.121:54216) 2024-04-20T10:21:33.406 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.405+0000 7f3543e00640 1 -- 172.21.15.121:0/833932747 learned_addr learned my addr 172.21.15.121:0/833932747 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:33.406 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.406+0000 7f3543400640 1 --2- 172.21.15.121:0/833932747 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f354415e8a0 0x7f354406e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:33.406 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.406+0000 7f353aa00640 1 --2- 172.21.15.121:0/833932747 >> v2:172.21.15.123:3300/0 conn(0x7f354406dbb0 0x7f35440d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:33.406 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.406+0000 7f3543e00640 1 -- 172.21.15.121:0/833932747 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f354415e8a0 msgr2=0x7f354406e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:33.406 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.406+0000 7f3543e00640 1 --2- 172.21.15.121:0/833932747 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f354415e8a0 0x7f354406e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:33.406 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.406+0000 7f3543e00640 1 -- 172.21.15.121:0/833932747 >> v2:172.21.15.123:3300/0 conn(0x7f354406dbb0 msgr2=0x7f35440d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:33.406 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.406+0000 7f3543e00640 1 --2- 172.21.15.121:0/833932747 >> v2:172.21.15.123:3300/0 conn(0x7f354406dbb0 0x7f35440d0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:33.406 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.406+0000 7f3543e00640 1 -- 172.21.15.121:0/833932747 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f354406ed80 con 0x7f35440d08a0 2024-04-20T10:21:33.407 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.407+0000 7f3543e00640 1 --2- 172.21.15.121:0/833932747 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35440d08a0 0x7f354406b900 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f35440d0f40 tx=0x7f353c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89fe2e84f54998f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:33.407 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.408+0000 7f3542a00640 1 -- 172.21.15.121:0/833932747 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c02ca80 con 0x7f35440d08a0 2024-04-20T10:21:33.407 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.408+0000 7f3542a00640 1 -- 172.21.15.121:0/833932747 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f353c02cc20 con 0x7f35440d08a0 2024-04-20T10:21:33.407 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.408+0000 7f3542a00640 1 -- 172.21.15.121:0/833932747 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c0343f0 con 0x7f35440d08a0 2024-04-20T10:21:33.408 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.408+0000 7f354a800640 1 -- 172.21.15.121:0/833932747 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35440d08a0 msgr2=0x7f354406b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:33.408 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.408+0000 7f354a800640 1 --2- 172.21.15.121:0/833932747 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35440d08a0 0x7f354406b900 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f35440d0f40 tx=0x7f353c02bee0 comp rx=0 tx=0).stop 2024-04-20T10:21:33.408 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.409+0000 7f354a800640 1 -- 172.21.15.121:0/833932747 shutdown_connections 2024-04-20T10:21:33.408 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.409+0000 7f354a800640 1 --2- 172.21.15.121:0/833932747 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f35440d08a0 0x7f354406b900 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:33.408 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.409+0000 7f354a800640 1 --2- 172.21.15.121:0/833932747 >> v2:172.21.15.123:3300/0 conn(0x7f354406dbb0 0x7f35440d0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:33.408 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.409+0000 7f354a800640 1 --2- 172.21.15.121:0/833932747 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f354415e8a0 0x7f354406e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:33.408 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.409+0000 7f354a800640 1 -- 172.21.15.121:0/833932747 >> 172.21.15.121:0/833932747 conn(0x7f35440b3350 msgr2=0x7f35440b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:33.408 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.409+0000 7f354a800640 1 -- 172.21.15.121:0/833932747 shutdown_connections 2024-04-20T10:21:33.408 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.409+0000 7f354a800640 1 -- 172.21.15.121:0/833932747 wait complete. 2024-04-20T10:21:33.410 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.411+0000 7f354a800640 1 Processor -- start 2024-04-20T10:21:33.411 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.411+0000 7f354a800640 1 -- start start 2024-04-20T10:21:33.412 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.412+0000 7f354a800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f354406dbb0 0x7f35441f7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:33.412 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.412+0000 7f354a800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35440d08a0 0x7f35441f6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:33.412 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.412+0000 7f354a800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f354415e8a0 0x7f35441f7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:33.412 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.412+0000 7f354a800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f354406fef0 con 0x7f354415e8a0 2024-04-20T10:21:33.412 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.412+0000 7f354a800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f354406fd70 con 0x7f354406dbb0 2024-04-20T10:21:33.412 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.412+0000 7f354a800640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f354406be40 con 0x7f35440d08a0 2024-04-20T10:21:33.414 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.413+0000 7f3543e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f354415e8a0 0x7f35441f7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:33.414 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.413+0000 7f3543e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f354415e8a0 0x7f35441f7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54218/0 (socket says 172.21.15.121:54218) 2024-04-20T10:21:33.414 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.413+0000 7f3543e00640 1 -- 172.21.15.121:0/2497641472 learned_addr learned my addr 172.21.15.121:0/2497641472 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:33.414 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.413+0000 7f353aa00640 1 --2- 172.21.15.121:0/2497641472 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35440d08a0 0x7f35441f6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:33.414 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.413+0000 7f3543400640 1 --2- 172.21.15.121:0/2497641472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f354406dbb0 0x7f35441f7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:33.414 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.413+0000 7f3543e00640 1 -- 172.21.15.121:0/2497641472 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35440d08a0 msgr2=0x7f35441f6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:33.414 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.413+0000 7f3543e00640 1 --2- 172.21.15.121:0/2497641472 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35440d08a0 0x7f35441f6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:33.414 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.413+0000 7f3543e00640 1 -- 172.21.15.121:0/2497641472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f354406dbb0 msgr2=0x7f35441f7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:33.414 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.413+0000 7f3543e00640 1 --2- 172.21.15.121:0/2497641472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f354406dbb0 0x7f35441f7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:33.414 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.413+0000 7f3543e00640 1 -- 172.21.15.121:0/2497641472 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f354409ca80 con 0x7f354415e8a0 2024-04-20T10:21:33.415 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.414+0000 7f3543e00640 1 --2- 172.21.15.121:0/2497641472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f354415e8a0 0x7f35441f7200 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f353c009f40 tx=0x7f353c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:33.415 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.414+0000 7f353be00640 1 -- 172.21.15.121:0/2497641472 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c043020 con 0x7f354415e8a0 2024-04-20T10:21:33.415 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.414+0000 7f353be00640 1 -- 172.21.15.121:0/2497641472 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f353c03e450 con 0x7f354415e8a0 2024-04-20T10:21:33.415 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.414+0000 7f353be00640 1 -- 172.21.15.121:0/2497641472 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c009510 con 0x7f354415e8a0 2024-04-20T10:21:33.415 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.414+0000 7f354a800640 1 -- 172.21.15.121:0/2497641472 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f354414b4a0 con 0x7f354415e8a0 2024-04-20T10:21:33.415 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.414+0000 7f354a800640 1 -- 172.21.15.121:0/2497641472 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f354414ba60 con 0x7f354415e8a0 2024-04-20T10:21:33.416 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.416+0000 7f354a800640 1 -- 172.21.15.121:0/2497641472 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3544068980 con 0x7f354415e8a0 2024-04-20T10:21:33.416 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.416+0000 7f353be00640 1 -- 172.21.15.121:0/2497641472 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f353c046020 con 0x7f354415e8a0 2024-04-20T10:21:33.417 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.417+0000 7f353be00640 1 --2- 172.21.15.121:0/2497641472 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f352006c8f0 0x7f352006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:33.417 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.418+0000 7f353be00640 1 -- 172.21.15.121:0/2497641472 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f353c0ae690 con 0x7f354415e8a0 2024-04-20T10:21:33.417 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.418+0000 7f3543400640 1 --2- 172.21.15.121:0/2497641472 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f352006c8f0 0x7f352006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:33.417 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.418+0000 7f3543400640 1 --2- 172.21.15.121:0/2497641472 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f352006c8f0 0x7f352006edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3534006fd0 tx=0x7f3534008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:33.421 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.422+0000 7f353be00640 1 -- 172.21.15.121:0/2497641472 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f353c04d030 con 0x7f354415e8a0 2024-04-20T10:21:33.625 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.625+0000 7f354a800640 1 -- 172.21.15.121:0/2497641472 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f354406cbd0 con 0x7f354415e8a0 2024-04-20T10:21:33.627 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.628+0000 7f353be00640 1 -- 172.21.15.121:0/2497641472 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7f353c07c780 con 0x7f354415e8a0 2024-04-20T10:21:33.628 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:21:33.628 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":25,"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","created":"2024-04-20T10:20:38.076076+0000","modified":"2024-04-20T10:21:07.336542+0000","last_up_change":"2024-04-20T10:20:50.162700+0000","last_in_change":"2024-04-20T10:20:41.510382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:20:51.141365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:21:00.436937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:21:01.026957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:21:04.862693+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"629a7b8c-a40a-4a2a-a3cc-d9f357774310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6809","nonce":1044588835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6811","nonce":1044588835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6815","nonce":1044588835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6813","nonce":1044588835}]},"public_addr":"172.21.15.121:6809/1044588835","cluster_addr":"172.21.15.121:6811/1044588835","heartbeat_back_addr":"172.21.15.121:6815/1044588835","heartbeat_front_addr":"172.21.15.121:6813/1044588835","state":["exists","up"]},{"osd":1,"uuid":"f2570399-aa50-4ff3-bf5d-3d09207290db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6801","nonce":3505199624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6803","nonce":3505199624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6807","nonce":3505199624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6805","nonce":3505199624}]},"public_addr":"172.21.15.121:6801/3505199624","cluster_addr":"172.21.15.121:6803/3505199624","heartbeat_back_addr":"172.21.15.121:6807/3505199624","heartbeat_front_addr":"172.21.15.121:6805/3505199624","state":["exists","up"]},{"osd":2,"uuid":"50893807-dc30-49f4-b41f-739366626658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6817","nonce":3656724207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6819","nonce":3656724207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6827","nonce":3656724207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6822","nonce":3656724207}]},"public_addr":"172.21.15.121:6817/3656724207","cluster_addr":"172.21.15.121:6819/3656724207","heartbeat_back_addr":"172.21.15.121:6827/3656724207","heartbeat_front_addr":"172.21.15.121:6822/3656724207","state":["exists","up"]},{"osd":3,"uuid":"dadb01bc-5491-4ffc-8824-d48a057240b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6823","nonce":2232342832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6826","nonce":2232342832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6831","nonce":2232342832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6829","nonce":2232342832}]},"public_addr":"172.21.15.121:6823/2232342832","cluster_addr":"172.21.15.121:6826/2232342832","heartbeat_back_addr":"172.21.15.121:6831/2232342832","heartbeat_front_addr":"172.21.15.121:6829/2232342832","state":["exists","up"]},{"osd":4,"uuid":"b3f3b4a1-b375-4526-b69e-344ab24c2da1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6801","nonce":99353111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6803","nonce":99353111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6807","nonce":99353111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6805","nonce":99353111}]},"public_addr":"172.21.15.123:6801/99353111","cluster_addr":"172.21.15.123:6803/99353111","heartbeat_back_addr":"172.21.15.123:6807/99353111","heartbeat_front_addr":"172.21.15.123:6805/99353111","state":["exists","up"]},{"osd":5,"uuid":"2ee24a08-1624-4961-9642-d7b510282484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6818","nonce":2504374688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6822","nonce":2504374688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6830","nonce":2504374688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6826","nonce":2504374688}]},"public_addr":"172.21.15.123:6818/2504374688","cluster_addr":"172.21.15.123:6822/2504374688","heartbeat_back_addr":"172.21.15.123:6830/2504374688","heartbeat_front_addr":"172.21.15.123:6826/2504374688","state":["exists","up"]},{"osd":6,"uuid":"186118cc-6128-49de-acca-55260945c1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6819","nonce":4139299599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6823","nonce":4139299599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6829","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6831","nonce":4139299599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6827","nonce":4139299599}]},"public_addr":"172.21.15.123:6819/4139299599","cluster_addr":"172.21.15.123:6823/4139299599","heartbeat_back_addr":"172.21.15.123:6831/4139299599","heartbeat_front_addr":"172.21.15.123:6827/4139299599","state":["exists","up"]},{"osd":7,"uuid":"a00aac6e-4bd9-415a-a93d-ae85210865ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6809","nonce":4043981069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6811","nonce":4043981069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6815","nonce":4043981069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6813","nonce":4043981069}]},"public_addr":"172.21.15.123:6809/4043981069","cluster_addr":"172.21.15.123:6811/4043981069","heartbeat_back_addr":"172.21.15.123:6815/4043981069","heartbeat_front_addr":"172.21.15.123:6813/4043981069","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.904540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.958585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.616367+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.885244+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.103127+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.167248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.884348+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.862356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:21:33.630 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.631+0000 7f354a800640 1 -- 172.21.15.121:0/2497641472 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f352006c8f0 msgr2=0x7f352006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:33.630 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.631+0000 7f354a800640 1 --2- 172.21.15.121:0/2497641472 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f352006c8f0 0x7f352006edb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3534006fd0 tx=0x7f3534008040 comp rx=0 tx=0).stop 2024-04-20T10:21:33.630 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.631+0000 7f354a800640 1 -- 172.21.15.121:0/2497641472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f354415e8a0 msgr2=0x7f35441f7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:33.631 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.631+0000 7f354a800640 1 --2- 172.21.15.121:0/2497641472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f354415e8a0 0x7f35441f7200 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f353c009f40 tx=0x7f353c004a40 comp rx=0 tx=0).stop 2024-04-20T10:21:33.631 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.631+0000 7f354a800640 1 -- 172.21.15.121:0/2497641472 shutdown_connections 2024-04-20T10:21:33.631 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.631+0000 7f354a800640 1 --2- 172.21.15.121:0/2497641472 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f354415e8a0 0x7f35441f7200 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:33.631 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.631+0000 7f354a800640 1 --2- 172.21.15.121:0/2497641472 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f35440d08a0 0x7f35441f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:33.631 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.631+0000 7f354a800640 1 --2- 172.21.15.121:0/2497641472 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f352006c8f0 0x7f352006edb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:33.631 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.631+0000 7f354a800640 1 --2- 172.21.15.121:0/2497641472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f354406dbb0 0x7f35441f7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:33.632 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.631+0000 7f354a800640 1 -- 172.21.15.121:0/2497641472 >> 172.21.15.121:0/2497641472 conn(0x7f35440b3350 msgr2=0x7f35440723e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:33.632 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.632+0000 7f354a800640 1 -- 172.21.15.121:0/2497641472 shutdown_connections 2024-04-20T10:21:33.632 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.632+0000 7f354a800640 1 -- 172.21.15.121:0/2497641472 wait complete. 2024-04-20T10:21:33.651 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-20T10:21:33.652 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-20T10:21:33.652 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-20T10:21:33.652 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-20T10:21:33.653 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-20T10:21:33.653 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-20T10:21:33.653 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-20T10:21:33.654 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-20T10:21:33.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.942+0000 7f8347800640 1 Processor -- start 2024-04-20T10:21:33.943 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.944+0000 7efe9ea00640 1 Processor -- start 2024-04-20T10:21:33.945 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.943+0000 7f8347800640 1 -- start start 2024-04-20T10:21:33.946 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.944+0000 7f8347800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f83380078d0 0x7f83380edd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:33.946 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.944+0000 7f8347800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8338058560 con 0x7f833803e830 2024-04-20T10:21:33.946 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.945+0000 7f8347800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8338058840 con 0x7f83380e30b0 2024-04-20T10:21:33.946 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.945+0000 7f8347800640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f8338058b20 con 0x7f83380078d0 2024-04-20T10:21:33.946 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.946+0000 7f8345800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f83380078d0 0x7f83380edd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:33.947 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.946+0000 7f8344e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f83380e30b0 0x7f8338057d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.121:46642/0 (socket says 172.21.15.121:46642) 2024-04-20T10:21:33.947 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.946+0000 7f8344e00640 1 -- 172.21.15.121:0/2086949410 learned_addr learned my addr 172.21.15.121:0/2086949410 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:33.948 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.944+0000 7efe9ea00640 1 -- start start 2024-04-20T10:21:33.948 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.946+0000 7efe9ea00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe9815e600 0x7efe980b6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:33.948 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.946+0000 7efe9ea00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe980b6cc0 con 0x7efe980d4930 2024-04-20T10:21:33.948 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.946+0000 7f833f400640 1 -- 172.21.15.121:0/2086949410 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3347137313 0 0) 0x7f8338058840 con 0x7f83380e30b0 2024-04-20T10:21:33.948 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.947+0000 7f833f400640 1 -- 172.21.15.121:0/2086949410 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8328003520 con 0x7f83380e30b0 2024-04-20T10:21:33.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.946+0000 7efe9ea00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe9806ed90 con 0x7efe980b38e0 2024-04-20T10:21:33.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.946+0000 7efe9ea00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7efe980d1d00 con 0x7efe9815e600 2024-04-20T10:21:33.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.947+0000 7f8345800640 1 -- 172.21.15.121:0/2086949410 >> v1:172.21.15.123:6789/0 conn(0x7f83380e30b0 legacy=0x7f8338057d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:33.950 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.947+0000 7f8345800640 1 -- 172.21.15.121:0/2086949410 >> v1:172.21.15.121:6789/0 conn(0x7f833803e830 legacy=0x7f8338053be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:33.950 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.948+0000 7efe9e000640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7efe980d4930 0x7efe980d1f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:50010/0 (socket says 172.21.15.121:50010) 2024-04-20T10:21:33.950 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.948+0000 7f8345800640 1 -- 172.21.15.121:0/2086949410 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83380063a0 con 0x7f83380078d0 2024-04-20T10:21:33.951 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.948+0000 7efe9e000640 1 -- 172.21.15.121:0/1854441554 learned_addr learned my addr 172.21.15.121:0/1854441554 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:33.951 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.948+0000 7efe9d600640 1 --2- 172.21.15.121:0/1854441554 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe9815e600 0x7efe980b6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:33.951 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.949+0000 7efe97e00640 1 -- 172.21.15.121:0/1854441554 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2365532345 0 0) 0x7efe9806ed90 con 0x7efe980b38e0 2024-04-20T10:21:33.951 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.949+0000 7efe97e00640 1 -- 172.21.15.121:0/1854441554 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe80003520 con 0x7efe980b38e0 2024-04-20T10:21:33.951 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.949+0000 7efe97e00640 1 -- 172.21.15.121:0/1854441554 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3790812835 0 0) 0x7efe980b6cc0 con 0x7efe980d4930 2024-04-20T10:21:33.951 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.949+0000 7efe97e00640 1 -- 172.21.15.121:0/1854441554 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe9806ed90 con 0x7efe980d4930 2024-04-20T10:21:33.952 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.949+0000 7f9751200640 1 Processor -- start 2024-04-20T10:21:33.952 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.949+0000 7efe9d600640 1 -- 172.21.15.121:0/1854441554 >> v1:172.21.15.123:6789/0 conn(0x7efe980b38e0 legacy=0x7efe980d4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:33.952 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.949+0000 7efe9d600640 1 -- 172.21.15.121:0/1854441554 >> v1:172.21.15.121:6789/0 conn(0x7efe980d4930 legacy=0x7efe980d1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:33.952 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.949+0000 7efe9d600640 1 -- 172.21.15.121:0/1854441554 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe9806f070 con 0x7efe9815e600 2024-04-20T10:21:33.952 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.950+0000 7f8345800640 1 --2- 172.21.15.121:0/2086949410 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f83380078d0 0x7f83380edd80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8340050210 tx=0x7f83400a71f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d756744e7c853790 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:33.953 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.950+0000 7f9751200640 1 -- start start 2024-04-20T10:21:33.953 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.950+0000 7f833f400640 1 -- 172.21.15.121:0/2086949410 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83400b4070 con 0x7f83380078d0 2024-04-20T10:21:33.953 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.951+0000 7f833f400640 1 -- 172.21.15.121:0/2086949410 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83400af640 con 0x7f83380078d0 2024-04-20T10:21:33.953 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.951+0000 7f833f400640 1 -- 172.21.15.121:0/2086949410 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83400a7e60 con 0x7f83380078d0 2024-04-20T10:21:33.954 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.951+0000 7f0d5ca00640 1 Processor -- start 2024-04-20T10:21:33.954 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.952+0000 7f9751200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f974c06bdd0 0x7f974c0ad210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:33.954 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.952+0000 7f9751200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f974c0ad810 con 0x7f974c0b3bf0 2024-04-20T10:21:33.954 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.952+0000 7f9751200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f974c0adaf0 con 0x7f974c06e510 2024-04-20T10:21:33.954 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.952+0000 7f9751200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f974c0addd0 con 0x7f974c06bdd0 2024-04-20T10:21:33.955 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.952+0000 7f8347800640 1 -- 172.21.15.121:0/2086949410 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f83380078d0 msgr2=0x7f83380edd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:33.955 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.952+0000 7f8347800640 1 --2- 172.21.15.121:0/2086949410 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f83380078d0 0x7f83380edd80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8340050210 tx=0x7f83400a71f0 comp rx=0 tx=0).stop 2024-04-20T10:21:33.955 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.952+0000 7f8347800640 1 -- 172.21.15.121:0/2086949410 shutdown_connections 2024-04-20T10:21:33.955 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.952+0000 7f8347800640 1 --2- 172.21.15.121:0/2086949410 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f83380078d0 0x7f83380edd80 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:33.955 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.951+0000 7f0d5ca00640 1 -- start start 2024-04-20T10:21:33.956 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.952+0000 7f8347800640 1 -- 172.21.15.121:0/2086949410 >> 172.21.15.121:0/2086949410 conn(0x7f83380559d0 msgr2=0x7f8338055de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:33.956 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.952+0000 7f9749e00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f974c0b3bf0 0x7f974c06de00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:50012/0 (socket says 172.21.15.121:50012) 2024-04-20T10:21:33.956 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.952+0000 7f9749e00640 1 -- 172.21.15.121:0/1135637045 learned_addr learned my addr 172.21.15.121:0/1135637045 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:33.956 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.953+0000 7f8347800640 1 -- 172.21.15.121:0/2086949410 shutdown_connections 2024-04-20T10:21:33.956 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.953+0000 7f974a800640 1 --2- 172.21.15.121:0/1135637045 >> v2:172.21.15.121:3300/0 conn(0x7f974c06bdd0 0x7f974c0ad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:33.957 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.953+0000 7efe9d600640 1 --2- 172.21.15.121:0/1854441554 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe9815e600 0x7efe980b6780 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7efe90009dc0 tx=0x7efe9002bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74732d31196ce2db server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:33.957 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.953+0000 7f9748a00640 1 -- 172.21.15.121:0/1135637045 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2827575700 0 0) 0x7f974c0ad810 con 0x7f974c0b3bf0 2024-04-20T10:21:33.957 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.953+0000 7f9748a00640 1 -- 172.21.15.121:0/1135637045 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9730003520 con 0x7f974c0b3bf0 2024-04-20T10:21:33.957 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.953+0000 7f974a800640 1 -- 172.21.15.121:0/1135637045 >> v1:172.21.15.123:6789/0 conn(0x7f974c06e510 legacy=0x7f974c0ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:33.958 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.953+0000 7f0d5ca00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d5815e8c0 0x7f0d5806ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:33.958 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.953+0000 7f0d5ca00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0d580d46f0 0x7f0d580d4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:33.958 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.953+0000 7f0d5ca00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d580d15b0 con 0x7f0d580d1010 2024-04-20T10:21:33.958 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.953+0000 7f0d5ca00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f0d580b6d10 con 0x7f0d580d46f0 2024-04-20T10:21:33.958 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.953+0000 7f0d5ca00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f0d580b6170 con 0x7f0d5815e8c0 2024-04-20T10:21:33.961 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.953+0000 7f974a800640 1 -- 172.21.15.121:0/1135637045 >> v1:172.21.15.121:6789/0 conn(0x7f974c0b3bf0 legacy=0x7f974c06de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:33.961 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.953+0000 7f974a800640 1 -- 172.21.15.121:0/1135637045 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f974c0ae0b0 con 0x7f974c06bdd0 2024-04-20T10:21:33.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.954+0000 7f0d55e00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f0d580d1010 0x7f0d580b6600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:50028/0 (socket says 172.21.15.121:50028) 2024-04-20T10:21:33.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.954+0000 7f0d55e00640 1 -- 172.21.15.121:0/2959520242 learned_addr learned my addr 172.21.15.121:0/2959520242 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:33.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.954+0000 7f0d56800640 1 --2- 172.21.15.121:0/2959520242 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d5815e8c0 0x7f0d5806ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:33.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.955+0000 7f0d57200640 1 --2- 172.21.15.121:0/2959520242 >> v2:172.21.15.121:3300/0 conn(0x7f0d580d46f0 0x7f0d580d4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:33.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.955+0000 7f974a800640 1 --2- 172.21.15.121:0/1135637045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f974c06bdd0 0x7f974c0ad210 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f97380097a0 tx=0x7f973802ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba0a63d51833cec server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:33.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.955+0000 7f0d57200640 1 -- 172.21.15.121:0/2959520242 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d5815e8c0 msgr2=0x7f0d5806ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:33.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.953+0000 7f8347800640 1 -- 172.21.15.121:0/2086949410 wait complete. 2024-04-20T10:21:33.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.955+0000 7f0d57200640 1 --2- 172.21.15.121:0/2959520242 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d5815e8c0 0x7f0d5806ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:33.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.955+0000 7f0d57200640 1 -- 172.21.15.121:0/2959520242 >> v1:172.21.15.121:6789/0 conn(0x7f0d580d1010 legacy=0x7f0d580b6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:33.964 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.955+0000 7f0d57200640 1 -- 172.21.15.121:0/2959520242 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d580d1890 con 0x7f0d580d46f0 2024-04-20T10:21:33.964 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.956+0000 7f8347800640 1 Processor -- start 2024-04-20T10:21:33.964 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.956+0000 7f9748a00640 1 -- 172.21.15.121:0/1135637045 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973802f830 con 0x7f974c06bdd0 2024-04-20T10:21:33.964 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.956+0000 7f9748a00640 1 -- 172.21.15.121:0/1135637045 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f973802f9d0 con 0x7f974c06bdd0 2024-04-20T10:21:33.964 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.956+0000 7f9748a00640 1 -- 172.21.15.121:0/1135637045 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973802fd80 con 0x7f974c06bdd0 2024-04-20T10:21:33.965 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.956+0000 7efe97e00640 1 -- 172.21.15.121:0/1854441554 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe90039070 con 0x7efe9815e600 2024-04-20T10:21:33.965 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.956+0000 7efe97e00640 1 -- 172.21.15.121:0/1854441554 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe9002cd50 con 0x7efe9815e600 2024-04-20T10:21:33.965 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.956+0000 7efe97e00640 1 -- 172.21.15.121:0/1854441554 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe90039070 con 0x7efe9815e600 2024-04-20T10:21:33.965 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.956+0000 7f8347800640 1 -- start start 2024-04-20T10:21:33.965 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.957+0000 7f9751200640 1 -- 172.21.15.121:0/1135637045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f974c06bdd0 msgr2=0x7f974c0ad210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:33.966 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.957+0000 7efe9ea00640 1 -- 172.21.15.121:0/1854441554 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe9815e600 msgr2=0x7efe980b6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:33.966 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.957+0000 7efe9ea00640 1 --2- 172.21.15.121:0/1854441554 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe9815e600 0x7efe980b6780 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7efe90009dc0 tx=0x7efe9002bfa0 comp rx=0 tx=0).stop 2024-04-20T10:21:33.966 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.957+0000 7f0d57200640 1 --2- 172.21.15.121:0/2959520242 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d580d46f0 0x7f0d580d4b80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0d5000b050 tx=0x7f0d5002c100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87c116d50cadc94 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:33.966 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.957+0000 7f9751200640 1 --2- 172.21.15.121:0/1135637045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f974c06bdd0 0x7f974c0ad210 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f97380097a0 tx=0x7f973802ba10 comp rx=0 tx=0).stop 2024-04-20T10:21:33.967 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.958+0000 7efe9ea00640 1 -- 172.21.15.121:0/1854441554 shutdown_connections 2024-04-20T10:21:33.967 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.958+0000 7efe9ea00640 1 --2- 172.21.15.121:0/1854441554 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe9815e600 0x7efe980b6780 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:33.967 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.957+0000 7f9751200640 1 -- 172.21.15.121:0/1135637045 shutdown_connections 2024-04-20T10:21:33.967 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.957+0000 7f9751200640 1 --2- 172.21.15.121:0/1135637045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f974c06bdd0 0x7f974c0ad210 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:33.967 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.957+0000 7f9751200640 1 -- 172.21.15.121:0/1135637045 >> 172.21.15.121:0/1135637045 conn(0x7f974c072090 msgr2=0x7f974c0b4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:33.967 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.958+0000 7f9751200640 1 -- 172.21.15.121:0/1135637045 shutdown_connections 2024-04-20T10:21:33.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.958+0000 7efe9ea00640 1 -- 172.21.15.121:0/1854441554 >> 172.21.15.121:0/1854441554 conn(0x7efe9809d6c0 msgr2=0x7efe980b9320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:33.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.958+0000 7f9751200640 1 -- 172.21.15.121:0/1135637045 wait complete. 2024-04-20T10:21:33.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.958+0000 7efe9ea00640 1 -- 172.21.15.121:0/1854441554 shutdown_connections 2024-04-20T10:21:33.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.958+0000 7efe9ea00640 1 -- 172.21.15.121:0/1854441554 wait complete. 2024-04-20T10:21:33.969 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.958+0000 7f0d55400640 1 -- 172.21.15.121:0/2959520242 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d50039070 con 0x7f0d580d46f0 2024-04-20T10:21:33.969 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.958+0000 7f8347800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83380078d0 0x7f8338052050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:33.969 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.958+0000 7f0d55400640 1 -- 172.21.15.121:0/2959520242 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d5002cca0 con 0x7f0d580d46f0 2024-04-20T10:21:33.969 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.958+0000 7f0d55400640 1 -- 172.21.15.121:0/2959520242 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d50034660 con 0x7f0d580d46f0 2024-04-20T10:21:33.970 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.958+0000 7f8347800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f833803e830 0x7f8338050a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:33.970 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.959+0000 7f8347800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f83380e30b0 0x7f8338050f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:33.970 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.959+0000 7f8347800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f83380542f0 con 0x7f833803e830 2024-04-20T10:21:33.970 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.959+0000 7f8347800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f83380e3490 con 0x7f83380078d0 2024-04-20T10:21:33.970 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.959+0000 7f8347800640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f8338054470 con 0x7f83380e30b0 2024-04-20T10:21:33.971 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.959+0000 7f0d5ca00640 1 -- 172.21.15.121:0/2959520242 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d580d46f0 msgr2=0x7f0d580d4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:33.971 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.959+0000 7f0d5ca00640 1 --2- 172.21.15.121:0/2959520242 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d580d46f0 0x7f0d580d4b80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0d5000b050 tx=0x7f0d5002c100 comp rx=0 tx=0).stop 2024-04-20T10:21:33.971 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.959+0000 7f8344e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83380078d0 0x7f8338052050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:33.972 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.959+0000 7f833fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f833803e830 0x7f8338050a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:33.972 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.960+0000 7f833fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f833803e830 0x7f8338050a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54242/0 (socket says 172.21.15.121:54242) 2024-04-20T10:21:33.972 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.960+0000 7f833fe00640 1 -- 172.21.15.121:0/1652080621 learned_addr learned my addr 172.21.15.121:0/1652080621 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:33.972 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.960+0000 7f8345800640 1 --2- 172.21.15.121:0/1652080621 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f83380e30b0 0x7f8338050f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:33.972 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.960+0000 7f833fe00640 1 -- 172.21.15.121:0/1652080621 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f83380e30b0 msgr2=0x7f8338050f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:33.972 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.960+0000 7f833fe00640 1 --2- 172.21.15.121:0/1652080621 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f83380e30b0 0x7f8338050f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:33.972 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.960+0000 7f833fe00640 1 -- 172.21.15.121:0/1652080621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83380078d0 msgr2=0x7f8338052050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:33.972 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.960+0000 7f833fe00640 1 --2- 172.21.15.121:0/1652080621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83380078d0 0x7f8338052050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:33.972 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.960+0000 7f833fe00640 1 -- 172.21.15.121:0/1652080621 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83380517d0 con 0x7f833803e830 2024-04-20T10:21:33.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.960+0000 7f0d5ca00640 1 -- 172.21.15.121:0/2959520242 shutdown_connections 2024-04-20T10:21:33.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.960+0000 7f0d5ca00640 1 --2- 172.21.15.121:0/2959520242 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d580d46f0 0x7f0d580d4b80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:33.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.960+0000 7f0d5ca00640 1 --2- 172.21.15.121:0/2959520242 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d5815e8c0 0x7f0d5806ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:33.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.960+0000 7f0d5ca00640 1 -- 172.21.15.121:0/2959520242 >> 172.21.15.121:0/2959520242 conn(0x7f0d5809d6f0 msgr2=0x7f0d580b9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:33.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.961+0000 7f0d5ca00640 1 -- 172.21.15.121:0/2959520242 shutdown_connections 2024-04-20T10:21:33.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.961+0000 7f833fe00640 1 --2- 172.21.15.121:0/1652080621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f833803e830 0x7f8338050a30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f832c00c450 tx=0x7f832c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:33.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.961+0000 7f0d5ca00640 1 -- 172.21.15.121:0/2959520242 wait complete. 2024-04-20T10:21:33.974 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.961+0000 7f833cc00640 1 -- 172.21.15.121:0/1652080621 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c0052d0 con 0x7f833803e830 2024-04-20T10:21:33.974 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.961+0000 7f8347800640 1 -- 172.21.15.121:0/1652080621 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83380d9d40 con 0x7f833803e830 2024-04-20T10:21:33.974 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.961+0000 7f8347800640 1 -- 172.21.15.121:0/1652080621 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83380514b0 con 0x7f833803e830 2024-04-20T10:21:33.974 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.962+0000 7efe9ea00640 1 Processor -- start 2024-04-20T10:21:33.975 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.962+0000 7f833cc00640 1 -- 172.21.15.121:0/1652080621 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f832c005470 con 0x7f833803e830 2024-04-20T10:21:33.975 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.962+0000 7f833cc00640 1 -- 172.21.15.121:0/1652080621 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c0052d0 con 0x7f833803e830 2024-04-20T10:21:33.975 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.962+0000 7efe9ea00640 1 -- start start 2024-04-20T10:21:33.975 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.962+0000 7f9751200640 1 Processor -- start 2024-04-20T10:21:33.975 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.963+0000 7f8332000640 1 -- 172.21.15.121:0/1652080621 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f831c001030 con 0x7f833803e830 2024-04-20T10:21:33.976 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.963+0000 7f0d5ca00640 1 Processor -- start 2024-04-20T10:21:33.976 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.962+0000 7f9751200640 1 -- start start 2024-04-20T10:21:33.976 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.964+0000 7f9751200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f974c06bdd0 0x7f974c1599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:33.976 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.964+0000 7f9751200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f974c06e510 0x7f974c168870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:33.976 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.964+0000 7f9751200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f974c0b3bf0 0x7f974c146920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:33.976 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.964+0000 7f9751200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f974c06baf0 con 0x7f974c06e510 2024-04-20T10:21:33.976 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.964+0000 7f9751200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f974c06b970 con 0x7f974c06bdd0 2024-04-20T10:21:33.977 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.964+0000 7f9751200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f974c0aefc0 con 0x7f974c0b3bf0 2024-04-20T10:21:33.977 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.964+0000 7efe9ea00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe980b38e0 0x7efe98154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:33.977 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.964+0000 7efe9ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe980d4930 0x7efe9806c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:33.977 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.964+0000 7efe9ea00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe9815e600 0x7efe9806dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:33.977 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.964+0000 7efe9ea00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efe980b62f0 con 0x7efe9815e600 2024-04-20T10:21:33.977 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.964+0000 7efe9ea00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efe9806eb40 con 0x7efe980d4930 2024-04-20T10:21:33.977 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.964+0000 7efe9ea00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7efe9815e9e0 con 0x7efe980b38e0 2024-04-20T10:21:33.977 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.964+0000 7efe9e000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe9815e600 0x7efe9806dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:33.977 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.965+0000 7efe9d600640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe980b38e0 0x7efe98154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:33.978 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.965+0000 7f9749e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f974c06bdd0 0x7f974c1599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:33.978 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.965+0000 7f9749400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f974c06e510 0x7f974c168870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:33.978 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.965+0000 7efe9d600640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe980b38e0 0x7efe98154180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.121:48798/0 (socket says 172.21.15.121:48798) 2024-04-20T10:21:33.979 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.965+0000 7efe9d600640 1 -- 172.21.15.121:0/2890459069 learned_addr learned my addr 172.21.15.121:0/2890459069 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:33.979 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.965+0000 7efe9cc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe980d4930 0x7efe9806c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:33.979 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.965+0000 7f9749400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f974c06e510 0x7f974c168870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54258/0 (socket says 172.21.15.121:54258) 2024-04-20T10:21:33.979 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.965+0000 7f9749400640 1 -- 172.21.15.121:0/1611060524 learned_addr learned my addr 172.21.15.121:0/1611060524 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:33.979 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.965+0000 7f0d5ca00640 1 -- start start 2024-04-20T10:21:33.979 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.966+0000 7f833cc00640 1 -- 172.21.15.121:0/1652080621 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f832c012630 con 0x7f833803e830 2024-04-20T10:21:33.980 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.966+0000 7efe9e000640 1 -- 172.21.15.121:0/2890459069 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe980b38e0 msgr2=0x7efe98154180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:33.980 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.966+0000 7efe9e000640 1 --2- 172.21.15.121:0/2890459069 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efe980b38e0 0x7efe98154180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:33.980 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.966+0000 7efe9e000640 1 -- 172.21.15.121:0/2890459069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe980d4930 msgr2=0x7efe9806c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:33.980 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.966+0000 7efe9e000640 1 --2- 172.21.15.121:0/2890459069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe980d4930 0x7efe9806c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:33.980 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.966+0000 7efe9e000640 1 -- 172.21.15.121:0/2890459069 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe981f4e60 con 0x7efe9815e600 2024-04-20T10:21:33.980 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.967+0000 7efe9e000640 1 --2- 172.21.15.121:0/2890459069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe9815e600 0x7efe9806dea0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efe8c00c390 tx=0x7efe8c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:33.980 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.966+0000 7f974a800640 1 --2- 172.21.15.121:0/1611060524 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f974c0b3bf0 0x7f974c146920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:33.981 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.966+0000 7f9749400640 1 -- 172.21.15.121:0/1611060524 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f974c0b3bf0 msgr2=0x7f974c146920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:33.981 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.966+0000 7f9749400640 1 --2- 172.21.15.121:0/1611060524 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f974c0b3bf0 0x7f974c146920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:33.981 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.966+0000 7f9749400640 1 -- 172.21.15.121:0/1611060524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f974c06bdd0 msgr2=0x7f974c1599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:33.981 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.966+0000 7f9749400640 1 --2- 172.21.15.121:0/1611060524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f974c06bdd0 0x7f974c1599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:33.981 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.966+0000 7f9749400640 1 -- 172.21.15.121:0/1611060524 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f974c0cfe20 con 0x7f974c06e510 2024-04-20T10:21:33.981 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.966+0000 7f974a800640 1 --2- 172.21.15.121:0/1611060524 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f974c0b3bf0 0x7f974c146920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:33.981 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.967+0000 7f9749400640 1 --2- 172.21.15.121:0/1611060524 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f974c06e510 0x7f974c168870 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f973c00c750 tx=0x7f973c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:33.981 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.967+0000 7f0d5ca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d580d1010 0x7f0d5806ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:33.981 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.967+0000 7f0d5ca00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d580d46f0 0x7f0d5806e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:33.981 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.967+0000 7f0d5ca00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d5815e8c0 0x7f0d5806e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:33.982 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.967+0000 7f0d5ca00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0d5806f180 con 0x7f0d5815e8c0 2024-04-20T10:21:33.982 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.967+0000 7f0d5ca00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0d580b5e00 con 0x7f0d580d1010 2024-04-20T10:21:33.982 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.967+0000 7f0d5ca00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f0d580cfa10 con 0x7f0d580d46f0 2024-04-20T10:21:33.982 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.966+0000 7f53f9800640 1 Processor -- start 2024-04-20T10:21:33.982 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.967+0000 7f9742000640 1 -- 172.21.15.121:0/1611060524 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c0050d0 con 0x7f974c06e510 2024-04-20T10:21:33.982 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.967+0000 7f9751200640 1 -- 172.21.15.121:0/1611060524 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f974c0d0100 con 0x7f974c06e510 2024-04-20T10:21:33.983 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.967+0000 7f9742000640 1 -- 172.21.15.121:0/1611060524 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f973c005270 con 0x7f974c06e510 2024-04-20T10:21:33.983 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.967+0000 7f0d56800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d580d1010 0x7f0d5806ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:33.983 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.967+0000 7f0d57200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d5815e8c0 0x7f0d5806e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:33.983 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.968+0000 7f9751200640 1 -- 172.21.15.121:0/1611060524 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f974c152580 con 0x7f974c06e510 2024-04-20T10:21:33.984 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.967+0000 7f0d55e00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d580d46f0 0x7f0d5806e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:33.984 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.968+0000 7f0d56800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d580d1010 0x7f0d5806ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.121:35264/0 (socket says 172.21.15.121:35264) 2024-04-20T10:21:33.984 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.968+0000 7f0d56800640 1 -- 172.21.15.121:0/999655314 learned_addr learned my addr 172.21.15.121:0/999655314 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:33.984 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.966+0000 7f53f9800640 1 -- start start 2024-04-20T10:21:33.985 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.968+0000 7f9742000640 1 -- 172.21.15.121:0/1611060524 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c0050d0 con 0x7f974c06e510 2024-04-20T10:21:33.985 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.967+0000 7f0d57200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d5815e8c0 0x7f0d5806e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54264/0 (socket says 172.21.15.121:54264) 2024-04-20T10:21:33.986 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.968+0000 7f53f9800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f53f40d1a00 0x7f53f40b1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:33.986 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.968+0000 7f53f9800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f53f40b17d0 0x7f53f40d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:33.986 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.968+0000 7f53f9800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53f40d1e90 con 0x7f53f40d4270 2024-04-20T10:21:33.986 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.968+0000 7f53f9800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f53f40d40c0 con 0x7f53f40d1a00 2024-04-20T10:21:33.986 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.968+0000 7f53f9800640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f53f40d3820 con 0x7f53f40b17d0 2024-04-20T10:21:33.987 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.969+0000 7f53f2a00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f53f40d4270 0x7f53f40d4650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:50036/0 (socket says 172.21.15.121:50036) 2024-04-20T10:21:33.987 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.969+0000 7f53f2a00640 1 -- 172.21.15.121:0/3981287690 learned_addr learned my addr 172.21.15.121:0/3981287690 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:33.987 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.968+0000 7efe95600640 1 -- 172.21.15.121:0/2890459069 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8c017070 con 0x7efe9815e600 2024-04-20T10:21:33.987 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.968+0000 7efe95600640 1 -- 172.21.15.121:0/2890459069 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe8c005ce0 con 0x7efe9815e600 2024-04-20T10:21:33.987 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.968+0000 7efe95600640 1 -- 172.21.15.121:0/2890459069 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8c0072f0 con 0x7efe9815e600 2024-04-20T10:21:33.987 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.968+0000 7efe9ea00640 1 -- 172.21.15.121:0/2890459069 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe9806bb50 con 0x7efe9815e600 2024-04-20T10:21:33.987 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.968+0000 7efe9ea00640 1 -- 172.21.15.121:0/2890459069 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe9806c110 con 0x7efe9815e600 2024-04-20T10:21:33.987 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.970+0000 7efe9ea00640 1 -- 172.21.15.121:0/2890459069 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efe9806b900 con 0x7efe9815e600 2024-04-20T10:21:33.987 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.969+0000 7f0d56800640 1 -- 172.21.15.121:0/999655314 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d580d46f0 msgr2=0x7f0d5806e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:33.988 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.969+0000 7f0d56800640 1 --2- 172.21.15.121:0/999655314 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0d580d46f0 0x7f0d5806e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:33.988 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.969+0000 7f0d56800640 1 -- 172.21.15.121:0/999655314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d5815e8c0 msgr2=0x7f0d5806e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:33.988 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.969+0000 7f0d56800640 1 --2- 172.21.15.121:0/999655314 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0d5815e8c0 0x7f0d5806e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:33.988 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.969+0000 7f0d56800640 1 -- 172.21.15.121:0/999655314 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d5806c000 con 0x7f0d580d1010 2024-04-20T10:21:33.988 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.969+0000 7f53f3400640 1 --2- 172.21.15.121:0/3981287690 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f53f40b17d0 0x7f53f40d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:33.988 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.970+0000 7f53f3400640 1 -- 172.21.15.121:0/3981287690 >> v2:172.21.15.121:3300/0 conn(0x7f53f40d1a00 msgr2=0x7f53f40b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:33.988 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.970+0000 7f53f3400640 1 --2- 172.21.15.121:0/3981287690 >> v2:172.21.15.121:3300/0 conn(0x7f53f40d1a00 0x7f53f40b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:33.988 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.970+0000 7f53f3400640 1 -- 172.21.15.121:0/3981287690 >> v1:172.21.15.121:6789/0 conn(0x7f53f40d4270 legacy=0x7f53f40d4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:33.988 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.970+0000 7f53f3400640 1 -- 172.21.15.121:0/3981287690 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f40d3b00 con 0x7f53f40b17d0 2024-04-20T10:21:33.989 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.970+0000 7f9751200640 1 -- 172.21.15.121:0/1611060524 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9724001030 con 0x7f974c06e510 2024-04-20T10:21:33.989 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.970+0000 7f0d56800640 1 --2- 172.21.15.121:0/999655314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d580d1010 0x7f0d5806ba20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0d4000e8a0 tx=0x7f0d4000ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:33.989 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.970+0000 7f5de7400640 1 Processor -- start 2024-04-20T10:21:33.990 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.971+0000 7f5de7400640 1 -- start start 2024-04-20T10:21:33.990 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.971+0000 7f0d4ea00640 1 -- 172.21.15.121:0/999655314 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d40017070 con 0x7f0d580d1010 2024-04-20T10:21:33.990 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.971+0000 7f0d4ea00640 1 -- 172.21.15.121:0/999655314 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d4000c940 con 0x7f0d580d1010 2024-04-20T10:21:33.990 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.972+0000 7efe95600640 1 -- 172.21.15.121:0/2890459069 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efe8c01a070 con 0x7efe9815e600 2024-04-20T10:21:33.991 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.971+0000 7f0d4ea00640 1 -- 172.21.15.121:0/999655314 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4000cd30 con 0x7f0d580d1010 2024-04-20T10:21:33.991 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.973+0000 7f0d5ca00640 1 -- 172.21.15.121:0/999655314 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d5806c2e0 con 0x7f0d580d1010 2024-04-20T10:21:33.991 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.973+0000 7f0d5ca00640 1 -- 172.21.15.121:0/999655314 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d5806c8a0 con 0x7f0d580d1010 2024-04-20T10:21:33.992 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.973+0000 7efe95600640 1 --2- 172.21.15.121:0/2890459069 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7efe7c06c940 0x7efe7c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:33.992 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.973+0000 7efe95600640 1 -- 172.21.15.121:0/2890459069 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7efe8c08afc0 con 0x7efe9815e600 2024-04-20T10:21:33.992 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.973+0000 7f53f3400640 1 --2- 172.21.15.121:0/3981287690 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f53f40b17d0 0x7f53f40d0380 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f53ec009d90 tx=0x7f53ec02be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e12a50a5bd8c718 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:33.992 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.973+0000 7f9742000640 1 -- 172.21.15.121:0/1611060524 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f973c00a640 con 0x7f974c06e510 2024-04-20T10:21:33.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.970+0000 7f833cc00640 1 --2- 172.21.15.121:0/1652080621 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f832006caf0 0x7f832006efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:33.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.971+0000 7f833cc00640 1 -- 172.21.15.121:0/1652080621 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f832c08b4d0 con 0x7f833803e830 2024-04-20T10:21:33.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.971+0000 7f8344e00640 1 --2- 172.21.15.121:0/1652080621 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f832006caf0 0x7f832006efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:33.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.971+0000 7f833cc00640 1 --2- 172.21.15.121:0/1652080621 >> [v2:172.21.15.121:6808/1044588835,v1:172.21.15.121:6809/1044588835] conn(0x7f83200761f0 0x7f8320078630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:33.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.972+0000 7f833cc00640 1 -- 172.21.15.121:0/1652080621 --> [v2:172.21.15.121:6808/1044588835,v1:172.21.15.121:6809/1044588835] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f832c00ee50 con 0x7f83200761f0 2024-04-20T10:21:33.994 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.972+0000 7f833cc00640 1 -- 172.21.15.121:0/1652080621 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f832c092050 con 0x7f833803e830 2024-04-20T10:21:33.994 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.972+0000 7f8345800640 1 --2- 172.21.15.121:0/1652080621 >> [v2:172.21.15.121:6808/1044588835,v1:172.21.15.121:6809/1044588835] conn(0x7f83200761f0 0x7f8320078630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:33.994 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.973+0000 7f8344e00640 1 --2- 172.21.15.121:0/1652080621 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f832006caf0 0x7f832006efb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f83340011e0 tx=0x7f8334007d30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:33.994 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.974+0000 7efe95600640 1 --2- 172.21.15.121:0/2890459069 >> [v2:172.21.15.121:6800/3505199624,v1:172.21.15.121:6801/3505199624] conn(0x7efe7c076030 0x7efe7c078470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:33.994 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.974+0000 7efe95600640 1 -- 172.21.15.121:0/2890459069 --> [v2:172.21.15.121:6800/3505199624,v1:172.21.15.121:6801/3505199624] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efe7c078b40 con 0x7efe7c076030 2024-04-20T10:21:33.994 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.974+0000 7efe95600640 1 -- 172.21.15.121:0/2890459069 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efe8c092050 con 0x7efe9815e600 2024-04-20T10:21:33.995 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.974+0000 7f8345800640 1 --2- 172.21.15.121:0/1652080621 >> [v2:172.21.15.121:6808/1044588835,v1:172.21.15.121:6809/1044588835] conn(0x7f83200761f0 0x7f8320078630 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:33.995 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.974+0000 7f53f1600640 1 -- 172.21.15.121:0/3981287690 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ec039070 con 0x7f53f40b17d0 2024-04-20T10:21:33.995 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.974+0000 7f53f1600640 1 -- 172.21.15.121:0/3981287690 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53ec02cbd0 con 0x7f53f40b17d0 2024-04-20T10:21:33.995 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.974+0000 7f53f1600640 1 -- 172.21.15.121:0/3981287690 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ec034440 con 0x7f53f40b17d0 2024-04-20T10:21:33.995 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.974+0000 7f5de7400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5de006c010 0x7f5de00b9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:33.995 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.974+0000 7f5de7400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5de006c5d0 con 0x7f5de0162e50 2024-04-20T10:21:33.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.974+0000 7f5de7400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5de0073ba0 con 0x7f5de00b9540 2024-04-20T10:21:33.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.974+0000 7f5de7400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f5de0073950 con 0x7f5de006c010 2024-04-20T10:21:33.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.974+0000 7f0d5ca00640 1 -- 172.21.15.121:0/999655314 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0d58075b30 con 0x7f0d580d1010 2024-04-20T10:21:33.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.974+0000 7efe9d600640 1 --2- 172.21.15.121:0/2890459069 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7efe7c06c940 0x7efe7c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:33.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.975+0000 7efe9cc00640 1 --2- 172.21.15.121:0/2890459069 >> [v2:172.21.15.121:6800/3505199624,v1:172.21.15.121:6801/3505199624] conn(0x7efe7c076030 0x7efe7c078470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:33.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.975+0000 7ff4d0400640 1 Processor -- start 2024-04-20T10:21:33.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.975+0000 7f53f9800640 1 -- 172.21.15.121:0/3981287690 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f53f40b17d0 msgr2=0x7f53f40d0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:33.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.975+0000 7f53f9800640 1 --2- 172.21.15.121:0/3981287690 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f53f40b17d0 0x7f53f40d0380 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f53ec009d90 tx=0x7f53ec02be20 comp rx=0 tx=0).stop 2024-04-20T10:21:33.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.976+0000 7efe9cc00640 1 --2- 172.21.15.121:0/2890459069 >> [v2:172.21.15.121:6800/3505199624,v1:172.21.15.121:6801/3505199624] conn(0x7efe7c076030 0x7efe7c078470 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:33.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.976+0000 7efe9d600640 1 --2- 172.21.15.121:0/2890459069 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7efe7c06c940 0x7efe7c06ee00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efe90000f80 tx=0x7efe90030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:33.998 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.975+0000 7f5de4a00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f5de0162e50 0x7f5de006b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:50044/0 (socket says 172.21.15.121:50044) 2024-04-20T10:21:33.998 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.975+0000 7f5de4a00640 1 -- 172.21.15.121:0/2824360603 learned_addr learned my addr 172.21.15.121:0/2824360603 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:33.998 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.976+0000 7f5dd7e00640 1 --2- 172.21.15.121:0/2824360603 >> v2:172.21.15.121:3300/0 conn(0x7f5de006c010 0x7f5de00b9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:33.998 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.976+0000 7f53f9800640 1 -- 172.21.15.121:0/3981287690 shutdown_connections 2024-04-20T10:21:33.998 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.976+0000 7f53f9800640 1 --2- 172.21.15.121:0/3981287690 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f53f40b17d0 0x7f53f40d0380 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:33.998 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.976+0000 7f53f9800640 1 --2- 172.21.15.121:0/3981287690 >> v2:172.21.15.121:3300/0 conn(0x7f53f40d1a00 0x7f53f40b1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:33.999 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.976+0000 7f53f9800640 1 -- 172.21.15.121:0/3981287690 >> 172.21.15.121:0/3981287690 conn(0x7f53f40b99c0 msgr2=0x7f53f40b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:33.999 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.976+0000 7f5ddfe00640 1 -- 172.21.15.121:0/2824360603 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3232313809 0 0) 0x7f5de006c5d0 con 0x7f5de0162e50 2024-04-20T10:21:33.999 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.976+0000 7f5ddfe00640 1 -- 172.21.15.121:0/2824360603 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5dc4003520 con 0x7f5de0162e50 2024-04-20T10:21:33.999 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.975+0000 7f9742000640 1 --2- 172.21.15.121:0/1611060524 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f972c06caf0 0x7f972c06efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:33.999 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.976+0000 7f9742000640 1 -- 172.21.15.121:0/1611060524 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f973c08b680 con 0x7f974c06e510 2024-04-20T10:21:33.999 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.976+0000 7f9742000640 1 --2- 172.21.15.121:0/1611060524 >> [v2:172.21.15.121:6816/3656724207,v1:172.21.15.121:6817/3656724207] conn(0x7f972c0761e0 0x7f972c078620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.000 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.976+0000 7f9742000640 1 -- 172.21.15.121:0/1611060524 --> [v2:172.21.15.121:6816/3656724207,v1:172.21.15.121:6817/3656724207] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f972c078cf0 con 0x7f972c0761e0 2024-04-20T10:21:34.000 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.976+0000 7f9742000640 1 -- 172.21.15.121:0/1611060524 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f973c092050 con 0x7f974c06e510 2024-04-20T10:21:34.000 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.977+0000 7f9749e00640 1 --2- 172.21.15.121:0/1611060524 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f972c06caf0 0x7f972c06efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.000 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.977+0000 7f0d4ea00640 1 -- 172.21.15.121:0/999655314 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0d4001a070 con 0x7f0d580d1010 2024-04-20T10:21:34.000 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.976+0000 7f53f9800640 1 -- 172.21.15.121:0/3981287690 shutdown_connections 2024-04-20T10:21:34.000 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.976+0000 7f53f9800640 1 -- 172.21.15.121:0/3981287690 wait complete. 2024-04-20T10:21:34.000 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.977+0000 7f5ddfe00640 1 -- 172.21.15.121:0/2824360603 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2885500460 0 0) 0x7f5dc4003520 con 0x7f5de0162e50 2024-04-20T10:21:34.001 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.977+0000 7f5dd7e00640 1 -- 172.21.15.121:0/2824360603 >> v1:172.21.15.123:6789/0 conn(0x7f5de00b9540 legacy=0x7f5de00b99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.001 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.977+0000 7f5dd7e00640 1 -- 172.21.15.121:0/2824360603 >> v1:172.21.15.121:6789/0 conn(0x7f5de0162e50 legacy=0x7f5de006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.001 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.977+0000 7f5dd7e00640 1 -- 172.21.15.121:0/2824360603 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de00be4d0 con 0x7f5de006c010 2024-04-20T10:21:34.001 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.978+0000 7f833cc00640 1 -- 172.21.15.121:0/1652080621 <== osd.0 v2:172.21.15.121:6808/1044588835 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f832c00ee50 con 0x7f83200761f0 2024-04-20T10:21:34.001 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.976+0000 7ff4d0400640 1 -- start start 2024-04-20T10:21:34.002 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.978+0000 7f90b1800640 1 Processor -- start 2024-04-20T10:21:34.002 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.978+0000 7efe95600640 1 -- 172.21.15.121:0/2890459069 <== osd.1 v2:172.21.15.121:6800/3505199624 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efe7c078b40 con 0x7efe7c076030 2024-04-20T10:21:34.003 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.978+0000 7f0d4ea00640 1 --2- 172.21.15.121:0/999655314 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f0d3406c940 0x7f0d3406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.003 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.978+0000 7f9749e00640 1 --2- 172.21.15.121:0/1611060524 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f972c06caf0 0x7f972c06efb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9744005c00 tx=0x7f9744009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.005 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.978+0000 7f0d4ea00640 1 -- 172.21.15.121:0/999655314 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0d4008c3e0 con 0x7f0d580d1010 2024-04-20T10:21:34.005 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.978+0000 7f0d55e00640 1 --2- 172.21.15.121:0/999655314 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f0d3406c940 0x7f0d3406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.005 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.979+0000 7f974a800640 1 --2- 172.21.15.121:0/1611060524 >> [v2:172.21.15.121:6816/3656724207,v1:172.21.15.121:6817/3656724207] conn(0x7f972c0761e0 0x7f972c078620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.979+0000 7f0d4ea00640 1 --2- 172.21.15.121:0/999655314 >> [v2:172.21.15.121:6821/2232342832,v1:172.21.15.121:6823/2232342832] conn(0x7f0d340760c0 0x7f0d34078500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.979+0000 7f0d4ea00640 1 -- 172.21.15.121:0/999655314 --> [v2:172.21.15.121:6821/2232342832,v1:172.21.15.121:6823/2232342832] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0d34078bd0 con 0x7f0d340760c0 2024-04-20T10:21:34.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.979+0000 7f0d4ea00640 1 -- 172.21.15.121:0/999655314 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0d4008c790 con 0x7f0d580d1010 2024-04-20T10:21:34.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.979+0000 7f0d57200640 1 --2- 172.21.15.121:0/999655314 >> [v2:172.21.15.121:6821/2232342832,v1:172.21.15.121:6823/2232342832] conn(0x7f0d340760c0 0x7f0d34078500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.979+0000 7f974a800640 1 --2- 172.21.15.121:0/1611060524 >> [v2:172.21.15.121:6816/3656724207,v1:172.21.15.121:6817/3656724207] conn(0x7f972c0761e0 0x7f972c078620 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.007 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.979+0000 7f53f9800640 1 Processor -- start 2024-04-20T10:21:34.007 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.980+0000 7f5dd7e00640 1 --2- 172.21.15.121:0/2824360603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5de006c010 0x7f5de00b9000 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5dd00097a0 tx=0x7f5dd002b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e70fef33f21c4c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.007 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.978+0000 7f90b1800640 1 -- start start 2024-04-20T10:21:34.007 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.980+0000 7f90b1800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f90ac0d1a00 0x7f90ac0b1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.008 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.980+0000 7ff4d0400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff4c80d1a00 0x7ff4c80b1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.008 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.980+0000 7ff4d0400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff4c80b17d0 0x7ff4c80d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.008 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.980+0000 7ff4d0400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4c80d1e90 con 0x7ff4c80d4270 2024-04-20T10:21:34.008 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.980+0000 7ff4d0400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7ff4c80d40c0 con 0x7ff4c80b17d0 2024-04-20T10:21:34.008 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.980+0000 7ff4d0400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ff4c80d3820 con 0x7ff4c80d1a00 2024-04-20T10:21:34.008 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.980+0000 7f90b1800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90ac0d1e90 con 0x7f90ac0d4270 2024-04-20T10:21:34.008 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.980+0000 7f90b1800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90ac0d3820 con 0x7f90ac0b17d0 2024-04-20T10:21:34.009 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.980+0000 7f90b1800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f90ac0d40c0 con 0x7f90ac0d1a00 2024-04-20T10:21:34.009 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.980+0000 7f0d57200640 1 --2- 172.21.15.121:0/999655314 >> [v2:172.21.15.121:6821/2232342832,v1:172.21.15.121:6823/2232342832] conn(0x7f0d340760c0 0x7f0d34078500 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.009 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.980+0000 7f0d55e00640 1 --2- 172.21.15.121:0/999655314 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f0d3406c940 0x7f0d3406ee00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0d580b40c0 tx=0x7f0d48009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.010 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.981+0000 7f90aa000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f90ac0d1a00 0x7f90ac0b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.010 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.981+0000 7f90aa000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f90ac0d1a00 0x7f90ac0b1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54302/0 (socket says 172.21.15.121:54302) 2024-04-20T10:21:34.010 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.981+0000 7f90aa000640 1 -- 172.21.15.121:0/317078537 learned_addr learned my addr 172.21.15.121:0/317078537 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:34.010 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.981+0000 7ff4ce400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff4c80b17d0 0x7ff4c80d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.010 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.981+0000 7f9742000640 1 -- 172.21.15.121:0/1611060524 <== osd.2 v2:172.21.15.121:6816/3656724207 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f972c078cf0 con 0x7f972c0761e0 2024-04-20T10:21:34.011 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.981+0000 7ff4cda00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7ff4c80d4270 0x7ff4c80d4650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:50060/0 (socket says 172.21.15.121:50060) 2024-04-20T10:21:34.011 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.981+0000 7ff4ce400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff4c80b17d0 0x7ff4c80d0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54310/0 (socket says 172.21.15.121:54310) 2024-04-20T10:21:34.011 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.981+0000 7ff4ce400640 1 -- 172.21.15.121:0/82238572 learned_addr learned my addr 172.21.15.121:0/82238572 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:34.011 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.981+0000 7f53f9800640 1 -- start start 2024-04-20T10:21:34.012 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.982+0000 7f5ddfe00640 1 -- 172.21.15.121:0/2824360603 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0037070 con 0x7f5de006c010 2024-04-20T10:21:34.012 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.982+0000 7f5ddfe00640 1 -- 172.21.15.121:0/2824360603 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dd0030510 con 0x7f5de006c010 2024-04-20T10:21:34.012 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.982+0000 7f5ddfe00640 1 -- 172.21.15.121:0/2824360603 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc8001670 con 0x7f5de006c010 2024-04-20T10:21:34.012 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.982+0000 7f90a9600640 1 -- 172.21.15.121:0/317078537 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 911964630 0 0) 0x7f90a4001490 con 0x7f90ac0d4270 2024-04-20T10:21:34.012 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.982+0000 7f90a9600640 1 -- 172.21.15.121:0/317078537 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9090003520 con 0x7f90ac0d4270 2024-04-20T10:21:34.012 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.982+0000 7f90a9600640 1 -- 172.21.15.121:0/317078537 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1215283479 0 0) 0x7f90ac0d3820 con 0x7f90ac0b17d0 2024-04-20T10:21:34.012 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.982+0000 7f90a9600640 1 -- 172.21.15.121:0/317078537 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f90a4001490 con 0x7f90ac0b17d0 2024-04-20T10:21:34.013 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.982+0000 7f53f9800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f53f40b17d0 0x7f53f40ce010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.013 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.982+0000 7f53f9800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53f40d1a00 0x7f53f41fc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.013 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.982+0000 7f90aa000640 1 -- 172.21.15.121:0/317078537 >> v1:172.21.15.123:6789/0 conn(0x7f90ac0b17d0 legacy=0x7f90ac0d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.013 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.982+0000 7f53f9800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53f40d4270 0x7f53f40ca190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.014 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.982+0000 7f0d4ea00640 1 -- 172.21.15.121:0/999655314 <== osd.3 v2:172.21.15.121:6821/2232342832 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0d34078bd0 con 0x7f0d340760c0 2024-04-20T10:21:34.014 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.983+0000 7f5de7400640 1 -- 172.21.15.121:0/2824360603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5de006c010 msgr2=0x7f5de00b9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.014 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.983+0000 7f5de7400640 1 --2- 172.21.15.121:0/2824360603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5de006c010 0x7f5de00b9000 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5dd00097a0 tx=0x7f5dd002b790 comp rx=0 tx=0).stop 2024-04-20T10:21:34.014 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.982+0000 7ff4ce400640 1 -- 172.21.15.121:0/82238572 >> v2:172.21.15.123:3300/0 conn(0x7ff4c80d1a00 msgr2=0x7ff4c80b1290 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T10:21:34.015 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.982+0000 7ff4ce400640 1 --2- 172.21.15.121:0/82238572 >> v2:172.21.15.123:3300/0 conn(0x7ff4c80d1a00 0x7ff4c80b1290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.015 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.982+0000 7ff4ce400640 1 -- 172.21.15.121:0/82238572 >> v1:172.21.15.121:6789/0 conn(0x7ff4c80d4270 legacy=0x7ff4c80d4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.015 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.982+0000 7ff4ce400640 1 -- 172.21.15.121:0/82238572 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c80d3b00 con 0x7ff4c80b17d0 2024-04-20T10:21:34.015 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.984+0000 7f5de7400640 1 -- 172.21.15.121:0/2824360603 shutdown_connections 2024-04-20T10:21:34.015 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.984+0000 7f5de7400640 1 --2- 172.21.15.121:0/2824360603 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5de006c010 0x7f5de00b9000 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.015 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.984+0000 7f5de7400640 1 -- 172.21.15.121:0/2824360603 >> 172.21.15.121:0/2824360603 conn(0x7f5de00ba580 msgr2=0x7f5de00ba990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:34.016 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.982+0000 7f90aa000640 1 -- 172.21.15.121:0/317078537 >> v1:172.21.15.121:6789/0 conn(0x7f90ac0d4270 legacy=0x7f90ac0d4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.016 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.982+0000 7f90aa000640 1 -- 172.21.15.121:0/317078537 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90ac0d3b00 con 0x7f90ac0d1a00 2024-04-20T10:21:34.016 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.984+0000 7f90aa000640 1 --2- 172.21.15.121:0/317078537 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90ac0d1a00 0x7f90ac0b1290 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f909c009870 tx=0x7f909c02ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d6856b824af6c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.016 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.984+0000 7f90a9600640 1 -- 172.21.15.121:0/317078537 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909c02f840 con 0x7f90ac0d1a00 2024-04-20T10:21:34.016 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.984+0000 7f90a9600640 1 -- 172.21.15.121:0/317078537 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f909c02f9e0 con 0x7f90ac0d1a00 2024-04-20T10:21:34.016 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.984+0000 7f90a9600640 1 -- 172.21.15.121:0/317078537 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909c02fd90 con 0x7f90ac0d1a00 2024-04-20T10:21:34.017 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.982+0000 7f53f9800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f53f40d53b0 con 0x7f53f40d4270 2024-04-20T10:21:34.017 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.982+0000 7f53f9800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f53f40cfe60 con 0x7f53f40d1a00 2024-04-20T10:21:34.017 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.984+0000 7ff4ce400640 1 --2- 172.21.15.121:0/82238572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4c80b17d0 0x7ff4c80d0380 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff4c80b8060 tx=0x7ff4c002bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e00d0aa970432dc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.017 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.982+0000 7f53f9800640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f53f4161f70 con 0x7f53f40b17d0 2024-04-20T10:21:34.018 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.985+0000 7ff4c7e00640 1 -- 172.21.15.121:0/82238572 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c002cb30 con 0x7ff4c80b17d0 2024-04-20T10:21:34.018 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.985+0000 7ff4c7e00640 1 -- 172.21.15.121:0/82238572 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4c002ccd0 con 0x7ff4c80b17d0 2024-04-20T10:21:34.018 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.985+0000 7ff4c7e00640 1 -- 172.21.15.121:0/82238572 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c0034610 con 0x7ff4c80b17d0 2024-04-20T10:21:34.019 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.983+0000 7f53f2a00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f53f40b17d0 0x7f53f40ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.019 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.983+0000 7f53f2a00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f53f40b17d0 0x7f53f40ce010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.121:48842/0 (socket says 172.21.15.121:48842) 2024-04-20T10:21:34.019 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.983+0000 7f53f2a00640 1 -- 172.21.15.121:0/2890674187 learned_addr learned my addr 172.21.15.121:0/2890674187 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:34.019 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.984+0000 7f53f2a00640 1 -- 172.21.15.121:0/2890674187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53f40d1a00 msgr2=0x7f53f41fc300 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-20T10:21:34.019 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.984+0000 7f53f2a00640 1 --2- 172.21.15.121:0/2890674187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53f40d1a00 0x7f53f41fc300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.019 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.985+0000 7f90b1800640 1 -- 172.21.15.121:0/317078537 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90ac0d1a00 msgr2=0x7f90ac0b1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.019 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.985+0000 7f90b1800640 1 --2- 172.21.15.121:0/317078537 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90ac0d1a00 0x7f90ac0b1290 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f909c009870 tx=0x7f909c02ba20 comp rx=0 tx=0).stop 2024-04-20T10:21:34.020 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.984+0000 7f53f2a00640 1 -- 172.21.15.121:0/2890674187 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53f40d4270 msgr2=0x7f53f40ca190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-20T10:21:34.020 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.984+0000 7f53f2a00640 1 --2- 172.21.15.121:0/2890674187 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f53f40d4270 0x7f53f40ca190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.020 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.986+0000 7f5de7400640 1 -- 172.21.15.121:0/2824360603 shutdown_connections 2024-04-20T10:21:34.020 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.986+0000 7f5de7400640 1 -- 172.21.15.121:0/2824360603 wait complete. 2024-04-20T10:21:34.021 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.984+0000 7f53f2a00640 1 -- 172.21.15.121:0/2890674187 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f40ce550 con 0x7f53f40b17d0 2024-04-20T10:21:34.021 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.984+0000 7f53f2a00640 1 --2- 172.21.15.121:0/2890674187 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f53f40b17d0 0x7f53f40ce010 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f53e400c460 tx=0x7f53e400c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.021 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.985+0000 7f53eaa00640 1 -- 172.21.15.121:0/2890674187 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e4005ce0 con 0x7f53f40b17d0 2024-04-20T10:21:34.021 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.985+0000 7f53eaa00640 1 -- 172.21.15.121:0/2890674187 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53e4007070 con 0x7f53f40b17d0 2024-04-20T10:21:34.021 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.985+0000 7f53eaa00640 1 -- 172.21.15.121:0/2890674187 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e4007460 con 0x7f53f40b17d0 2024-04-20T10:21:34.021 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.986+0000 7f53f9800640 1 -- 172.21.15.121:0/2890674187 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f406eab0 con 0x7f53f40b17d0 2024-04-20T10:21:34.021 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.986+0000 7f53f9800640 1 -- 172.21.15.121:0/2890674187 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f406d8d0 con 0x7f53f40b17d0 2024-04-20T10:21:34.022 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.987+0000 7f90b1800640 1 -- 172.21.15.121:0/317078537 shutdown_connections 2024-04-20T10:21:34.022 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.987+0000 7f90b1800640 1 --2- 172.21.15.121:0/317078537 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90ac0d1a00 0x7f90ac0b1290 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.022 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.987+0000 7f90b1800640 1 -- 172.21.15.121:0/317078537 >> 172.21.15.121:0/317078537 conn(0x7f90ac0b99c0 msgr2=0x7f90ac0b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:34.022 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.987+0000 7f53f9800640 1 -- 172.21.15.121:0/2890674187 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f53f4068980 con 0x7f53f40b17d0 2024-04-20T10:21:34.022 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.987+0000 7f90b1800640 1 -- 172.21.15.121:0/317078537 shutdown_connections 2024-04-20T10:21:34.023 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.987+0000 7f90b1800640 1 -- 172.21.15.121:0/317078537 wait complete. 2024-04-20T10:21:34.023 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.987+0000 7ff4d0400640 1 -- 172.21.15.121:0/82238572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4c80b17d0 msgr2=0x7ff4c80d0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.024 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.987+0000 7ff4d0400640 1 --2- 172.21.15.121:0/82238572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4c80b17d0 0x7ff4c80d0380 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff4c80b8060 tx=0x7ff4c002bf90 comp rx=0 tx=0).stop 2024-04-20T10:21:34.024 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.987+0000 7ff4d0400640 1 -- 172.21.15.121:0/82238572 shutdown_connections 2024-04-20T10:21:34.024 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.987+0000 7ff4d0400640 1 --2- 172.21.15.121:0/82238572 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4c80b17d0 0x7ff4c80d0380 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.024 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.987+0000 7ff4d0400640 1 --2- 172.21.15.121:0/82238572 >> v2:172.21.15.123:3300/0 conn(0x7ff4c80d1a00 0x7ff4c80b1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.024 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.988+0000 7ff4d0400640 1 -- 172.21.15.121:0/82238572 >> 172.21.15.121:0/82238572 conn(0x7ff4c80b99c0 msgr2=0x7ff4c80b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:34.024 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.988+0000 7ff4d0400640 1 -- 172.21.15.121:0/82238572 shutdown_connections 2024-04-20T10:21:34.024 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.988+0000 7f5de7400640 1 Processor -- start 2024-04-20T10:21:34.025 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.988+0000 7ff4d0400640 1 -- 172.21.15.121:0/82238572 wait complete. 2024-04-20T10:21:34.025 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.990+0000 7f90b1800640 1 Processor -- start 2024-04-20T10:21:34.025 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.990+0000 7f53eaa00640 1 -- 172.21.15.121:0/2890674187 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f53e4007600 con 0x7f53f40b17d0 2024-04-20T10:21:34.025 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.991+0000 7f53eaa00640 1 --2- 172.21.15.121:0/2890674187 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f53d006c940 0x7f53d006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.025 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.991+0000 7f53eaa00640 1 -- 172.21.15.121:0/2890674187 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f53e408c400 con 0x7f53f40b17d0 2024-04-20T10:21:34.026 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.991+0000 7ff4d0400640 1 Processor -- start 2024-04-20T10:21:34.026 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.991+0000 7f53eaa00640 1 --2- 172.21.15.121:0/2890674187 >> [v2:172.21.15.123:6800/99353111,v1:172.21.15.123:6801/99353111] conn(0x7f53d00760c0 0x7f53d0078500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.026 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.991+0000 7f53eaa00640 1 -- 172.21.15.121:0/2890674187 --> [v2:172.21.15.123:6800/99353111,v1:172.21.15.123:6801/99353111] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f53d0078bd0 con 0x7f53d00760c0 2024-04-20T10:21:34.026 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.991+0000 7f53eaa00640 1 -- 172.21.15.121:0/2890674187 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f53e408c8e0 con 0x7f53f40b17d0 2024-04-20T10:21:34.027 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.991+0000 7ff4d0400640 1 -- start start 2024-04-20T10:21:34.027 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.992+0000 7f53f2000640 1 --2- 172.21.15.121:0/2890674187 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f53d006c940 0x7f53d006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.027 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.990+0000 7f5de7400640 1 -- start start 2024-04-20T10:21:34.027 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.993+0000 7f90b1800640 1 -- start start 2024-04-20T10:21:34.028 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.993+0000 7ff4d0400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4c80b17d0 0x7ff4c806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.028 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.993+0000 7ff4d0400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4c80d1a00 0x7ff4c80cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.028 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.993+0000 7ff4d0400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4c80d4270 0x7ff4c815b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.028 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.993+0000 7ff4d0400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c80d53b0 con 0x7ff4c80d4270 2024-04-20T10:21:34.028 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.993+0000 7ff4d0400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c80cfe60 con 0x7ff4c80b17d0 2024-04-20T10:21:34.028 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.993+0000 7ff4d0400640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ff4c8161f70 con 0x7ff4c80d1a00 2024-04-20T10:21:34.028 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.993+0000 7f53f3400640 1 --2- 172.21.15.121:0/2890674187 >> [v2:172.21.15.123:6800/99353111,v1:172.21.15.123:6801/99353111] conn(0x7f53d00760c0 0x7f53d0078500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.029 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.994+0000 7f90b1800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90ac0b17d0 0x7f90ac0cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.029 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.994+0000 7f90b1800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90ac0d1a00 0x7f90ac15bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.029 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.994+0000 7f90b1800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90ac0d4270 0x7f90ac06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.029 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.994+0000 7f90b1800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac0d53b0 con 0x7f90ac0b17d0 2024-04-20T10:21:34.029 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.994+0000 7f90b1800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac0cfe60 con 0x7f90ac0d1a00 2024-04-20T10:21:34.030 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.994+0000 7f90b1800640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f90ac161f70 con 0x7f90ac0d4270 2024-04-20T10:21:34.030 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.994+0000 7f5de7400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5de006c010 0x7f5de00b6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.030 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.994+0000 7f5de7400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5de00b9540 0x7f5de0159220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.030 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.994+0000 7ff4cda00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4c80b17d0 0x7ff4c806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.030 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.994+0000 7ff4cd000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4c80d1a00 0x7ff4c80cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.031 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.994+0000 7f90aa000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90ac0d1a00 0x7f90ac15bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.031 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.994+0000 7f5de7400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5de0162e50 0x7f5de0161610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.031 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.994+0000 7f90aa000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90ac0d1a00 0x7f90ac15bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.121:35288/0 (socket says 172.21.15.121:35288) 2024-04-20T10:21:34.031 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.994+0000 7f90aa000640 1 -- 172.21.15.121:0/1191770851 learned_addr learned my addr 172.21.15.121:0/1191770851 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:34.032 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.995+0000 7f5de4a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5de006c010 0x7f5de00b6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.032 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.995+0000 7f5de4a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5de006c010 0x7f5de00b6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54340/0 (socket says 172.21.15.121:54340) 2024-04-20T10:21:34.032 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.995+0000 7f90aaa00640 1 --2- 172.21.15.121:0/1191770851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90ac0b17d0 0x7f90ac0cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.032 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.994+0000 7ff4ce400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4c80d4270 0x7ff4c815b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.033 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.994+0000 7ff4ce400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4c80d4270 0x7ff4c815b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54312/0 (socket says 172.21.15.121:54312) 2024-04-20T10:21:34.033 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.994+0000 7ff4ce400640 1 -- 172.21.15.121:0/4112201040 learned_addr learned my addr 172.21.15.121:0/4112201040 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:34.033 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.995+0000 7f5de4a00640 1 -- 172.21.15.121:0/2390395801 learned_addr learned my addr 172.21.15.121:0/2390395801 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:34.033 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.995+0000 7f5de7400640 1 -- 172.21.15.121:0/2390395801 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5de0163230 con 0x7f5de006c010 2024-04-20T10:21:34.033 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.995+0000 7f5de5400640 1 --2- 172.21.15.121:0/2390395801 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5de0162e50 0x7f5de0161610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.033 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.995+0000 7f53f2000640 1 --2- 172.21.15.121:0/2890674187 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f53d006c940 0x7f53d006ee00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f53f40c7a50 tx=0x7f53dc006c20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.034 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.995+0000 7f5de7400640 1 -- 172.21.15.121:0/2390395801 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5de00d2cf0 con 0x7f5de00b9540 2024-04-20T10:21:34.034 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.995+0000 7f90aaa00640 1 -- 172.21.15.121:0/1191770851 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90ac0d4270 msgr2=0x7f90ac06d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.034 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.995+0000 7f90aaa00640 1 --2- 172.21.15.121:0/1191770851 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f90ac0d4270 0x7f90ac06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.034 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.995+0000 7f90aaa00640 1 -- 172.21.15.121:0/1191770851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90ac0d1a00 msgr2=0x7f90ac15bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.034 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.995+0000 7f90aaa00640 1 --2- 172.21.15.121:0/1191770851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90ac0d1a00 0x7f90ac15bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.035 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.995+0000 7f90aaa00640 1 -- 172.21.15.121:0/1191770851 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90ac0cd860 con 0x7f90ac0b17d0 2024-04-20T10:21:34.035 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.996+0000 7f90aaa00640 1 --2- 172.21.15.121:0/1191770851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90ac0b17d0 0x7f90ac0cc0c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f90a400ca70 tx=0x7f90a400cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.035 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.995+0000 7ff4cda00640 1 -- 172.21.15.121:0/4112201040 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4c80d1a00 msgr2=0x7ff4c80cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.035 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.996+0000 7ff4cda00640 1 --2- 172.21.15.121:0/4112201040 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4c80d1a00 0x7ff4c80cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.035 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.996+0000 7ff4cda00640 1 -- 172.21.15.121:0/4112201040 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4c80d4270 msgr2=0x7ff4c815b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.036 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.996+0000 7f90a2a00640 1 -- 172.21.15.121:0/1191770851 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a4003b70 con 0x7f90ac0b17d0 2024-04-20T10:21:34.036 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.996+0000 7f53f3400640 1 --2- 172.21.15.121:0/2890674187 >> [v2:172.21.15.123:6800/99353111,v1:172.21.15.123:6801/99353111] conn(0x7f53d00760c0 0x7f53d0078500 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.036 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.996+0000 7ff4cda00640 1 --2- 172.21.15.121:0/4112201040 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4c80d4270 0x7ff4c815b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.036 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.996+0000 7ff4cda00640 1 -- 172.21.15.121:0/4112201040 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c80cdec0 con 0x7ff4c80b17d0 2024-04-20T10:21:34.037 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.996+0000 7f5dd7e00640 1 --2- 172.21.15.121:0/2390395801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5de00b9540 0x7f5de0159220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.037 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.996+0000 7ff4cd000640 1 --2- 172.21.15.121:0/4112201040 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff4c80d1a00 0x7ff4c80cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:21:34.037 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.997+0000 7f5de7400640 1 -- 172.21.15.121:0/2390395801 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f5de00721d0 con 0x7f5de0162e50 2024-04-20T10:21:34.038 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.996+0000 7ff4cda00640 1 --2- 172.21.15.121:0/4112201040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4c80b17d0 0x7ff4c806e320 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00c3c0 tx=0x7ff4bc00c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.038 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.997+0000 7ff4c5600640 1 -- 172.21.15.121:0/4112201040 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc0074e0 con 0x7ff4c80b17d0 2024-04-20T10:21:34.038 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.997+0000 7ff4c5600640 1 -- 172.21.15.121:0/4112201040 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4bc007680 con 0x7ff4c80b17d0 2024-04-20T10:21:34.038 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.997+0000 7ff4c5600640 1 -- 172.21.15.121:0/4112201040 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc0049e0 con 0x7ff4c80b17d0 2024-04-20T10:21:34.039 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.997+0000 7f5de5400640 1 -- 172.21.15.121:0/2390395801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5de00b9540 msgr2=0x7f5de0159220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.039 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.997+0000 7f5de5400640 1 --2- 172.21.15.121:0/2390395801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5de00b9540 0x7f5de0159220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.039 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.997+0000 7f5de5400640 1 -- 172.21.15.121:0/2390395801 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5de006c010 msgr2=0x7f5de00b6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.039 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.997+0000 7f5de5400640 1 --2- 172.21.15.121:0/2390395801 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5de006c010 0x7f5de00b6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.039 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.998+0000 7f5de5400640 1 -- 172.21.15.121:0/2390395801 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de00b7e70 con 0x7f5de0162e50 2024-04-20T10:21:34.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.998+0000 7ff4d0400640 1 -- 172.21.15.121:0/4112201040 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4c80ce1a0 con 0x7ff4c80b17d0 2024-04-20T10:21:34.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.998+0000 7ff4d0400640 1 -- 172.21.15.121:0/4112201040 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4c806c2e0 con 0x7ff4c80b17d0 2024-04-20T10:21:34.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.999+0000 7ff4d0400640 1 -- 172.21.15.121:0/4112201040 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff4c8068980 con 0x7ff4c80b17d0 2024-04-20T10:21:34.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.996+0000 7f90b1800640 1 -- 172.21.15.121:0/1191770851 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90ac0cdb40 con 0x7f90ac0b17d0 2024-04-20T10:21:34.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.996+0000 7f90b1800640 1 -- 172.21.15.121:0/1191770851 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90ac0ce100 con 0x7f90ac0b17d0 2024-04-20T10:21:34.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.996+0000 7f90a2a00640 1 -- 172.21.15.121:0/1191770851 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90a4003d10 con 0x7f90ac0b17d0 2024-04-20T10:21:34.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.996+0000 7f90a2a00640 1 -- 172.21.15.121:0/1191770851 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a400a9f0 con 0x7f90ac0b17d0 2024-04-20T10:21:34.041 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:33.997+0000 7f90b1800640 1 -- 172.21.15.121:0/1191770851 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9084001030 con 0x7f90ac0b17d0 2024-04-20T10:21:34.041 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.000+0000 7f5de5400640 1 --2- 172.21.15.121:0/2390395801 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5de0162e50 0x7f5de0161610 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800c730 tx=0x7f5dd800cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.041 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.000+0000 7f53eaa00640 1 -- 172.21.15.121:0/2890674187 <== osd.4 v2:172.21.15.123:6800/99353111 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f53d0078bd0 con 0x7f53d00760c0 2024-04-20T10:21:34.041 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.000+0000 7f90a2a00640 1 -- 172.21.15.121:0/1191770851 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f90a400ab90 con 0x7f90ac0b17d0 2024-04-20T10:21:34.042 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.000+0000 7f5ddd600640 1 -- 172.21.15.121:0/2390395801 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8003b70 con 0x7f5de0162e50 2024-04-20T10:21:34.042 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.000+0000 7f5ddd600640 1 -- 172.21.15.121:0/2390395801 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5dd8003d10 con 0x7f5de0162e50 2024-04-20T10:21:34.042 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.000+0000 7f5ddd600640 1 -- 172.21.15.121:0/2390395801 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8005310 con 0x7f5de0162e50 2024-04-20T10:21:34.042 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.000+0000 7f5de7400640 1 -- 172.21.15.121:0/2390395801 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5de00b8150 con 0x7f5de0162e50 2024-04-20T10:21:34.042 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.001+0000 7f5de7400640 1 -- 172.21.15.121:0/2390395801 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5de00b7580 con 0x7f5de0162e50 2024-04-20T10:21:34.043 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.001+0000 7f90a2a00640 1 --2- 172.21.15.121:0/1191770851 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f908c06c8f0 0x7f908c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.043 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.001+0000 7f90a2a00640 1 -- 172.21.15.121:0/1191770851 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f90a408c980 con 0x7f90ac0b17d0 2024-04-20T10:21:34.043 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.001+0000 7f90a2a00640 1 --2- 172.21.15.121:0/1191770851 >> [v2:172.21.15.123:6817/4139299599,v1:172.21.15.123:6819/4139299599] conn(0x7f908c075ff0 0x7f908c078430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.043 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.002+0000 7f90a2a00640 1 -- 172.21.15.121:0/1191770851 --> [v2:172.21.15.123:6817/4139299599,v1:172.21.15.123:6819/4139299599] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f90a401aec0 con 0x7f908c075ff0 2024-04-20T10:21:34.043 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.002+0000 7f90a2a00640 1 -- 172.21.15.121:0/1191770851 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f90a408ce10 con 0x7f90ac0b17d0 2024-04-20T10:21:34.043 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.002+0000 7f90aa000640 1 --2- 172.21.15.121:0/1191770851 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f908c06c8f0 0x7f908c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.043 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.003+0000 7ff4c5600640 1 -- 172.21.15.121:0/4112201040 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff4bc004b80 con 0x7ff4c80b17d0 2024-04-20T10:21:34.044 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.003+0000 7f5de7400640 1 -- 172.21.15.121:0/2390395801 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5de00cfd70 con 0x7f5de0162e50 2024-04-20T10:21:34.044 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.002+0000 7f90aa000640 1 --2- 172.21.15.121:0/1191770851 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f908c06c8f0 0x7f908c06edb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f909c009870 tx=0x7f909c039040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.044 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.004+0000 7ff4c5600640 1 --2- 172.21.15.121:0/4112201040 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff4a806c8f0 0x7ff4a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.044 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.004+0000 7ff4c5600640 1 -- 172.21.15.121:0/4112201040 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff4bc08be40 con 0x7ff4c80b17d0 2024-04-20T10:21:34.045 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.004+0000 7ff4c5600640 1 --2- 172.21.15.121:0/4112201040 >> [v2:172.21.15.123:6808/4043981069,v1:172.21.15.123:6809/4043981069] conn(0x7ff4a8075fe0 0x7ff4a8078420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.045 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.004+0000 7ff4cd000640 1 --2- 172.21.15.121:0/4112201040 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff4a806c8f0 0x7ff4a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.045 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.004+0000 7ff4c5600640 1 -- 172.21.15.121:0/4112201040 --> [v2:172.21.15.123:6808/4043981069,v1:172.21.15.123:6809/4043981069] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff4a8078af0 con 0x7ff4a8075fe0 2024-04-20T10:21:34.045 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.004+0000 7ff4c5600640 1 -- 172.21.15.121:0/4112201040 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff4bc08c2d0 con 0x7ff4c80b17d0 2024-04-20T10:21:34.045 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.004+0000 7f90ab400640 1 --2- 172.21.15.121:0/1191770851 >> [v2:172.21.15.123:6817/4139299599,v1:172.21.15.123:6819/4139299599] conn(0x7f908c075ff0 0x7f908c078430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.045 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.005+0000 7f90ab400640 1 --2- 172.21.15.121:0/1191770851 >> [v2:172.21.15.123:6817/4139299599,v1:172.21.15.123:6819/4139299599] conn(0x7f908c075ff0 0x7f908c078430 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.046 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.006+0000 7ff4cd000640 1 --2- 172.21.15.121:0/4112201040 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff4a806c8f0 0x7ff4a806edb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff4c80ce790 tx=0x7ff4b40090e0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.046 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.006+0000 7ff4ce400640 1 --2- 172.21.15.121:0/4112201040 >> [v2:172.21.15.123:6808/4043981069,v1:172.21.15.123:6809/4043981069] conn(0x7ff4a8075fe0 0x7ff4a8078420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.046 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.007+0000 7f5ddd600640 1 -- 172.21.15.121:0/2390395801 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5dd80054b0 con 0x7f5de0162e50 2024-04-20T10:21:34.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.008+0000 7ff4ce400640 1 --2- 172.21.15.121:0/4112201040 >> [v2:172.21.15.123:6808/4043981069,v1:172.21.15.123:6809/4043981069] conn(0x7ff4a8075fe0 0x7ff4a8078420 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.008+0000 7f5ddd600640 1 --2- 172.21.15.121:0/2390395801 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f5dbc06c8f0 0x7f5dbc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.008+0000 7f0d5ca00640 1 -- 172.21.15.121:0/999655314 --> [v2:172.21.15.121:6821/2232342832,v1:172.21.15.121:6823/2232342832] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0d580b3e10 con 0x7f0d340760c0 2024-04-20T10:21:34.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.009+0000 7f90a2a00640 1 -- 172.21.15.121:0/1191770851 <== osd.6 v2:172.21.15.123:6817/4139299599 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f90a401aec0 con 0x7f908c075ff0 2024-04-20T10:21:34.048 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.009+0000 7f5de4a00640 1 --2- 172.21.15.121:0/2390395801 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f5dbc06c8f0 0x7f5dbc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.048 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.009+0000 7f5ddd600640 1 -- 172.21.15.121:0/2390395801 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5dd808bc30 con 0x7f5de0162e50 2024-04-20T10:21:34.048 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.009+0000 7f0d4ea00640 1 -- 172.21.15.121:0/999655314 <== osd.3 v2:172.21.15.121:6821/2232342832 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f0d580b3e10 con 0x7f0d340760c0 2024-04-20T10:21:34.049 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.009+0000 7f5ddd600640 1 --2- 172.21.15.121:0/2390395801 >> [v2:172.21.15.123:6816/2504374688,v1:172.21.15.123:6818/2504374688] conn(0x7f5dbc075ff0 0x7f5dbc078430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.049 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.009+0000 7f5ddd600640 1 -- 172.21.15.121:0/2390395801 --> [v2:172.21.15.123:6816/2504374688,v1:172.21.15.123:6818/2504374688] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5dd8014e50 con 0x7f5dbc075ff0 2024-04-20T10:21:34.049 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.009+0000 7f5ddd600640 1 -- 172.21.15.121:0/2390395801 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5dd80902a0 con 0x7f5de0162e50 2024-04-20T10:21:34.049 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.010+0000 7f5de4a00640 1 --2- 172.21.15.121:0/2390395801 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f5dbc06c8f0 0x7f5dbc06edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8001eb0 tx=0x7f5dc8001170 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.049 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.010+0000 7f0d5ca00640 1 -- 172.21.15.121:0/999655314 >> [v2:172.21.15.121:6821/2232342832,v1:172.21.15.121:6823/2232342832] conn(0x7f0d340760c0 msgr2=0x7f0d34078500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.050 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.010+0000 7f0d5ca00640 1 --2- 172.21.15.121:0/999655314 >> [v2:172.21.15.121:6821/2232342832,v1:172.21.15.121:6823/2232342832] conn(0x7f0d340760c0 0x7f0d34078500 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.050 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.010+0000 7f5dd7e00640 1 --2- 172.21.15.121:0/2390395801 >> [v2:172.21.15.123:6816/2504374688,v1:172.21.15.123:6818/2504374688] conn(0x7f5dbc075ff0 0x7f5dbc078430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.050 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.010+0000 7ff4c5600640 1 -- 172.21.15.121:0/4112201040 <== osd.7 v2:172.21.15.123:6808/4043981069 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff4a8078af0 con 0x7ff4a8075fe0 2024-04-20T10:21:34.051 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.010+0000 7f0d5ca00640 1 -- 172.21.15.121:0/999655314 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f0d3406c940 msgr2=0x7f0d3406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.051 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.011+0000 7f0d5ca00640 1 --2- 172.21.15.121:0/999655314 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f0d3406c940 0x7f0d3406ee00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0d580b40c0 tx=0x7f0d48009040 comp rx=0 tx=0).stop 2024-04-20T10:21:34.051 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.011+0000 7f0d5ca00640 1 -- 172.21.15.121:0/999655314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d580d1010 msgr2=0x7f0d5806ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.051 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.011+0000 7f0d5ca00640 1 --2- 172.21.15.121:0/999655314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0d580d1010 0x7f0d5806ba20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0d4000e8a0 tx=0x7f0d4000ed60 comp rx=0 tx=0).stop 2024-04-20T10:21:34.051 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.011+0000 7f0d57200640 1 -- 172.21.15.121:0/999655314 reap_dead start 2024-04-20T10:21:34.051 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.011+0000 7f0d5ca00640 1 -- 172.21.15.121:0/999655314 shutdown_connections 2024-04-20T10:21:34.052 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.011+0000 7f5dd7e00640 1 --2- 172.21.15.121:0/2390395801 >> [v2:172.21.15.123:6816/2504374688,v1:172.21.15.123:6818/2504374688] conn(0x7f5dbc075ff0 0x7f5dbc078430 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.052 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.011+0000 7f0d5ca00640 1 -- 172.21.15.121:0/999655314 >> 172.21.15.121:0/999655314 conn(0x7f0d5809d6f0 msgr2=0x7f0d580b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:34.052 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.012+0000 7f0d5ca00640 1 -- 172.21.15.121:0/999655314 shutdown_connections 2024-04-20T10:21:34.052 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.012+0000 7f0d5ca00640 1 -- 172.21.15.121:0/999655314 wait complete. 2024-04-20T10:21:34.053 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.014+0000 7f5ddd600640 1 -- 172.21.15.121:0/2390395801 <== osd.5 v2:172.21.15.123:6816/2504374688 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5dd8014e50 con 0x7f5dbc075ff0 2024-04-20T10:21:34.053 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.020+0000 7efe9ea00640 1 -- 172.21.15.121:0/2890459069 --> [v2:172.21.15.121:6800/3505199624,v1:172.21.15.121:6801/3505199624] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efe98075b30 con 0x7efe7c076030 2024-04-20T10:21:34.053 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.021+0000 7f8332000640 1 -- 172.21.15.121:0/1652080621 --> [v2:172.21.15.121:6808/1044588835,v1:172.21.15.121:6809/1044588835] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f831c002dd0 con 0x7f83200761f0 2024-04-20T10:21:34.054 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.021+0000 7efe95600640 1 -- 172.21.15.121:0/2890459069 <== osd.1 v2:172.21.15.121:6800/3505199624 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7efe98075b30 con 0x7efe7c076030 2024-04-20T10:21:34.054 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.022+0000 7f833cc00640 1 -- 172.21.15.121:0/1652080621 <== osd.0 v2:172.21.15.121:6808/1044588835 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f831c002dd0 con 0x7f83200761f0 2024-04-20T10:21:34.054 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.022+0000 7efe9ea00640 1 -- 172.21.15.121:0/2890459069 >> [v2:172.21.15.121:6800/3505199624,v1:172.21.15.121:6801/3505199624] conn(0x7efe7c076030 msgr2=0x7efe7c078470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.054 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.022+0000 7efe9ea00640 1 --2- 172.21.15.121:0/2890459069 >> [v2:172.21.15.121:6800/3505199624,v1:172.21.15.121:6801/3505199624] conn(0x7efe7c076030 0x7efe7c078470 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.055 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.022+0000 7efe9ea00640 1 -- 172.21.15.121:0/2890459069 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7efe7c06c940 msgr2=0x7efe7c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.055 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.022+0000 7efe9ea00640 1 --2- 172.21.15.121:0/2890459069 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7efe7c06c940 0x7efe7c06ee00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efe90000f80 tx=0x7efe90030000 comp rx=0 tx=0).stop 2024-04-20T10:21:34.055 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.022+0000 7efe9ea00640 1 -- 172.21.15.121:0/2890459069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe9815e600 msgr2=0x7efe9806dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.056 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.022+0000 7efe9ea00640 1 --2- 172.21.15.121:0/2890459069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efe9815e600 0x7efe9806dea0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efe8c00c390 tx=0x7efe8c00c850 comp rx=0 tx=0).stop 2024-04-20T10:21:34.056 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.023+0000 7efe9e000640 1 -- 172.21.15.121:0/2890459069 reap_dead start 2024-04-20T10:21:34.056 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.023+0000 7efe9ea00640 1 -- 172.21.15.121:0/2890459069 shutdown_connections 2024-04-20T10:21:34.056 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.023+0000 7efe9ea00640 1 -- 172.21.15.121:0/2890459069 >> 172.21.15.121:0/2890459069 conn(0x7efe9809d6c0 msgr2=0x7efe980cac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:34.056 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.023+0000 7efe9ea00640 1 -- 172.21.15.121:0/2890459069 shutdown_connections 2024-04-20T10:21:34.056 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.023+0000 7efe9ea00640 1 -- 172.21.15.121:0/2890459069 wait complete. 2024-04-20T10:21:34.057 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.023+0000 7f8332000640 1 -- 172.21.15.121:0/1652080621 >> [v2:172.21.15.121:6808/1044588835,v1:172.21.15.121:6809/1044588835] conn(0x7f83200761f0 msgr2=0x7f8320078630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.057 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.023+0000 7f8332000640 1 --2- 172.21.15.121:0/1652080621 >> [v2:172.21.15.121:6808/1044588835,v1:172.21.15.121:6809/1044588835] conn(0x7f83200761f0 0x7f8320078630 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.057 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.024+0000 7f8332000640 1 -- 172.21.15.121:0/1652080621 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f832006caf0 msgr2=0x7f832006efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.057 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.024+0000 7f8332000640 1 --2- 172.21.15.121:0/1652080621 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f832006caf0 0x7f832006efb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f83340011e0 tx=0x7f8334007d30 comp rx=0 tx=0).stop 2024-04-20T10:21:34.057 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.024+0000 7f8332000640 1 -- 172.21.15.121:0/1652080621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f833803e830 msgr2=0x7f8338050a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.057 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.024+0000 7f8332000640 1 --2- 172.21.15.121:0/1652080621 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f833803e830 0x7f8338050a30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f832c00c450 tx=0x7f832c00c910 comp rx=0 tx=0).stop 2024-04-20T10:21:34.057 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.024+0000 7f8345800640 1 -- 172.21.15.121:0/1652080621 reap_dead start 2024-04-20T10:21:34.057 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.024+0000 7f8332000640 1 -- 172.21.15.121:0/1652080621 shutdown_connections 2024-04-20T10:21:34.057 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.024+0000 7f8332000640 1 -- 172.21.15.121:0/1652080621 >> 172.21.15.121:0/1652080621 conn(0x7f83380559d0 msgr2=0x7f833803de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:34.057 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.024+0000 7f8332000640 1 -- 172.21.15.121:0/1652080621 shutdown_connections 2024-04-20T10:21:34.057 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.025+0000 7f8332000640 1 -- 172.21.15.121:0/1652080621 wait complete. 2024-04-20T10:21:34.058 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.028+0000 7f53f9800640 1 -- 172.21.15.121:0/2890674187 --> [v2:172.21.15.123:6800/99353111,v1:172.21.15.123:6801/99353111] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f53f40b6850 con 0x7f53d00760c0 2024-04-20T10:21:34.058 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.029+0000 7f53eaa00640 1 -- 172.21.15.121:0/2890674187 <== osd.4 v2:172.21.15.123:6800/99353111 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f53f40b6850 con 0x7f53d00760c0 2024-04-20T10:21:34.058 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.029+0000 7f53f9800640 1 -- 172.21.15.121:0/2890674187 >> [v2:172.21.15.123:6800/99353111,v1:172.21.15.123:6801/99353111] conn(0x7f53d00760c0 msgr2=0x7f53d0078500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.058 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.030+0000 7f53f9800640 1 --2- 172.21.15.121:0/2890674187 >> [v2:172.21.15.123:6800/99353111,v1:172.21.15.123:6801/99353111] conn(0x7f53d00760c0 0x7f53d0078500 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.059 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.030+0000 7f9751200640 1 -- 172.21.15.121:0/1611060524 --> [v2:172.21.15.121:6816/3656724207,v1:172.21.15.121:6817/3656724207] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9724002dd0 con 0x7f972c0761e0 2024-04-20T10:21:34.059 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.031+0000 7f53f9800640 1 -- 172.21.15.121:0/2890674187 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f53d006c940 msgr2=0x7f53d006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.059 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.031+0000 7f53f9800640 1 --2- 172.21.15.121:0/2890674187 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f53d006c940 0x7f53d006ee00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f53f40c7a50 tx=0x7f53dc006c20 comp rx=0 tx=0).stop 2024-04-20T10:21:34.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.031+0000 7f53f9800640 1 -- 172.21.15.121:0/2890674187 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f53f40b17d0 msgr2=0x7f53f40ce010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.031+0000 7f53f9800640 1 --2- 172.21.15.121:0/2890674187 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f53f40b17d0 0x7f53f40ce010 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f53e400c460 tx=0x7f53e400c920 comp rx=0 tx=0).stop 2024-04-20T10:21:34.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.031+0000 7f53f3400640 1 -- 172.21.15.121:0/2890674187 reap_dead start 2024-04-20T10:21:34.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.031+0000 7f53f9800640 1 -- 172.21.15.121:0/2890674187 shutdown_connections 2024-04-20T10:21:34.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.031+0000 7f53f9800640 1 -- 172.21.15.121:0/2890674187 >> 172.21.15.121:0/2890674187 conn(0x7f53f40b99c0 msgr2=0x7f53f4160b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:34.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.031+0000 7f53f9800640 1 -- 172.21.15.121:0/2890674187 shutdown_connections 2024-04-20T10:21:34.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.031+0000 7f53f9800640 1 -- 172.21.15.121:0/2890674187 wait complete. 2024-04-20T10:21:34.061 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.031+0000 7f9742000640 1 -- 172.21.15.121:0/1611060524 <== osd.2 v2:172.21.15.121:6816/3656724207 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9724002dd0 con 0x7f972c0761e0 2024-04-20T10:21:34.061 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.032+0000 7f9751200640 1 -- 172.21.15.121:0/1611060524 >> [v2:172.21.15.121:6816/3656724207,v1:172.21.15.121:6817/3656724207] conn(0x7f972c0761e0 msgr2=0x7f972c078620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.061 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.032+0000 7f9751200640 1 --2- 172.21.15.121:0/1611060524 >> [v2:172.21.15.121:6816/3656724207,v1:172.21.15.121:6817/3656724207] conn(0x7f972c0761e0 0x7f972c078620 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.061 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.032+0000 7f9751200640 1 -- 172.21.15.121:0/1611060524 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f972c06caf0 msgr2=0x7f972c06efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.061 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.032+0000 7f9751200640 1 --2- 172.21.15.121:0/1611060524 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f972c06caf0 0x7f972c06efb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9744005c00 tx=0x7f9744009040 comp rx=0 tx=0).stop 2024-04-20T10:21:34.061 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.032+0000 7f9751200640 1 -- 172.21.15.121:0/1611060524 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f974c06e510 msgr2=0x7f974c168870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.061 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.033+0000 7f9751200640 1 --2- 172.21.15.121:0/1611060524 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f974c06e510 0x7f974c168870 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f973c00c750 tx=0x7f973c00cc10 comp rx=0 tx=0).stop 2024-04-20T10:21:34.062 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.033+0000 7f974a800640 1 -- 172.21.15.121:0/1611060524 reap_dead start 2024-04-20T10:21:34.062 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.033+0000 7f9751200640 1 -- 172.21.15.121:0/1611060524 shutdown_connections 2024-04-20T10:21:34.062 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.033+0000 7f9751200640 1 -- 172.21.15.121:0/1611060524 >> 172.21.15.121:0/1611060524 conn(0x7f974c072090 msgr2=0x7f974c073690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:34.062 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.033+0000 7f9751200640 1 -- 172.21.15.121:0/1611060524 shutdown_connections 2024-04-20T10:21:34.062 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.033+0000 7f9751200640 1 -- 172.21.15.121:0/1611060524 wait complete. 2024-04-20T10:21:34.062 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.035+0000 7f90b1800640 1 -- 172.21.15.121:0/1191770851 --> [v2:172.21.15.123:6817/4139299599,v1:172.21.15.123:6819/4139299599] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9084002dd0 con 0x7f908c075ff0 2024-04-20T10:21:34.063 INFO:teuthology.orchestra.run.smithi121.stdout:60129542154 2024-04-20T10:21:34.063 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-20T10:21:34.064 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.037+0000 7f90a2a00640 1 -- 172.21.15.121:0/1191770851 <== osd.6 v2:172.21.15.123:6817/4139299599 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9084002dd0 con 0x7f908c075ff0 2024-04-20T10:21:34.064 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.037+0000 7ff4d0400640 1 -- 172.21.15.121:0/4112201040 --> [v2:172.21.15.123:6808/4043981069,v1:172.21.15.123:6809/4043981069] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff4c81616f0 con 0x7ff4a8075fe0 2024-04-20T10:21:34.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.038+0000 7f90b1800640 1 -- 172.21.15.121:0/1191770851 >> [v2:172.21.15.123:6817/4139299599,v1:172.21.15.123:6819/4139299599] conn(0x7f908c075ff0 msgr2=0x7f908c078430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.038+0000 7f90b1800640 1 --2- 172.21.15.121:0/1191770851 >> [v2:172.21.15.123:6817/4139299599,v1:172.21.15.123:6819/4139299599] conn(0x7f908c075ff0 0x7f908c078430 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.038+0000 7f90b1800640 1 -- 172.21.15.121:0/1191770851 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f908c06c8f0 msgr2=0x7f908c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.038+0000 7f90b1800640 1 --2- 172.21.15.121:0/1191770851 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f908c06c8f0 0x7f908c06edb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f909c009870 tx=0x7f909c039040 comp rx=0 tx=0).stop 2024-04-20T10:21:34.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.038+0000 7f90b1800640 1 -- 172.21.15.121:0/1191770851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90ac0b17d0 msgr2=0x7f90ac0cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.038+0000 7f90b1800640 1 --2- 172.21.15.121:0/1191770851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f90ac0b17d0 0x7f90ac0cc0c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f90a400ca70 tx=0x7f90a400cf30 comp rx=0 tx=0).stop 2024-04-20T10:21:34.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.038+0000 7f90ab400640 1 -- 172.21.15.121:0/1191770851 reap_dead start 2024-04-20T10:21:34.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.038+0000 7ff4c5600640 1 -- 172.21.15.121:0/4112201040 <== osd.7 v2:172.21.15.123:6808/4043981069 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff4c81616f0 con 0x7ff4a8075fe0 2024-04-20T10:21:34.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.040+0000 7f90b1800640 1 -- 172.21.15.121:0/1191770851 shutdown_connections 2024-04-20T10:21:34.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.040+0000 7f90b1800640 1 -- 172.21.15.121:0/1191770851 >> 172.21.15.121:0/1191770851 conn(0x7f90ac0b99c0 msgr2=0x7f90ac0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:34.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.040+0000 7f90b1800640 1 -- 172.21.15.121:0/1191770851 shutdown_connections 2024-04-20T10:21:34.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.041+0000 7f90b1800640 1 -- 172.21.15.121:0/1191770851 wait complete. 2024-04-20T10:21:34.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.040+0000 7ff4d0400640 1 -- 172.21.15.121:0/4112201040 >> [v2:172.21.15.123:6808/4043981069,v1:172.21.15.123:6809/4043981069] conn(0x7ff4a8075fe0 msgr2=0x7ff4a8078420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.068 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.040+0000 7ff4d0400640 1 --2- 172.21.15.121:0/4112201040 >> [v2:172.21.15.123:6808/4043981069,v1:172.21.15.123:6809/4043981069] conn(0x7ff4a8075fe0 0x7ff4a8078420 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.068 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.041+0000 7ff4d0400640 1 -- 172.21.15.121:0/4112201040 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff4a806c8f0 msgr2=0x7ff4a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.068 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.041+0000 7ff4d0400640 1 --2- 172.21.15.121:0/4112201040 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff4a806c8f0 0x7ff4a806edb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff4c80ce790 tx=0x7ff4b40090e0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.068 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.041+0000 7ff4d0400640 1 -- 172.21.15.121:0/4112201040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4c80b17d0 msgr2=0x7ff4c806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.068 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.041+0000 7ff4d0400640 1 --2- 172.21.15.121:0/4112201040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4c80b17d0 0x7ff4c806e320 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00c3c0 tx=0x7ff4bc00c880 comp rx=0 tx=0).stop 2024-04-20T10:21:34.068 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.041+0000 7ff4ce400640 1 -- 172.21.15.121:0/4112201040 reap_dead start 2024-04-20T10:21:34.068 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.041+0000 7ff4d0400640 1 -- 172.21.15.121:0/4112201040 shutdown_connections 2024-04-20T10:21:34.068 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.041+0000 7ff4d0400640 1 -- 172.21.15.121:0/4112201040 >> 172.21.15.121:0/4112201040 conn(0x7ff4c80b99c0 msgr2=0x7ff4c80b1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:34.068 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.041+0000 7ff4d0400640 1 -- 172.21.15.121:0/4112201040 shutdown_connections 2024-04-20T10:21:34.068 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.042+0000 7ff4d0400640 1 -- 172.21.15.121:0/4112201040 wait complete. 2024-04-20T10:21:34.069 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.049+0000 7f5de7400640 1 -- 172.21.15.121:0/2390395801 --> [v2:172.21.15.123:6816/2504374688,v1:172.21.15.123:6818/2504374688] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5de00b7870 con 0x7f5dbc075ff0 2024-04-20T10:21:34.069 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.050+0000 7f5ddd600640 1 -- 172.21.15.121:0/2390395801 <== osd.5 v2:172.21.15.123:6816/2504374688 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5de00b7870 con 0x7f5dbc075ff0 2024-04-20T10:21:34.070 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.052+0000 7f5de7400640 1 -- 172.21.15.121:0/2390395801 >> [v2:172.21.15.123:6816/2504374688,v1:172.21.15.123:6818/2504374688] conn(0x7f5dbc075ff0 msgr2=0x7f5dbc078430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.070 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.052+0000 7f5de7400640 1 --2- 172.21.15.121:0/2390395801 >> [v2:172.21.15.123:6816/2504374688,v1:172.21.15.123:6818/2504374688] conn(0x7f5dbc075ff0 0x7f5dbc078430 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.070 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.052+0000 7f5de7400640 1 -- 172.21.15.121:0/2390395801 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f5dbc06c8f0 msgr2=0x7f5dbc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.070 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.052+0000 7f5de7400640 1 --2- 172.21.15.121:0/2390395801 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f5dbc06c8f0 0x7f5dbc06edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8001eb0 tx=0x7f5dc8001170 comp rx=0 tx=0).stop 2024-04-20T10:21:34.070 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.052+0000 7f5de7400640 1 -- 172.21.15.121:0/2390395801 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5de0162e50 msgr2=0x7f5de0161610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.070 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.052+0000 7f5de7400640 1 --2- 172.21.15.121:0/2390395801 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5de0162e50 0x7f5de0161610 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5dd800c730 tx=0x7f5dd800cbf0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.070 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.052+0000 7f5de5400640 1 -- 172.21.15.121:0/2390395801 reap_dead start 2024-04-20T10:21:34.071 INFO:teuthology.orchestra.run.smithi121.stdout:60129542156 2024-04-20T10:21:34.071 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.053+0000 7f5de7400640 1 -- 172.21.15.121:0/2390395801 shutdown_connections 2024-04-20T10:21:34.071 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.053+0000 7f5de7400640 1 -- 172.21.15.121:0/2390395801 >> 172.21.15.121:0/2390395801 conn(0x7f5de00ba580 msgr2=0x7f5de00cd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:34.072 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-20T10:21:34.072 INFO:teuthology.orchestra.run.smithi121.stdout:60129542156 2024-04-20T10:21:34.073 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-20T10:21:34.073 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.054+0000 7f5de7400640 1 -- 172.21.15.121:0/2390395801 shutdown_connections 2024-04-20T10:21:34.073 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.054+0000 7f5de7400640 1 -- 172.21.15.121:0/2390395801 wait complete. 2024-04-20T10:21:34.074 INFO:teuthology.orchestra.run.smithi121.stdout:60129542155 2024-04-20T10:21:34.074 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-20T10:21:34.075 INFO:teuthology.orchestra.run.smithi121.stdout:60129542154 2024-04-20T10:21:34.075 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-20T10:21:34.076 INFO:teuthology.orchestra.run.smithi121.stdout:60129542154 2024-04-20T10:21:34.076 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-20T10:21:34.077 INFO:teuthology.orchestra.run.smithi121.stdout:60129542154 2024-04-20T10:21:34.077 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-20T10:21:34.078 INFO:teuthology.orchestra.run.smithi121.stdout:60129542154 2024-04-20T10:21:34.078 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-20T10:21:34.387 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.387+0000 7f41e8c00640 1 Processor -- start 2024-04-20T10:21:34.389 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.388+0000 7f41e8c00640 1 -- start start 2024-04-20T10:21:34.389 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.390+0000 7f41e8c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f41dc0078d0 0x7f41dc0edd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.389 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.390+0000 7f41e8c00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41dc058560 con 0x7f41dc0e30b0 2024-04-20T10:21:34.389 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.390+0000 7f41e8c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41dc058840 con 0x7f41dc03e830 2024-04-20T10:21:34.390 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.390+0000 7f41e8c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f41dc058b20 con 0x7f41dc0078d0 2024-04-20T10:21:34.390 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.391+0000 7f41e1e00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f41dc0e30b0 0x7f41dc057d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:50074/0 (socket says 172.21.15.121:50074) 2024-04-20T10:21:34.390 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.391+0000 7f41e1e00640 1 -- 172.21.15.121:0/614715501 learned_addr learned my addr 172.21.15.121:0/614715501 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:34.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.391+0000 7f41e2800640 1 --2- 172.21.15.121:0/614715501 >> v2:172.21.15.123:3300/0 conn(0x7f41dc0078d0 0x7f41dc0edd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.392+0000 7f41e0a00640 1 -- 172.21.15.121:0/614715501 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1032756541 0 0) 0x7f41dc058560 con 0x7f41dc0e30b0 2024-04-20T10:21:34.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.392+0000 7f41e0a00640 1 -- 172.21.15.121:0/614715501 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41c8003520 con 0x7f41dc0e30b0 2024-04-20T10:21:34.393 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.393+0000 7f41e2800640 1 -- 172.21.15.121:0/614715501 >> v1:172.21.15.123:6789/0 conn(0x7f41dc03e830 legacy=0x7f41dc053be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.393 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.393+0000 7f41e2800640 1 -- 172.21.15.121:0/614715501 >> v1:172.21.15.121:6789/0 conn(0x7f41dc0e30b0 legacy=0x7f41dc057d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.393 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.393+0000 7f41e2800640 1 -- 172.21.15.121:0/614715501 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41dc0063a0 con 0x7f41dc0078d0 2024-04-20T10:21:34.394 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.394+0000 7f41e2800640 1 --2- 172.21.15.121:0/614715501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41dc0078d0 0x7f41dc0edd80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f41e4050210 tx=0x7f41e40a7280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90df44c0a0bac14a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.394 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.395+0000 7f41e0a00640 1 -- 172.21.15.121:0/614715501 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e40b4070 con 0x7f41dc0078d0 2024-04-20T10:21:34.394 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.395+0000 7f41e0a00640 1 -- 172.21.15.121:0/614715501 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41e40af640 con 0x7f41dc0078d0 2024-04-20T10:21:34.395 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.395+0000 7f41e0a00640 1 -- 172.21.15.121:0/614715501 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e40afa30 con 0x7f41dc0078d0 2024-04-20T10:21:34.396 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.396+0000 7f41e8c00640 1 -- 172.21.15.121:0/614715501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41dc0078d0 msgr2=0x7f41dc0edd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.396 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.396+0000 7f41e8c00640 1 --2- 172.21.15.121:0/614715501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41dc0078d0 0x7f41dc0edd80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f41e4050210 tx=0x7f41e40a7280 comp rx=0 tx=0).stop 2024-04-20T10:21:34.396 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.397+0000 7f41e8c00640 1 -- 172.21.15.121:0/614715501 shutdown_connections 2024-04-20T10:21:34.396 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.397+0000 7f41e8c00640 1 --2- 172.21.15.121:0/614715501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41dc0078d0 0x7f41dc0edd80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.396 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.397+0000 7f3769000640 1 Processor -- start 2024-04-20T10:21:34.397 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.397+0000 7f41e8c00640 1 -- 172.21.15.121:0/614715501 >> 172.21.15.121:0/614715501 conn(0x7f41dc0559d0 msgr2=0x7f41dc055de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:34.397 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.397+0000 7f3769000640 1 -- start start 2024-04-20T10:21:34.398 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.398+0000 7f41e8c00640 1 -- 172.21.15.121:0/614715501 shutdown_connections 2024-04-20T10:21:34.398 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.398+0000 7f41e8c00640 1 -- 172.21.15.121:0/614715501 wait complete. 2024-04-20T10:21:34.400 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.400+0000 7f3769000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f37640b3330 0x7f37640d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.400 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.401+0000 7f3769000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f376406c190 con 0x7f3764161d50 2024-04-20T10:21:34.401 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.401+0000 7f3769000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f376406c470 con 0x7f376406cde0 2024-04-20T10:21:34.401 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.401+0000 7f41e8c00640 1 Processor -- start 2024-04-20T10:21:34.401 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.401+0000 7f41e8c00640 1 -- start start 2024-04-20T10:21:34.402 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.401+0000 7f3769000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f3764162130 con 0x7f37640b3330 2024-04-20T10:21:34.402 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.402+0000 7f3763400640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f3764161d50 0x7f376406c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:50084/0 (socket says 172.21.15.121:50084) 2024-04-20T10:21:34.402 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.402+0000 7f375aa00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f376406cde0 0x7f37640b7660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.121:46686/0 (socket says 172.21.15.121:46686) 2024-04-20T10:21:34.402 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.402+0000 7f375aa00640 1 -- 172.21.15.121:0/2934070691 learned_addr learned my addr 172.21.15.121:0/2934070691 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:34.402 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.402+0000 7f41e8c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41dc0078d0 0x7f41dc0df530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.403 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.403+0000 7f41e8c00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f41dc03e830 0x7f41dc0c6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.403 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.402+0000 7f3763e00640 1 --2- 172.21.15.121:0/2934070691 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f37640b3330 0x7f37640d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.403 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.403+0000 7f41e1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41dc0078d0 0x7f41dc0df530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.404 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.403+0000 7f41e1e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41dc0078d0 0x7f41dc0df530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.121:35308/0 (socket says 172.21.15.121:35308) 2024-04-20T10:21:34.404 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.403+0000 7f41e1e00640 1 -- 172.21.15.121:0/935832473 learned_addr learned my addr 172.21.15.121:0/935832473 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:34.404 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.403+0000 7f41e1400640 1 --2- 172.21.15.121:0/935832473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f41dc03e830 0x7f41dc0c6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.404 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.404+0000 7f41e8c00640 1 --2- 172.21.15.121:0/935832473 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41dc0e30b0 0x7f41dc04e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.404 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.404+0000 7f3762a00640 1 -- 172.21.15.121:0/2934070691 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1727375072 0 0) 0x7f376406c470 con 0x7f376406cde0 2024-04-20T10:21:34.404 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.404+0000 7f3762a00640 1 -- 172.21.15.121:0/2934070691 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3748003520 con 0x7f376406cde0 2024-04-20T10:21:34.404 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.404+0000 7f3762a00640 1 -- 172.21.15.121:0/2934070691 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4151646671 0 0) 0x7f376406c190 con 0x7f3764161d50 2024-04-20T10:21:34.404 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.404+0000 7f3762a00640 1 -- 172.21.15.121:0/2934070691 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f376406c470 con 0x7f3764161d50 2024-04-20T10:21:34.405 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.404+0000 7fe996400640 1 Processor -- start 2024-04-20T10:21:34.405 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.404+0000 7fe996400640 1 -- start start 2024-04-20T10:21:34.405 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.404+0000 7f41e8c00640 1 -- 172.21.15.121:0/935832473 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc0542f0 con 0x7f41dc0e30b0 2024-04-20T10:21:34.405 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.404+0000 7f41e8c00640 1 -- 172.21.15.121:0/935832473 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f41dc0e3490 con 0x7f41dc0078d0 2024-04-20T10:21:34.405 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.404+0000 7f41e8c00640 1 -- 172.21.15.121:0/935832473 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f41dc054470 con 0x7f41dc03e830 2024-04-20T10:21:34.406 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.405+0000 7f41e2800640 1 --2- 172.21.15.121:0/935832473 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41dc0e30b0 0x7f41dc04e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.406 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.405+0000 7f3763e00640 1 -- 172.21.15.121:0/2934070691 >> v1:172.21.15.123:6789/0 conn(0x7f376406cde0 legacy=0x7f37640b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.406 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.405+0000 7f3763e00640 1 -- 172.21.15.121:0/2934070691 >> v1:172.21.15.121:6789/0 conn(0x7f3764161d50 legacy=0x7f376406c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.406 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.405+0000 7f3762a00640 1 -- 172.21.15.121:0/2934070691 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2773873841 0 0) 0x7f376406c470 con 0x7f3764161d50 2024-04-20T10:21:34.407 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.405+0000 7f41e1400640 1 -- 172.21.15.121:0/935832473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41dc0078d0 msgr2=0x7f41dc0df530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.407 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.405+0000 7f41e1400640 1 --2- 172.21.15.121:0/935832473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41dc0078d0 0x7f41dc0df530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.407 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.406+0000 7f41e1400640 1 -- 172.21.15.121:0/935832473 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41dc0e30b0 msgr2=0x7f41dc04e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.407 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.406+0000 7f41e1400640 1 --2- 172.21.15.121:0/935832473 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41dc0e30b0 0x7f41dc04e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.407 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.406+0000 7f41e1400640 1 -- 172.21.15.121:0/935832473 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41dc0e1190 con 0x7f41dc03e830 2024-04-20T10:21:34.407 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.405+0000 7f3763e00640 1 -- 172.21.15.121:0/2934070691 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37640d2e00 con 0x7f37640b3330 2024-04-20T10:21:34.408 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.406+0000 7fe996400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9900b3bf0 0x7fe9900ad210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.408 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.407+0000 7fe996400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9900adaf0 con 0x7fe99006e510 2024-04-20T10:21:34.408 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.407+0000 7fe996400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9900addd0 con 0x7fe99006bdd0 2024-04-20T10:21:34.408 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.407+0000 7fe996400640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe9900ae0b0 con 0x7fe9900b3bf0 2024-04-20T10:21:34.408 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.407+0000 7f41e1400640 1 --2- 172.21.15.121:0/935832473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f41dc03e830 0x7f41dc0c6fc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f41d000c390 tx=0x7f41d000c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.408 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.407+0000 7f3763e00640 1 --2- 172.21.15.121:0/2934070691 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f37640b3330 0x7f37640d38e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f375c009750 tx=0x7f375c02b680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c73d927999c3e07b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.409 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.407+0000 7fdc56600640 1 Processor -- start 2024-04-20T10:21:34.409 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.407+0000 7fe98fe00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fe99006e510 0x7fe99015a560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:50096/0 (socket says 172.21.15.121:50096) 2024-04-20T10:21:34.409 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.407+0000 7fe98fe00640 1 -- 172.21.15.121:0/152747176 learned_addr learned my addr 172.21.15.121:0/152747176 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:34.409 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.407+0000 7fe986a00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fe99006bdd0 0x7fe99006de00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.121:46696/0 (socket says 172.21.15.121:46696) 2024-04-20T10:21:34.409 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.408+0000 7fe98f400640 1 --2- 172.21.15.121:0/152747176 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9900b3bf0 0x7fe9900ad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.409 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.407+0000 7fe6a3000640 1 Processor -- start 2024-04-20T10:21:34.410 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.408+0000 7fdc56600640 1 -- start start 2024-04-20T10:21:34.410 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.408+0000 7f3762a00640 1 -- 172.21.15.121:0/2934070691 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c037070 con 0x7f37640b3330 2024-04-20T10:21:34.410 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.408+0000 7f3762a00640 1 -- 172.21.15.121:0/2934070691 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f375c030510 con 0x7f37640b3330 2024-04-20T10:21:34.410 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.408+0000 7f3762a00640 1 -- 172.21.15.121:0/2934070691 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c037070 con 0x7f37640b3330 2024-04-20T10:21:34.411 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.408+0000 7fe98ea00640 1 -- 172.21.15.121:0/152747176 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1538416448 0 0) 0x7fe9900adaf0 con 0x7fe99006e510 2024-04-20T10:21:34.411 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.408+0000 7fe98ea00640 1 -- 172.21.15.121:0/152747176 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe974003520 con 0x7fe99006e510 2024-04-20T10:21:34.411 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.408+0000 7f41da000640 1 -- 172.21.15.121:0/935832473 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d0017070 con 0x7f41dc03e830 2024-04-20T10:21:34.411 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.409+0000 7f41da000640 1 -- 172.21.15.121:0/935832473 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41d00052d0 con 0x7f41dc03e830 2024-04-20T10:21:34.411 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.409+0000 7f41da000640 1 -- 172.21.15.121:0/935832473 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d0003830 con 0x7f41dc03e830 2024-04-20T10:21:34.411 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.409+0000 7f41e8c00640 1 -- 172.21.15.121:0/935832473 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41dc0c9d40 con 0x7f41dc03e830 2024-04-20T10:21:34.412 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.409+0000 7f41e8c00640 1 -- 172.21.15.121:0/935832473 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f41dc0e16f0 con 0x7f41dc03e830 2024-04-20T10:21:34.412 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.408+0000 7fe6a3000640 1 -- start start 2024-04-20T10:21:34.412 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.410+0000 7fe6a3000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe69c0d4270 0x7fe69c0d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.412 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.410+0000 7fe6a3000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe69c0d0380 0x7fe69c0d07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.412 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.411+0000 7f41e8c00640 1 -- 172.21.15.121:0/935832473 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41dc0057d0 con 0x7f41dc03e830 2024-04-20T10:21:34.413 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.411+0000 7f3769000640 1 -- 172.21.15.121:0/2934070691 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f37640b3330 msgr2=0x7f37640d38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.413 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.411+0000 7f3769000640 1 --2- 172.21.15.121:0/2934070691 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f37640b3330 0x7f37640d38e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f375c009750 tx=0x7f375c02b680 comp rx=0 tx=0).stop 2024-04-20T10:21:34.413 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.410+0000 7fdc56600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdc500d1a00 0x7fdc500b1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.413 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.410+0000 7fdc56600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdc500b17d0 0x7fdc500d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.414 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.410+0000 7fdc56600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc500d1e90 con 0x7fdc500d4270 2024-04-20T10:21:34.414 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.410+0000 7fdc56600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fdc500d40c0 con 0x7fdc500b17d0 2024-04-20T10:21:34.414 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.410+0000 7fdc56600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fdc500d3820 con 0x7fdc500d1a00 2024-04-20T10:21:34.414 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.410+0000 7fdc4fe00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdc500b17d0 0x7fdc500d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.414 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.410+0000 7fdc4fe00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdc500b17d0 0x7fdc500d0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54354/0 (socket says 172.21.15.121:54354) 2024-04-20T10:21:34.414 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.410+0000 7fdc4fe00640 1 -- 172.21.15.121:0/3210786826 learned_addr learned my addr 172.21.15.121:0/3210786826 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:34.414 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.411+0000 7fdc4ea00640 1 --2- 172.21.15.121:0/3210786826 >> v2:172.21.15.123:3300/0 conn(0x7fdc500d1a00 0x7fdc500b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.414 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.412+0000 7f9e65600640 1 Processor -- start 2024-04-20T10:21:34.415 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.411+0000 7f3769000640 1 -- 172.21.15.121:0/2934070691 shutdown_connections 2024-04-20T10:21:34.415 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.411+0000 7f3769000640 1 --2- 172.21.15.121:0/2934070691 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f37640b3330 0x7f37640d38e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.415 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.411+0000 7f3769000640 1 -- 172.21.15.121:0/2934070691 >> 172.21.15.121:0/2934070691 conn(0x7f37640d4450 msgr2=0x7f37640d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:34.415 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.409+0000 7fe98ea00640 1 -- 172.21.15.121:0/152747176 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2671431071 0 0) 0x7fe974003520 con 0x7fe99006e510 2024-04-20T10:21:34.415 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.412+0000 7f3769000640 1 -- 172.21.15.121:0/2934070691 shutdown_connections 2024-04-20T10:21:34.415 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.412+0000 7f3769000640 1 -- 172.21.15.121:0/2934070691 wait complete. 2024-04-20T10:21:34.416 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.412+0000 7fe6a3000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe69c0d0d10 con 0x7fe69c0d4650 2024-04-20T10:21:34.416 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.412+0000 7fe6a3000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fe69c0d40c0 con 0x7fe69c0d4270 2024-04-20T10:21:34.416 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.412+0000 7fe6a3000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fe69c0d3820 con 0x7fe69c0d0380 2024-04-20T10:21:34.416 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.410+0000 7fe98ea00640 1 -- 172.21.15.121:0/152747176 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9900b3bf0 msgr2=0x7fe9900ad210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:34.416 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.410+0000 7fe98ea00640 1 --2- 172.21.15.121:0/152747176 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9900b3bf0 0x7fe9900ad210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.416 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.411+0000 7fe98ea00640 1 -- 172.21.15.121:0/152747176 >> v1:172.21.15.123:6789/0 conn(0x7fe99006bdd0 legacy=0x7fe99006de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.417 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.411+0000 7fe98ea00640 1 -- 172.21.15.121:0/152747176 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9900ae390 con 0x7fe99006e510 2024-04-20T10:21:34.417 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.411+0000 7fe98ea00640 1 -- 172.21.15.121:0/152747176 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7fe9880019d0 con 0x7fe99006e510 2024-04-20T10:21:34.417 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.411+0000 7fe98ea00640 1 -- 172.21.15.121:0/152747176 >> v1:172.21.15.121:6789/0 conn(0x7fe99006e510 legacy=0x7fe99015a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.417 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.411+0000 7fe98ea00640 1 --2- 172.21.15.121:0/152747176 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9740039e0 0x7fe974003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.417 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.411+0000 7fe98ea00640 1 --2- 172.21.15.121:0/152747176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe974004380 0x7fe974004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.417 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.411+0000 7fe98ea00640 1 --2- 172.21.15.121:0/152747176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe974005010 0x7fe9740141d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.417 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.411+0000 7fe98ea00640 1 -- 172.21.15.121:0/152747176 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe9900ae0b0 con 0x7fe974004380 2024-04-20T10:21:34.417 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.411+0000 7fe98ea00640 1 -- 172.21.15.121:0/152747176 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe974014740 con 0x7fe974005010 2024-04-20T10:21:34.417 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.411+0000 7fe98ea00640 1 -- 172.21.15.121:0/152747176 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe974014940 con 0x7fe9740039e0 2024-04-20T10:21:34.417 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.413+0000 7fe98f400640 1 --2- 172.21.15.121:0/152747176 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9740039e0 0x7fe974003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.417 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.413+0000 7fe98fe00640 1 --2- 172.21.15.121:0/152747176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe974005010 0x7fe9740141d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.418 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.412+0000 7f9e65600640 1 -- start start 2024-04-20T10:21:34.418 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.412+0000 7fdc4ea00640 1 -- 172.21.15.121:0/3210786826 >> v2:172.21.15.121:3300/0 conn(0x7fdc500b17d0 msgr2=0x7fdc500d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:34.418 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.412+0000 7fdc4ea00640 1 --2- 172.21.15.121:0/3210786826 >> v2:172.21.15.121:3300/0 conn(0x7fdc500b17d0 0x7fdc500d0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.418 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.412+0000 7fdc4ea00640 1 -- 172.21.15.121:0/3210786826 >> v1:172.21.15.121:6789/0 conn(0x7fdc500d4270 legacy=0x7fdc500d4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.418 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.412+0000 7fdc4ea00640 1 -- 172.21.15.121:0/3210786826 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc500d3b00 con 0x7fdc500d1a00 2024-04-20T10:21:34.418 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.413+0000 7fe69be00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe69c0d4270 0x7fe69c0d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.419 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.414+0000 7f29a0600640 1 Processor -- start 2024-04-20T10:21:34.419 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.414+0000 7fdc4ea00640 1 --2- 172.21.15.121:0/3210786826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc500d1a00 0x7fdc500b1290 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdc500b8060 tx=0x7fdc4002b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f35d84e9340fb3c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.419 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.414+0000 7fe986a00640 1 --2- 172.21.15.121:0/152747176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe974004380 0x7fe974004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.419 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.414+0000 7fe98fe00640 1 -- 172.21.15.121:0/152747176 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9740039e0 msgr2=0x7fe974003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:34.419 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.414+0000 7fe98fe00640 1 --2- 172.21.15.121:0/152747176 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9740039e0 0x7fe974003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.419 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.414+0000 7fe98fe00640 1 -- 172.21.15.121:0/152747176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe974004380 msgr2=0x7fe974004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:34.420 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.414+0000 7f9e65600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9e60162b20 0x7f9e600c7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.420 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.414+0000 7f9e65600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9e600d19e0 0x7f9e600d3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.420 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.414+0000 7f29a0600640 1 -- start start 2024-04-20T10:21:34.420 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.414+0000 7fe69be00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe69c0d4270 0x7fe69c0d1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54366/0 (socket says 172.21.15.121:54366) 2024-04-20T10:21:34.420 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.414+0000 7fe69be00640 1 -- 172.21.15.121:0/3394054623 learned_addr learned my addr 172.21.15.121:0/3394054623 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:34.421 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.415+0000 7fdc4e000640 1 -- 172.21.15.121:0/3210786826 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc40035020 con 0x7fdc500d1a00 2024-04-20T10:21:34.421 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.414+0000 7fe69b400640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fe69c0d4650 0x7fe69c0b1290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:50108/0 (socket says 172.21.15.121:50108) 2024-04-20T10:21:34.421 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.414+0000 7fe6a1000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe69c0d0380 0x7fe69c0d07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.421 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.415+0000 7fe98fe00640 1 --2- 172.21.15.121:0/152747176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe974004380 0x7fe974004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.421 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.415+0000 7fe98fe00640 1 -- 172.21.15.121:0/152747176 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9900ae390 con 0x7fe974005010 2024-04-20T10:21:34.422 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.415+0000 7f9e65600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e600d3880 con 0x7f9e600d4fe0 2024-04-20T10:21:34.422 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.415+0000 7f9e65600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f9e600c79d0 con 0x7f9e60162b20 2024-04-20T10:21:34.422 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.415+0000 7f9e65600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f9e600d3b60 con 0x7f9e600d19e0 2024-04-20T10:21:34.422 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.414+0000 7f41da000640 1 -- 172.21.15.121:0/935832473 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f41d001a070 con 0x7f41dc03e830 2024-04-20T10:21:34.422 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.415+0000 7f9e5f400640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f9e600d4fe0 0x7f9e600d12d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.121:46712/0 (socket says 172.21.15.121:46712) 2024-04-20T10:21:34.422 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.415+0000 7f9e5f400640 1 -- 172.21.15.121:0/919226430 learned_addr learned my addr 172.21.15.121:0/919226430 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:34.423 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.415+0000 7fe98f400640 1 --2- 172.21.15.121:0/152747176 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9740039e0 0x7fe974003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:21:34.423 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.415+0000 7fe986a00640 1 --2- 172.21.15.121:0/152747176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe974004380 0x7fe974004860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:21:34.423 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.415+0000 7fdc4e000640 1 -- 172.21.15.121:0/3210786826 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc40031640 con 0x7fdc500d1a00 2024-04-20T10:21:34.423 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.415+0000 7fdc4e000640 1 -- 172.21.15.121:0/3210786826 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc40035020 con 0x7fdc500d1a00 2024-04-20T10:21:34.424 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.415+0000 7f9e5fe00640 1 --2- 172.21.15.121:0/919226430 >> v2:172.21.15.121:3300/0 conn(0x7f9e60162b20 0x7f9e600c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.424 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.416+0000 7f29a0600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f29980d4270 0x7f29980d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.424 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.416+0000 7fdc56600640 1 -- 172.21.15.121:0/3210786826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc500d1a00 msgr2=0x7fdc500b1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.424 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.416+0000 7fdc56600640 1 --2- 172.21.15.121:0/3210786826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc500d1a00 0x7fdc500b1290 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdc500b8060 tx=0x7fdc4002b8f0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.424 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.416+0000 7f29a0600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29980d3820 con 0x7f29980d0380 2024-04-20T10:21:34.425 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.416+0000 7f29a0600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29980d3b00 con 0x7f29980d4650 2024-04-20T10:21:34.425 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.416+0000 7f29a0600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f29980d40c0 con 0x7f29980d4270 2024-04-20T10:21:34.425 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.416+0000 7f3769000640 1 Processor -- start 2024-04-20T10:21:34.425 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.416+0000 7fe69be00640 1 -- 172.21.15.121:0/3394054623 >> v2:172.21.15.123:3300/0 conn(0x7fe69c0d0380 msgr2=0x7fe69c0d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:34.425 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.416+0000 7fe69be00640 1 --2- 172.21.15.121:0/3394054623 >> v2:172.21.15.123:3300/0 conn(0x7fe69c0d0380 0x7fe69c0d07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.425 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.416+0000 7fe69be00640 1 -- 172.21.15.121:0/3394054623 >> v1:172.21.15.121:6789/0 conn(0x7fe69c0d4650 legacy=0x7fe69c0b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.425 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.416+0000 7fe69be00640 1 -- 172.21.15.121:0/3394054623 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe69c0d3b00 con 0x7fe69c0d4270 2024-04-20T10:21:34.426 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.417+0000 7f3769000640 1 -- start start 2024-04-20T10:21:34.426 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.417+0000 7f299dc00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f29980d4270 0x7f29980d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.426 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.417+0000 7f299e600640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f29980d0380 0x7f29980d07d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:50120/0 (socket says 172.21.15.121:50120) 2024-04-20T10:21:34.426 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.417+0000 7f299e600640 1 -- 172.21.15.121:0/4147343630 learned_addr learned my addr 172.21.15.121:0/4147343630 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:34.427 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.417+0000 7f41da000640 1 --2- 172.21.15.121:0/935832473 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f41c406c9c0 0x7f41c406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.427 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.417+0000 7f41da000640 1 -- 172.21.15.121:0/935832473 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f41d008b6b0 con 0x7f41dc03e830 2024-04-20T10:21:34.427 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.417+0000 7f41e1e00640 1 --2- 172.21.15.121:0/935832473 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f41c406c9c0 0x7f41c406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.427 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.416+0000 7f9e5ea00640 1 -- 172.21.15.121:0/919226430 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1480847152 0 0) 0x7f9e600d3880 con 0x7f9e600d4fe0 2024-04-20T10:21:34.427 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.418+0000 7f299dc00640 1 -- 172.21.15.121:0/4147343630 >> v1:172.21.15.123:6789/0 conn(0x7f29980d4650 legacy=0x7f29980b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:34.428 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.417+0000 7f9e5ea00640 1 -- 172.21.15.121:0/919226430 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e44003520 con 0x7f9e600d4fe0 2024-04-20T10:21:34.428 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.417+0000 7f9e5fe00640 1 -- 172.21.15.121:0/919226430 >> v2:172.21.15.123:3300/0 conn(0x7f9e600d19e0 msgr2=0x7f9e600d3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:34.428 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.417+0000 7f9e5fe00640 1 --2- 172.21.15.121:0/919226430 >> v2:172.21.15.123:3300/0 conn(0x7f9e600d19e0 0x7f9e600d3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.428 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.417+0000 7f9e5fe00640 1 -- 172.21.15.121:0/919226430 >> v1:172.21.15.123:6789/0 conn(0x7f9e600d4fe0 legacy=0x7f9e600d12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.428 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.417+0000 7f9e5fe00640 1 -- 172.21.15.121:0/919226430 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e600d47e0 con 0x7f9e60162b20 2024-04-20T10:21:34.428 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.418+0000 7fdc56600640 1 -- 172.21.15.121:0/3210786826 shutdown_connections 2024-04-20T10:21:34.429 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.418+0000 7fdc56600640 1 --2- 172.21.15.121:0/3210786826 >> v2:172.21.15.121:3300/0 conn(0x7fdc500b17d0 0x7fdc500d0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.429 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.418+0000 7fdc56600640 1 --2- 172.21.15.121:0/3210786826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc500d1a00 0x7fdc500b1290 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.429 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.418+0000 7fdc56600640 1 -- 172.21.15.121:0/3210786826 >> 172.21.15.121:0/3210786826 conn(0x7fdc500b99c0 msgr2=0x7fdc500b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:34.429 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.418+0000 7fdc56600640 1 -- 172.21.15.121:0/3210786826 shutdown_connections 2024-04-20T10:21:34.429 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.419+0000 7fdc56600640 1 -- 172.21.15.121:0/3210786826 wait complete. 2024-04-20T10:21:34.429 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.418+0000 7f2997e00640 1 -- 172.21.15.121:0/4147343630 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1692457232 0 0) 0x7f29980d3820 con 0x7f29980d0380 2024-04-20T10:21:34.429 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.418+0000 7f299dc00640 1 -- 172.21.15.121:0/4147343630 >> v1:172.21.15.121:6789/0 conn(0x7f29980d0380 legacy=0x7f29980d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.429 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.418+0000 7f299dc00640 1 -- 172.21.15.121:0/4147343630 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29980d3de0 con 0x7f29980d4270 2024-04-20T10:21:34.430 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.419+0000 7fe69be00640 1 --2- 172.21.15.121:0/3394054623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe69c0d4270 0x7fe69c0d1a00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe69c068550 tx=0x7fe68c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5f0d96cd7206a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.430 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.419+0000 7f3769000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f376406cde0 0x7f37640cd930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.430 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.419+0000 7fe98fe00640 1 --2- 172.21.15.121:0/152747176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe974005010 0x7fe9740141d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe9900b1d20 tx=0x7fe974005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c243411891c3b256 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.430 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.419+0000 7f3769000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37640b3330 0x7f37640c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.431 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.419+0000 7f9e5fe00640 1 --2- 172.21.15.121:0/919226430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e60162b20 0x7f9e600c7130 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9e600c70c0 tx=0x7f9e5002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a94552bfe764e652 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.431 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.419+0000 7f41e1e00640 1 --2- 172.21.15.121:0/935832473 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f41c406c9c0 0x7f41c406ee80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f41d40011e0 tx=0x7f41d4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.431 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.419+0000 7f3769000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3764161d50 0x7f37641547c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.431 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.420+0000 7f3763400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f376406cde0 0x7f37640cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.432 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.420+0000 7fe98ea00640 1 -- 172.21.15.121:0/152747176 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98800a520 con 0x7fe974005010 2024-04-20T10:21:34.432 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.420+0000 7fe98ea00640 1 -- 172.21.15.121:0/152747176 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9880072f0 con 0x7fe974005010 2024-04-20T10:21:34.432 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.420+0000 7f9e5ea00640 1 -- 172.21.15.121:0/919226430 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e50035020 con 0x7f9e60162b20 2024-04-20T10:21:34.432 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.420+0000 7f9e5ea00640 1 -- 172.21.15.121:0/919226430 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e50031830 con 0x7f9e60162b20 2024-04-20T10:21:34.432 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.420+0000 7f9e5ea00640 1 -- 172.21.15.121:0/919226430 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e50031c00 con 0x7f9e60162b20 2024-04-20T10:21:34.433 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.420+0000 7f3763400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f376406cde0 0x7f37640cd930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.121:48902/0 (socket says 172.21.15.121:48902) 2024-04-20T10:21:34.433 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.420+0000 7f3763400640 1 -- 172.21.15.121:0/189863375 learned_addr learned my addr 172.21.15.121:0/189863375 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:34.433 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.420+0000 7f3769000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f37640b3880 con 0x7f37640b3330 2024-04-20T10:21:34.433 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.420+0000 7f3769000640 1 -- 172.21.15.121:0/189863375 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f37640d33c0 con 0x7f3764161d50 2024-04-20T10:21:34.433 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.420+0000 7f3769000640 1 -- 172.21.15.121:0/189863375 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f37640b3a00 con 0x7f376406cde0 2024-04-20T10:21:34.434 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.420+0000 7fe69aa00640 1 -- 172.21.15.121:0/3394054623 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c034070 con 0x7fe69c0d4270 2024-04-20T10:21:34.434 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.420+0000 7fe69aa00640 1 -- 172.21.15.121:0/3394054623 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe68c030860 con 0x7fe69c0d4270 2024-04-20T10:21:34.434 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.420+0000 7fe69aa00640 1 -- 172.21.15.121:0/3394054623 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c030c30 con 0x7fe69c0d4270 2024-04-20T10:21:34.434 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.420+0000 7fe98ea00640 1 -- 172.21.15.121:0/152747176 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe988007760 con 0x7fe974005010 2024-04-20T10:21:34.434 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.420+0000 7f375aa00640 1 --2- 172.21.15.121:0/189863375 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37640b3330 0x7f37640c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.434 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.420+0000 7f3763e00640 1 --2- 172.21.15.121:0/189863375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3764161d50 0x7f37641547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.435 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.420+0000 7f299dc00640 1 --2- 172.21.15.121:0/4147343630 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29980d4270 0x7f29980d1a00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f298c0097a0 tx=0x7f298c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d44d556a71139a8b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.435 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.421+0000 7fdc56600640 1 Processor -- start 2024-04-20T10:21:34.435 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.421+0000 7fe996400640 1 -- 172.21.15.121:0/152747176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe974005010 msgr2=0x7fe9740141d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.435 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.421+0000 7fe996400640 1 --2- 172.21.15.121:0/152747176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe974005010 0x7fe9740141d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe9900b1d20 tx=0x7fe974005830 comp rx=0 tx=0).stop 2024-04-20T10:21:34.435 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.421+0000 7fe98fe00640 1 -- 172.21.15.121:0/152747176 reap_dead start 2024-04-20T10:21:34.436 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.421+0000 7f2997e00640 1 -- 172.21.15.121:0/4147343630 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c035020 con 0x7f29980d4270 2024-04-20T10:21:34.436 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.421+0000 7fdc56600640 1 -- start start 2024-04-20T10:21:34.436 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.421+0000 7fe996400640 1 -- 172.21.15.121:0/152747176 shutdown_connections 2024-04-20T10:21:34.436 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.421+0000 7fe996400640 1 -- 172.21.15.121:0/152747176 >> 172.21.15.121:0/152747176 conn(0x7fe990072090 msgr2=0x7fe9900b4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:34.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.421+0000 7f2997e00640 1 -- 172.21.15.121:0/4147343630 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f298c031830 con 0x7f29980d4270 2024-04-20T10:21:34.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.421+0000 7f2997e00640 1 -- 172.21.15.121:0/4147343630 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c031c00 con 0x7f29980d4270 2024-04-20T10:21:34.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.421+0000 7f375aa00640 1 -- 172.21.15.121:0/189863375 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f376406cde0 msgr2=0x7f37640cd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.421+0000 7f375aa00640 1 --2- 172.21.15.121:0/189863375 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f376406cde0 0x7f37640cd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.421+0000 7f375aa00640 1 -- 172.21.15.121:0/189863375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3764161d50 msgr2=0x7f37641547c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.421+0000 7f375aa00640 1 --2- 172.21.15.121:0/189863375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3764161d50 0x7f37641547c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.421+0000 7f375aa00640 1 -- 172.21.15.121:0/189863375 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3764154d00 con 0x7f37640b3330 2024-04-20T10:21:34.438 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.421+0000 7f9e65600640 1 -- 172.21.15.121:0/919226430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e60162b20 msgr2=0x7f9e600c7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.438 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.421+0000 7f9e65600640 1 --2- 172.21.15.121:0/919226430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e60162b20 0x7f9e600c7130 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9e600c70c0 tx=0x7f9e5002b8d0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.438 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.422+0000 7f375aa00640 1 --2- 172.21.15.121:0/189863375 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37640b3330 0x7f37640c9530 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f374c00ca70 tx=0x7f374c00cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.438 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.422+0000 7fe996400640 1 -- 172.21.15.121:0/152747176 shutdown_connections 2024-04-20T10:21:34.439 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.422+0000 7fe996400640 1 -- 172.21.15.121:0/152747176 wait complete. 2024-04-20T10:21:34.439 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.422+0000 7f375be00640 1 -- 172.21.15.121:0/189863375 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c018070 con 0x7f37640b3330 2024-04-20T10:21:34.439 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.422+0000 7f29a0600640 1 -- 172.21.15.121:0/4147343630 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29980d4270 msgr2=0x7f29980d1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.439 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.422+0000 7f29a0600640 1 --2- 172.21.15.121:0/4147343630 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29980d4270 0x7f29980d1a00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f298c0097a0 tx=0x7f298c02b8d0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.439 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.423+0000 7f3769000640 1 -- 172.21.15.121:0/189863375 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37640c8490 con 0x7f37640b3330 2024-04-20T10:21:34.440 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.422+0000 7fe6a3000640 1 -- 172.21.15.121:0/3394054623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe69c0d4270 msgr2=0x7fe69c0d1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.440 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.422+0000 7f9e65600640 1 -- 172.21.15.121:0/919226430 shutdown_connections 2024-04-20T10:21:34.440 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.422+0000 7f9e65600640 1 --2- 172.21.15.121:0/919226430 >> v2:172.21.15.123:3300/0 conn(0x7f9e600d19e0 0x7f9e600d3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.440 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.422+0000 7f9e65600640 1 --2- 172.21.15.121:0/919226430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e60162b20 0x7f9e600c7130 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.440 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.422+0000 7f9e65600640 1 -- 172.21.15.121:0/919226430 >> 172.21.15.121:0/919226430 conn(0x7f9e6009d6f0 msgr2=0x7f9e600b9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:34.440 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.423+0000 7f9e65600640 1 -- 172.21.15.121:0/919226430 shutdown_connections 2024-04-20T10:21:34.441 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.422+0000 7f29a0600640 1 -- 172.21.15.121:0/4147343630 shutdown_connections 2024-04-20T10:21:34.441 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.422+0000 7f29a0600640 1 --2- 172.21.15.121:0/4147343630 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29980d4270 0x7f29980d1a00 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.441 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.422+0000 7f29a0600640 1 -- 172.21.15.121:0/4147343630 >> 172.21.15.121:0/4147343630 conn(0x7f29980b99c0 msgr2=0x7f29980b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:34.441 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.423+0000 7fdc56600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc500b17d0 0x7fdc501f49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.441 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.423+0000 7fdc56600640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdc500d1a00 0x7fdc500b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.441 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.423+0000 7fdc56600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc500d4270 0x7fdc50158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.442 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.423+0000 7f375be00640 1 -- 172.21.15.121:0/189863375 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f374c007500 con 0x7f37640b3330 2024-04-20T10:21:34.442 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.424+0000 7f375be00640 1 -- 172.21.15.121:0/189863375 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374c004a20 con 0x7f37640b3330 2024-04-20T10:21:34.442 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.424+0000 7f3769000640 1 -- 172.21.15.121:0/189863375 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37640c8a50 con 0x7f37640b3330 2024-04-20T10:21:34.442 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.424+0000 7fe996400640 1 Processor -- start 2024-04-20T10:21:34.442 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.424+0000 7fe996400640 1 -- start start 2024-04-20T10:21:34.443 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.424+0000 7f29a0600640 1 -- 172.21.15.121:0/4147343630 shutdown_connections 2024-04-20T10:21:34.443 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.424+0000 7f29a0600640 1 -- 172.21.15.121:0/4147343630 wait complete. 2024-04-20T10:21:34.443 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.422+0000 7fe6a3000640 1 --2- 172.21.15.121:0/3394054623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe69c0d4270 0x7fe69c0d1a00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe69c068550 tx=0x7fe68c02b9a0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.443 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.422+0000 7fe6a3000640 1 -- 172.21.15.121:0/3394054623 shutdown_connections 2024-04-20T10:21:34.443 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.422+0000 7fe6a3000640 1 --2- 172.21.15.121:0/3394054623 >> v2:172.21.15.123:3300/0 conn(0x7fe69c0d0380 0x7fe69c0d07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.443 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.422+0000 7fe6a3000640 1 --2- 172.21.15.121:0/3394054623 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe69c0d4270 0x7fe69c0d1a00 secure :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe69c068550 tx=0x7fe68c02b9a0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.443 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.422+0000 7fe6a3000640 1 -- 172.21.15.121:0/3394054623 >> 172.21.15.121:0/3394054623 conn(0x7fe69c0b99c0 msgr2=0x7fe69c0b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:34.443 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.424+0000 7fdc4f400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc500b17d0 0x7fdc501f49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.423+0000 7f9e65600640 1 -- 172.21.15.121:0/919226430 wait complete. 2024-04-20T10:21:34.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.424+0000 7fdc4f400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc500b17d0 0x7fdc501f49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54420/0 (socket says 172.21.15.121:54420) 2024-04-20T10:21:34.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.424+0000 7fdc4ea00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdc500d1a00 0x7fdc500b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.424+0000 7fdc4ea00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdc500d1a00 0x7fdc500b51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.121:48914/0 (socket says 172.21.15.121:48914) 2024-04-20T10:21:34.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.424+0000 7fdc4fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc500d4270 0x7fdc50158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.424+0000 7fdc4fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc500d4270 0x7fdc50158e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.121:35362/0 (socket says 172.21.15.121:35362) 2024-04-20T10:21:34.445 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.425+0000 7fdc4f400640 1 -- 172.21.15.121:0/4015925061 learned_addr learned my addr 172.21.15.121:0/4015925061 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:34.445 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.425+0000 7f3769000640 1 -- 172.21.15.121:0/189863375 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3764075b60 con 0x7f37640b3330 2024-04-20T10:21:34.445 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.425+0000 7fdc56600640 1 -- 172.21.15.121:0/4015925061 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdc500d53b0 con 0x7fdc500b17d0 2024-04-20T10:21:34.445 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.425+0000 7fdc56600640 1 -- 172.21.15.121:0/4015925061 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdc500cfe60 con 0x7fdc500d4270 2024-04-20T10:21:34.445 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.425+0000 7fdc56600640 1 -- 172.21.15.121:0/4015925061 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fdc50161f70 con 0x7fdc500d1a00 2024-04-20T10:21:34.446 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.425+0000 7fe996400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe974005010 0x7fe99009af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.446 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.425+0000 7fe996400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9901ef650 0x7fe99009d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.446 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.425+0000 7fe996400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe99009ded0 0x7fe990147190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.446 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.425+0000 7fe996400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe9900ad210 con 0x7fe99009ded0 2024-04-20T10:21:34.446 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.425+0000 7fe996400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe99006bb80 con 0x7fe974005010 2024-04-20T10:21:34.446 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.425+0000 7fe996400640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe9900ad390 con 0x7fe9901ef650 2024-04-20T10:21:34.446 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.426+0000 7fe6a3000640 1 -- 172.21.15.121:0/3394054623 shutdown_connections 2024-04-20T10:21:34.447 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.426+0000 7f9e65600640 1 Processor -- start 2024-04-20T10:21:34.447 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.426+0000 7f9e65600640 1 -- start start 2024-04-20T10:21:34.447 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.426+0000 7fdc4f400640 1 -- 172.21.15.121:0/4015925061 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdc500d1a00 msgr2=0x7fdc500b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.447 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.426+0000 7fdc4f400640 1 --2- 172.21.15.121:0/4015925061 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdc500d1a00 0x7fdc500b51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.447 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.426+0000 7fdc4f400640 1 -- 172.21.15.121:0/4015925061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc500d4270 msgr2=0x7fdc50158e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.447 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.426+0000 7fdc4f400640 1 --2- 172.21.15.121:0/4015925061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc500d4270 0x7fdc50158e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.447 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.426+0000 7fdc4f400640 1 -- 172.21.15.121:0/4015925061 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc500b5720 con 0x7fdc500b17d0 2024-04-20T10:21:34.448 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.426+0000 7fe98fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe99009ded0 0x7fe990147190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.448 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.426+0000 7fe98fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe99009ded0 0x7fe990147190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54430/0 (socket says 172.21.15.121:54430) 2024-04-20T10:21:34.448 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.426+0000 7fe98fe00640 1 -- 172.21.15.121:0/3266943418 learned_addr learned my addr 172.21.15.121:0/3266943418 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:34.448 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.426+0000 7fe98f400640 1 --2- 172.21.15.121:0/3266943418 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe974005010 0x7fe99009af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.448 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.427+0000 7fdc4fe00640 1 --2- 172.21.15.121:0/4015925061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc500d4270 0x7fdc50158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:34.448 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.427+0000 7fdc4f400640 1 --2- 172.21.15.121:0/4015925061 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc500b17d0 0x7fdc501f49e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c00c3c0 tx=0x7fdc3c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.449 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.427+0000 7f29a0600640 1 Processor -- start 2024-04-20T10:21:34.449 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.427+0000 7f29a0600640 1 -- start start 2024-04-20T10:21:34.449 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.427+0000 7fdc47400640 1 -- 172.21.15.121:0/4015925061 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c0052d0 con 0x7fdc500b17d0 2024-04-20T10:21:34.449 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.426+0000 7fe6a3000640 1 -- 172.21.15.121:0/3394054623 wait complete. 2024-04-20T10:21:34.450 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.429+0000 7fe6a3000640 1 Processor -- start 2024-04-20T10:21:34.450 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.429+0000 7fe6a3000640 1 -- start start 2024-04-20T10:21:34.450 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.428+0000 7fdc47400640 1 -- 172.21.15.121:0/4015925061 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc3c005490 con 0x7fdc500b17d0 2024-04-20T10:21:34.450 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.428+0000 7fdc56600640 1 -- 172.21.15.121:0/4015925061 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc500c7f50 con 0x7fdc500b17d0 2024-04-20T10:21:34.450 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.428+0000 7fdc56600640 1 -- 172.21.15.121:0/4015925061 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc500c8510 con 0x7fdc500b17d0 2024-04-20T10:21:34.450 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.429+0000 7fdc47400640 1 -- 172.21.15.121:0/4015925061 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c007260 con 0x7fdc500b17d0 2024-04-20T10:21:34.450 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.427+0000 7fe98f400640 1 -- 172.21.15.121:0/3266943418 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9901ef650 msgr2=0x7fe99009d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.451 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.427+0000 7fe98f400640 1 --2- 172.21.15.121:0/3266943418 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9901ef650 0x7fe99009d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.451 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.427+0000 7fe98f400640 1 -- 172.21.15.121:0/3266943418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe99009ded0 msgr2=0x7fe990147190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.451 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.427+0000 7fe98f400640 1 --2- 172.21.15.121:0/3266943418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe99009ded0 0x7fe990147190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.451 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.427+0000 7fe98f400640 1 -- 172.21.15.121:0/3266943418 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe990145230 con 0x7fe974005010 2024-04-20T10:21:34.451 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.427+0000 7fe98fe00640 1 --2- 172.21.15.121:0/3266943418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe99009ded0 0x7fe990147190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:21:34.451 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.427+0000 7fe98f400640 1 --2- 172.21.15.121:0/3266943418 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe974005010 0x7fe99009af70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe9900b5700 tx=0x7fe980011e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.451 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.428+0000 7f9e65600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e600d19e0 0x7f9e60201170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.451 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.428+0000 7f9e65600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e600d4fe0 0x7f9e600cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.451 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.428+0000 7f9e65600640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e60162b20 0x7f9e600705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.451 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.429+0000 7f9e65600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9e600b5e00 con 0x7f9e600d4fe0 2024-04-20T10:21:34.452 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.429+0000 7f9e65600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9e60162f00 con 0x7f9e600d19e0 2024-04-20T10:21:34.452 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.429+0000 7f9e65600640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f9e600b5f80 con 0x7f9e60162b20 2024-04-20T10:21:34.452 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.429+0000 7f9e5fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e600d19e0 0x7f9e60201170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.452 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.428+0000 7f98be000640 1 Processor -- start 2024-04-20T10:21:34.452 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.428+0000 7f98be000640 1 -- start start 2024-04-20T10:21:34.452 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.429+0000 7f98be000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f98b80d4270 0x7f98b80d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.452 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.429+0000 7f98be000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f98b80d0380 0x7f98b80d07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.452 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.429+0000 7f98be000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98b80d0d10 con 0x7f98b80d4650 2024-04-20T10:21:34.452 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.429+0000 7f98be000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f98b80d40c0 con 0x7f98b80d4270 2024-04-20T10:21:34.453 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.429+0000 7f98be000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f98b80d3820 con 0x7f98b80d0380 2024-04-20T10:21:34.453 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.430+0000 7f98b6a00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f98b80d4650 0x7f98b80b1290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:50126/0 (socket says 172.21.15.121:50126) 2024-04-20T10:21:34.453 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.430+0000 7f98b6a00640 1 -- 172.21.15.121:0/870816675 learned_addr learned my addr 172.21.15.121:0/870816675 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:34.453 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.430+0000 7f29a0600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29980d0380 0x7f29980c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.453 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.429+0000 7f9e64c00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e60162b20 0x7f9e600705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.453 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.430+0000 7f98b7400640 1 --2- 172.21.15.121:0/870816675 >> v2:172.21.15.121:3300/0 conn(0x7f98b80d4270 0x7f98b80d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.454 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.429+0000 7f9e5fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e600d19e0 0x7f9e60201170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.121:35382/0 (socket says 172.21.15.121:35382) 2024-04-20T10:21:34.454 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.430+0000 7f9e5fe00640 1 -- 172.21.15.121:0/69392825 learned_addr learned my addr 172.21.15.121:0/69392825 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:34.454 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.430+0000 7f29a0600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29980d4270 0x7f299806ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.454 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.431+0000 7fe6a3000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe69c0d0380 0x7fe69c0cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.455 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.431+0000 7f29a0600640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29980d4650 0x7f299815d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.455 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.431+0000 7fe6a3000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe69c0d4650 0x7fe69c200960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.455 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.431+0000 7fe6a3000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe69c06e5d0 0x7fe69c0b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.455 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.431+0000 7fe6a3000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe69c0b19a0 con 0x7fe69c06e5d0 2024-04-20T10:21:34.455 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.431+0000 7fe6a3000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe69c0cfe60 con 0x7fe69c0d4650 2024-04-20T10:21:34.455 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.431+0000 7fe6a3000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe69c0d53b0 con 0x7fe69c0d0380 2024-04-20T10:21:34.456 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.428+0000 7fe987e00640 1 -- 172.21.15.121:0/3266943418 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe980019070 con 0x7fe974005010 2024-04-20T10:21:34.456 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.428+0000 7fe996400640 1 -- 172.21.15.121:0/3266943418 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe990145510 con 0x7fe974005010 2024-04-20T10:21:34.456 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.428+0000 7fe996400640 1 -- 172.21.15.121:0/3266943418 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe990145ad0 con 0x7fe974005010 2024-04-20T10:21:34.456 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.429+0000 7fe996400640 1 -- 172.21.15.121:0/3266943418 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe990068980 con 0x7fe974005010 2024-04-20T10:21:34.456 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.431+0000 7f98b6000640 1 -- 172.21.15.121:0/870816675 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1794201383 0 0) 0x7f98b80d0d10 con 0x7f98b80d4650 2024-04-20T10:21:34.456 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.431+0000 7f98b6000640 1 -- 172.21.15.121:0/870816675 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f989c003520 con 0x7f98b80d4650 2024-04-20T10:21:34.456 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.431+0000 7f29a0600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f29980d53b0 con 0x7f29980d0380 2024-04-20T10:21:34.457 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.431+0000 7f29a0600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f29980b19a0 con 0x7f29980d4270 2024-04-20T10:21:34.457 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.431+0000 7f29a0600640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f29980d4ae0 con 0x7f29980d4650 2024-04-20T10:21:34.457 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.431+0000 7fe987e00640 1 -- 172.21.15.121:0/3266943418 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe980004240 con 0x7fe974005010 2024-04-20T10:21:34.457 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.431+0000 7fe6a1000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe69c06e5d0 0x7fe69c0b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.457 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.431+0000 7fe987e00640 1 -- 172.21.15.121:0/3266943418 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98000f8e0 con 0x7fe974005010 2024-04-20T10:21:34.458 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.432+0000 7f299e600640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29980d4650 0x7f299815d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.458 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.432+0000 7f98b6000640 1 -- 172.21.15.121:0/870816675 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2857761223 0 0) 0x7f989c003520 con 0x7f98b80d4650 2024-04-20T10:21:34.458 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.432+0000 7f98b6000640 1 -- 172.21.15.121:0/870816675 >> v2:172.21.15.123:3300/0 conn(0x7f98b80d0380 msgr2=0x7f98b80d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:34.458 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.432+0000 7f98b6000640 1 --2- 172.21.15.121:0/870816675 >> v2:172.21.15.123:3300/0 conn(0x7f98b80d0380 0x7f98b80d07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.458 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.432+0000 7f98b6000640 1 -- 172.21.15.121:0/870816675 >> v2:172.21.15.121:3300/0 conn(0x7f98b80d4270 msgr2=0x7f98b80d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:34.458 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.432+0000 7f98b6000640 1 --2- 172.21.15.121:0/870816675 >> v2:172.21.15.121:3300/0 conn(0x7f98b80d4270 0x7f98b80d1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.458 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.432+0000 7f98b6000640 1 -- 172.21.15.121:0/870816675 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98b80d3b00 con 0x7f98b80d4650 2024-04-20T10:21:34.458 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.432+0000 7f98b6000640 1 -- 172.21.15.121:0/870816675 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f98a4001670 con 0x7f98b80d4650 2024-04-20T10:21:34.459 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.432+0000 7f299e600640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29980d4650 0x7f299815d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.121:48950/0 (socket says 172.21.15.121:48950) 2024-04-20T10:21:34.459 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.432+0000 7f299d200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29980d4270 0x7f299806ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.459 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.432+0000 7f299e600640 1 -- 172.21.15.121:0/3291776127 learned_addr learned my addr 172.21.15.121:0/3291776127 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:34.459 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.432+0000 7fe69b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe69c0d4650 0x7fe69c200960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.459 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.432+0000 7fe69b400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe69c0d4650 0x7fe69c200960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.121:35422/0 (socket says 172.21.15.121:35422) 2024-04-20T10:21:34.459 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.432+0000 7fe69b400640 1 -- 172.21.15.121:0/1915970240 learned_addr learned my addr 172.21.15.121:0/1915970240 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:34.460 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.432+0000 7fe69be00640 1 --2- 172.21.15.121:0/1915970240 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe69c0d0380 0x7fe69c0cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.460 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.432+0000 7f98b6000640 1 -- 172.21.15.121:0/870816675 >> v1:172.21.15.121:6789/0 conn(0x7f98b80d4650 legacy=0x7f98b80b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.460 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.432+0000 7f98b6000640 1 --2- 172.21.15.121:0/870816675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f989c003a00 0x7f989c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.460 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.432+0000 7f98b6000640 1 --2- 172.21.15.121:0/870816675 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f989c004800 0x7f989c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.460 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.432+0000 7f98b6000640 1 --2- 172.21.15.121:0/870816675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f989c0051c0 0x7f989c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.460 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.433+0000 7f98b6000640 1 -- 172.21.15.121:0/870816675 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f98b80d40c0 con 0x7f989c003a00 2024-04-20T10:21:34.460 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.433+0000 7f98b6000640 1 -- 172.21.15.121:0/870816675 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f98b80d3820 con 0x7f989c0051c0 2024-04-20T10:21:34.460 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.433+0000 7f98b6000640 1 -- 172.21.15.121:0/870816675 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f989c00c140 con 0x7f989c004800 2024-04-20T10:21:34.461 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.433+0000 7f299dc00640 1 --2- 172.21.15.121:0/3291776127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29980d0380 0x7f29980c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.461 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.433+0000 7f98b7400640 1 --2- 172.21.15.121:0/870816675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f989c003a00 0x7f989c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.461 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.433+0000 7f98b6a00640 1 --2- 172.21.15.121:0/870816675 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f989c004800 0x7f989c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.461 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.433+0000 7f41da000640 1 -- 172.21.15.121:0/935832473 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f41d0021020 con 0x7f41dc03e830 2024-04-20T10:21:34.462 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.434+0000 7fe69be00640 1 -- 172.21.15.121:0/1915970240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe69c0d4650 msgr2=0x7fe69c200960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.462 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.434+0000 7fe69be00640 1 --2- 172.21.15.121:0/1915970240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe69c0d4650 0x7fe69c200960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.462 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.434+0000 7fe69be00640 1 -- 172.21.15.121:0/1915970240 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe69c06e5d0 msgr2=0x7fe69c0b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.462 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.434+0000 7fe69be00640 1 --2- 172.21.15.121:0/1915970240 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe69c06e5d0 0x7fe69c0b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.462 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.434+0000 7fe69be00640 1 -- 172.21.15.121:0/1915970240 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe69c0cbf50 con 0x7fe69c0d0380 2024-04-20T10:21:34.462 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.434+0000 7fe6a1000640 1 --2- 172.21.15.121:0/1915970240 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe69c06e5d0 0x7fe69c0b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:21:34.463 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.433+0000 7f9e5f400640 1 --2- 172.21.15.121:0/69392825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e600d4fe0 0x7f9e600cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.463 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.434+0000 7fe69be00640 1 --2- 172.21.15.121:0/1915970240 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe69c0d0380 0x7fe69c0cb6b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe68c033000 tx=0x7fe68c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.463 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.434+0000 7f98b7400640 1 -- 172.21.15.121:0/870816675 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f989c004800 msgr2=0x7f989c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:34.463 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.434+0000 7f98b7400640 1 --2- 172.21.15.121:0/870816675 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f989c004800 0x7f989c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.464 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.433+0000 7f299d200640 1 -- 172.21.15.121:0/3291776127 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29980d4650 msgr2=0x7f299815d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.464 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.433+0000 7f299d200640 1 --2- 172.21.15.121:0/3291776127 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29980d4650 0x7f299815d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.464 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.433+0000 7f299d200640 1 -- 172.21.15.121:0/3291776127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29980d0380 msgr2=0x7f29980c7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.464 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.433+0000 7f299d200640 1 --2- 172.21.15.121:0/3291776127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29980d0380 0x7f29980c7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.464 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.434+0000 7f299e600640 1 --2- 172.21.15.121:0/3291776127 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29980d4650 0x7f299815d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:21:34.464 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.433+0000 7f299d200640 1 -- 172.21.15.121:0/3291776127 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f299815d920 con 0x7f29980d4270 2024-04-20T10:21:34.464 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.434+0000 7f9e5fe00640 1 -- 172.21.15.121:0/69392825 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e60162b20 msgr2=0x7f9e600705f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.464 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.433+0000 7fdc56600640 1 -- 172.21.15.121:0/4015925061 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc28005290 con 0x7fdc500b17d0 2024-04-20T10:21:34.465 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.435+0000 7fdc47400640 1 -- 172.21.15.121:0/4015925061 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdc3c007400 con 0x7fdc500b17d0 2024-04-20T10:21:34.465 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.434+0000 7f9e5fe00640 1 --2- 172.21.15.121:0/69392825 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e60162b20 0x7f9e600705f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.465 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.435+0000 7fe693e00640 1 -- 172.21.15.121:0/1915970240 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c034040 con 0x7fe69c0d0380 2024-04-20T10:21:34.465 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.435+0000 7fe6a3000640 1 -- 172.21.15.121:0/1915970240 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe69c0cc230 con 0x7fe69c0d0380 2024-04-20T10:21:34.465 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.435+0000 7fe693e00640 1 -- 172.21.15.121:0/1915970240 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe68c030660 con 0x7fe69c0d0380 2024-04-20T10:21:34.465 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.435+0000 7fe6a3000640 1 -- 172.21.15.121:0/1915970240 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe69c0cc7f0 con 0x7fe69c0d0380 2024-04-20T10:21:34.466 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.432+0000 7f375be00640 1 -- 172.21.15.121:0/189863375 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f374c01b070 con 0x7f37640b3330 2024-04-20T10:21:34.466 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.434+0000 7f375be00640 1 --2- 172.21.15.121:0/189863375 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f374006c9c0 0x7f374006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.466 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.434+0000 7f375be00640 1 -- 172.21.15.121:0/189863375 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f374c08b850 con 0x7f37640b3330 2024-04-20T10:21:34.466 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.434+0000 7f3763400640 1 --2- 172.21.15.121:0/189863375 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f374006c9c0 0x7f374006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.466 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.435+0000 7f3763400640 1 --2- 172.21.15.121:0/189863375 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f374006c9c0 0x7f374006ee80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3754001d70 tx=0x7f3754001d00 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.466 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.436+0000 7fdc47400640 1 --2- 172.21.15.121:0/4015925061 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fdc3006c9c0 0x7fdc3006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.434+0000 7f98b7400640 1 -- 172.21.15.121:0/870816675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f989c0051c0 msgr2=0x7f989c00b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-20T10:21:34.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.434+0000 7f98b7400640 1 --2- 172.21.15.121:0/870816675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f989c0051c0 0x7f989c00b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.434+0000 7f98b7400640 1 -- 172.21.15.121:0/870816675 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f989c003520 con 0x7f989c003a00 2024-04-20T10:21:34.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.434+0000 7f98b6a00640 1 --2- 172.21.15.121:0/870816675 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f989c004800 0x7f989c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:21:34.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.435+0000 7f98b7e00640 1 -- 172.21.15.121:0/870816675 reap_dead start 2024-04-20T10:21:34.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.435+0000 7f299d200640 1 --2- 172.21.15.121:0/3291776127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29980d4270 0x7f299806ba20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f298400c410 tx=0x7f298400c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.434+0000 7f9e5fe00640 1 -- 172.21.15.121:0/69392825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e600d4fe0 msgr2=0x7f9e600cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.468 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.434+0000 7f9e5fe00640 1 --2- 172.21.15.121:0/69392825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e600d4fe0 0x7f9e600cc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.468 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.434+0000 7f9e5fe00640 1 -- 172.21.15.121:0/69392825 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e6006ca80 con 0x7f9e600d19e0 2024-04-20T10:21:34.468 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.435+0000 7f9e5fe00640 1 --2- 172.21.15.121:0/69392825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e600d19e0 0x7f9e60201170 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009890 tx=0x7f9e500046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.468 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.436+0000 7fdc47400640 1 -- 172.21.15.121:0/4015925061 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fdc3c08b1e0 con 0x7fdc500b17d0 2024-04-20T10:21:34.468 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.436+0000 7f98b7400640 1 --2- 172.21.15.121:0/870816675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f989c003a00 0x7f989c003e20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f98a8009d20 tx=0x7f989c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d35bcdcd12f04ae server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.469 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.437+0000 7fdc4ea00640 1 --2- 172.21.15.121:0/4015925061 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fdc3006c9c0 0x7fdc3006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.469 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.437+0000 7fe693e00640 1 -- 172.21.15.121:0/1915970240 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c039d90 con 0x7fe69c0d0380 2024-04-20T10:21:34.469 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.437+0000 7fe691600640 1 -- 172.21.15.121:0/1915970240 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe670005290 con 0x7fe69c0d0380 2024-04-20T10:21:34.469 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.437+0000 7fe987e00640 1 -- 172.21.15.121:0/3266943418 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe98001c070 con 0x7fe974005010 2024-04-20T10:21:34.469 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.437+0000 7fdc4ea00640 1 --2- 172.21.15.121:0/4015925061 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fdc3006c9c0 0x7fdc3006ee80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdc40036000 tx=0x7fdc40008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.470 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.438+0000 7fe987e00640 1 --2- 172.21.15.121:0/3266943418 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe96c06c920 0x7fe96c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.470 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.439+0000 7fe987e00640 1 -- 172.21.15.121:0/3266943418 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe98008c810 con 0x7fe974005010 2024-04-20T10:21:34.470 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.439+0000 7fe986a00640 1 --2- 172.21.15.121:0/3266943418 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe96c06c920 0x7fe96c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.470 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.441+0000 7fe986a00640 1 --2- 172.21.15.121:0/3266943418 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe96c06c920 0x7fe96c06ede0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe978007fd0 tx=0x7fe978007c80 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.471 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.440+0000 7fe693e00640 1 -- 172.21.15.121:0/1915970240 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe68c037070 con 0x7fe69c0d0380 2024-04-20T10:21:34.471 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.441+0000 7f375be00640 1 -- 172.21.15.121:0/189863375 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f374c059360 con 0x7f37640b3330 2024-04-20T10:21:34.471 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.442+0000 7fe693e00640 1 --2- 172.21.15.121:0/1915970240 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe67c06c920 0x7fe67c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.471 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.442+0000 7fe693e00640 1 -- 172.21.15.121:0/1915970240 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe68c0ae5e0 con 0x7fe69c0d0380 2024-04-20T10:21:34.471 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.442+0000 7fe69b400640 1 --2- 172.21.15.121:0/1915970240 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe67c06c920 0x7fe67c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.472 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.443+0000 7fe69b400640 1 --2- 172.21.15.121:0/1915970240 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe67c06c920 0x7fe67c06ede0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe69c0b6750 tx=0x7fe688007f10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.472 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.446+0000 7f98b6000640 1 -- 172.21.15.121:0/870816675 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a8039070 con 0x7f989c003a00 2024-04-20T10:21:34.472 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.446+0000 7f98b6000640 1 -- 172.21.15.121:0/870816675 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98a802c930 con 0x7f989c003a00 2024-04-20T10:21:34.472 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.446+0000 7f98b6000640 1 -- 172.21.15.121:0/870816675 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98a8009690 con 0x7f989c003a00 2024-04-20T10:21:34.472 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.447+0000 7fdc47400640 1 -- 172.21.15.121:0/4015925061 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdc500d1e90 con 0x7fdc500b17d0 2024-04-20T10:21:34.475 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.447+0000 7f98be000640 1 -- 172.21.15.121:0/870816675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f989c003a00 msgr2=0x7f989c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.475 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.447+0000 7f98be000640 1 --2- 172.21.15.121:0/870816675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f989c003a00 0x7f989c003e20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f98a8009d20 tx=0x7f989c005830 comp rx=0 tx=0).stop 2024-04-20T10:21:34.475 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.447+0000 7f98be000640 1 -- 172.21.15.121:0/870816675 shutdown_connections 2024-04-20T10:21:34.475 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.447+0000 7f98be000640 1 --2- 172.21.15.121:0/870816675 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f989c003a00 0x7f989c003e20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.476 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.447+0000 7f98be000640 1 -- 172.21.15.121:0/870816675 >> 172.21.15.121:0/870816675 conn(0x7f98b80b99c0 msgr2=0x7f98b80b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:34.476 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.448+0000 7f98be000640 1 -- 172.21.15.121:0/870816675 shutdown_connections 2024-04-20T10:21:34.476 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.448+0000 7f98be000640 1 -- 172.21.15.121:0/870816675 wait complete. 2024-04-20T10:21:34.476 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.450+0000 7f98be000640 1 Processor -- start 2024-04-20T10:21:34.476 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.451+0000 7f98be000640 1 -- start start 2024-04-20T10:21:34.476 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.452+0000 7f98be000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f989c003a00 0x7f98b80cd140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.476 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.452+0000 7f98be000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98b814f7c0 0x7f98b814fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.476 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.452+0000 7f98be000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98b80cb8e0 0x7f98b80cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.476 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.452+0000 7f98be000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f98b80d53b0 con 0x7f98b80cb8e0 2024-04-20T10:21:34.476 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.452+0000 7f98be000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f98b80cfe60 con 0x7f98b814f7c0 2024-04-20T10:21:34.476 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.452+0000 7f98be000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f98b80d4ae0 con 0x7f989c003a00 2024-04-20T10:21:34.476 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.453+0000 7f98b7e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98b80cb8e0 0x7f98b80cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.476 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.453+0000 7f98b7e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98b80cb8e0 0x7f98b80cbd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54500/0 (socket says 172.21.15.121:54500) 2024-04-20T10:21:34.476 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.453+0000 7f98b7e00640 1 -- 172.21.15.121:0/1519791427 learned_addr learned my addr 172.21.15.121:0/1519791427 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:34.476 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.453+0000 7f98b6a00640 1 --2- 172.21.15.121:0/1519791427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98b814f7c0 0x7f98b814fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.477 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.453+0000 7f98b7400640 1 --2- 172.21.15.121:0/1519791427 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f989c003a00 0x7f98b80cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.477 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.453+0000 7f98b7e00640 1 -- 172.21.15.121:0/1519791427 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f989c003a00 msgr2=0x7f98b80cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.477 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.454+0000 7f98b7e00640 1 --2- 172.21.15.121:0/1519791427 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f989c003a00 0x7f98b80cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.477 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.454+0000 7f98b7e00640 1 -- 172.21.15.121:0/1519791427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98b814f7c0 msgr2=0x7f98b814fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.477 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.454+0000 7f98b7e00640 1 --2- 172.21.15.121:0/1519791427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98b814f7c0 0x7f98b814fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.477 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.454+0000 7f98b7e00640 1 -- 172.21.15.121:0/1519791427 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98b8150160 con 0x7f98b80cb8e0 2024-04-20T10:21:34.477 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.454+0000 7f98b7400640 1 --2- 172.21.15.121:0/1519791427 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f989c003a00 0x7f98b80cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:21:34.477 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.454+0000 7f98b6a00640 1 --2- 172.21.15.121:0/1519791427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98b814f7c0 0x7f98b814fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:21:34.477 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.454+0000 7f98b7e00640 1 --2- 172.21.15.121:0/1519791427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98b80cb8e0 0x7f98b80cbd60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f98b80b74e0 tx=0x7f98b000cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.477 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.454+0000 7f98af400640 1 -- 172.21.15.121:0/1519791427 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b0017070 con 0x7f98b80cb8e0 2024-04-20T10:21:34.477 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.454+0000 7f98af400640 1 -- 172.21.15.121:0/1519791427 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98b000a7e0 con 0x7f98b80cb8e0 2024-04-20T10:21:34.477 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.454+0000 7f98af400640 1 -- 172.21.15.121:0/1519791427 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b000abd0 con 0x7f98b80cb8e0 2024-04-20T10:21:34.477 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.455+0000 7f98be000640 1 -- 172.21.15.121:0/1519791427 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98b80ccbf0 con 0x7f98b80cb8e0 2024-04-20T10:21:34.477 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.456+0000 7f98be000640 1 -- 172.21.15.121:0/1519791427 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98b80cc910 con 0x7f98b80cb8e0 2024-04-20T10:21:34.477 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.457+0000 7f98be000640 1 -- 172.21.15.121:0/1519791427 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98b8068980 con 0x7f98b80cb8e0 2024-04-20T10:21:34.478 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.458+0000 7f98af400640 1 -- 172.21.15.121:0/1519791427 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f98b001a070 con 0x7f98b80cb8e0 2024-04-20T10:21:34.478 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.458+0000 7f98af400640 1 --2- 172.21.15.121:0/1519791427 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f989806c970 0x7f989806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.478 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.459+0000 7f98af400640 1 -- 172.21.15.121:0/1519791427 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f98b008b4a0 con 0x7f98b80cb8e0 2024-04-20T10:21:34.478 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.459+0000 7f98b7400640 1 --2- 172.21.15.121:0/1519791427 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f989806c970 0x7f989806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.478 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.460+0000 7f98b7400640 1 --2- 172.21.15.121:0/1519791427 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f989806c970 0x7f989806ee30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f98a8036000 tx=0x7f98a8030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.478 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.463+0000 7f98af400640 1 -- 172.21.15.121:0/1519791427 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f98b0059670 con 0x7f98b80cb8e0 2024-04-20T10:21:34.478 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.466+0000 7f9e57e00640 1 -- 172.21.15.121:0/69392825 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e50040070 con 0x7f9e600d19e0 2024-04-20T10:21:34.478 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.466+0000 7f9e57e00640 1 -- 172.21.15.121:0/69392825 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e5003e070 con 0x7f9e600d19e0 2024-04-20T10:21:34.478 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.466+0000 7f9e57e00640 1 -- 172.21.15.121:0/69392825 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e50035040 con 0x7f9e600d19e0 2024-04-20T10:21:34.478 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.466+0000 7f9e65600640 1 -- 172.21.15.121:0/69392825 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e6006cd60 con 0x7f9e600d19e0 2024-04-20T10:21:34.479 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.466+0000 7f2995600640 1 -- 172.21.15.121:0/3291776127 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2984017070 con 0x7f29980d4270 2024-04-20T10:21:34.479 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.466+0000 7f2995600640 1 -- 172.21.15.121:0/3291776127 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2984005230 con 0x7f29980d4270 2024-04-20T10:21:34.479 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.466+0000 7f2995600640 1 -- 172.21.15.121:0/3291776127 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29840045e0 con 0x7f29980d4270 2024-04-20T10:21:34.479 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.466+0000 7f29a0600640 1 -- 172.21.15.121:0/3291776127 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f299806c190 con 0x7f29980d4270 2024-04-20T10:21:34.479 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.466+0000 7f9e65600640 1 -- 172.21.15.121:0/69392825 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e6006d320 con 0x7f9e600d19e0 2024-04-20T10:21:34.479 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.466+0000 7f29a0600640 1 -- 172.21.15.121:0/3291776127 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f299806c750 con 0x7f29980d4270 2024-04-20T10:21:34.480 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.468+0000 7f9e65600640 1 -- 172.21.15.121:0/69392825 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e60075b30 con 0x7f9e600d19e0 2024-04-20T10:21:34.480 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.468+0000 7fe693e00640 1 -- 172.21.15.121:0/1915970240 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe68c052020 con 0x7fe69c0d0380 2024-04-20T10:21:34.480 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.468+0000 7fe987e00640 1 -- 172.21.15.121:0/3266943418 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe98005a8e0 con 0x7fe974005010 2024-04-20T10:21:34.480 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.469+0000 7f29a0600640 1 -- 172.21.15.121:0/3291776127 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2974005290 con 0x7f29980d4270 2024-04-20T10:21:34.481 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.470+0000 7f9e57e00640 1 -- 172.21.15.121:0/69392825 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9e50038020 con 0x7f9e600d19e0 2024-04-20T10:21:34.481 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.472+0000 7f9e57e00640 1 --2- 172.21.15.121:0/69392825 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9e4006c9c0 0x7f9e4006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.481 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.472+0000 7f9e57e00640 1 -- 172.21.15.121:0/69392825 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9e500ada30 con 0x7f9e600d19e0 2024-04-20T10:21:34.481 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.472+0000 7f2995600640 1 -- 172.21.15.121:0/3291776127 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f298401a070 con 0x7f29980d4270 2024-04-20T10:21:34.482 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.473+0000 7f2995600640 1 --2- 172.21.15.121:0/3291776127 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f297806c9f0 0x7f297806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:34.482 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.473+0000 7f9e5f400640 1 --2- 172.21.15.121:0/69392825 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9e4006c9c0 0x7f9e4006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.482 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.474+0000 7f2995600640 1 -- 172.21.15.121:0/3291776127 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f298408b2e0 con 0x7f29980d4270 2024-04-20T10:21:34.482 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.474+0000 7f299dc00640 1 --2- 172.21.15.121:0/3291776127 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f297806c9f0 0x7f297806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:34.483 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.475+0000 7f9e5f400640 1 --2- 172.21.15.121:0/69392825 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9e4006c9c0 0x7f9e4006ee80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9e6006e600 tx=0x7f9e4c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.483 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.476+0000 7f299dc00640 1 --2- 172.21.15.121:0/3291776127 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f297806c9f0 0x7f297806eeb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f298c036000 tx=0x7f298c03c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:34.483 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.483+0000 7f9e57e00640 1 -- 172.21.15.121:0/69392825 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9e5007bb90 con 0x7f9e600d19e0 2024-04-20T10:21:34.496 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.496+0000 7f2995600640 1 -- 172.21.15.121:0/3291776127 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f299815d920 con 0x7f29980d4270 2024-04-20T10:21:34.838 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.838+0000 7f41e8c00640 1 -- 172.21.15.121:0/935832473 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f41dc000ce0 con 0x7f41dc03e830 2024-04-20T10:21:34.842 INFO:teuthology.orchestra.run.smithi121.stdout:60129542153 2024-04-20T10:21:34.842 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.839+0000 7f41da000640 1 -- 172.21.15.121:0/935832473 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f41dc0e1190 con 0x7f41dc03e830 2024-04-20T10:21:34.843 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.844+0000 7f41cf400640 1 -- 172.21.15.121:0/935832473 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f41c406c9c0 msgr2=0x7f41c406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.843 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.844+0000 7f41cf400640 1 --2- 172.21.15.121:0/935832473 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f41c406c9c0 0x7f41c406ee80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f41d40011e0 tx=0x7f41d4009040 comp rx=0 tx=0).stop 2024-04-20T10:21:34.844 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.844+0000 7f41cf400640 1 -- 172.21.15.121:0/935832473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f41dc03e830 msgr2=0x7f41dc0c6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.844 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.844+0000 7f41cf400640 1 --2- 172.21.15.121:0/935832473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f41dc03e830 0x7f41dc0c6fc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f41d000c390 tx=0x7f41d000c850 comp rx=0 tx=0).stop 2024-04-20T10:21:34.845 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.845+0000 7f41cf400640 1 -- 172.21.15.121:0/935832473 shutdown_connections 2024-04-20T10:21:34.845 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.845+0000 7f41cf400640 1 --2- 172.21.15.121:0/935832473 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41dc0e30b0 0x7f41dc04e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.845 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.845+0000 7f41cf400640 1 --2- 172.21.15.121:0/935832473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f41dc03e830 0x7f41dc0c6fc0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.845 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.845+0000 7f41cf400640 1 --2- 172.21.15.121:0/935832473 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f41c406c9c0 0x7f41c406ee80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.845 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.845+0000 7f41cf400640 1 --2- 172.21.15.121:0/935832473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41dc0078d0 0x7f41dc0df530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.845 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.845+0000 7f41cf400640 1 -- 172.21.15.121:0/935832473 >> 172.21.15.121:0/935832473 conn(0x7f41dc0559d0 msgr2=0x7f41dc0d9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:34.845 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.846+0000 7f41cf400640 1 -- 172.21.15.121:0/935832473 shutdown_connections 2024-04-20T10:21:34.846 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.846+0000 7f41cf400640 1 -- 172.21.15.121:0/935832473 wait complete. 2024-04-20T10:21:34.871 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542153 for osd.3 2024-04-20T10:21:34.881 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.881+0000 7f3769000640 1 -- 172.21.15.121:0/189863375 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f37640d38e0 con 0x7f37640b3330 2024-04-20T10:21:34.882 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.882+0000 7f375be00640 1 -- 172.21.15.121:0/189863375 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f374c05d660 con 0x7f37640b3330 2024-04-20T10:21:34.882 INFO:teuthology.orchestra.run.smithi121.stdout:60129542155 2024-04-20T10:21:34.883 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.883+0000 7fdc56600640 1 -- 172.21.15.121:0/4015925061 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fdc28005580 con 0x7fdc500b17d0 2024-04-20T10:21:34.883 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.884+0000 7fdc47400640 1 -- 172.21.15.121:0/4015925061 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdc3c058cf0 con 0x7fdc500b17d0 2024-04-20T10:21:34.884 INFO:teuthology.orchestra.run.smithi121.stdout:60129542154 2024-04-20T10:21:34.886 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.886+0000 7f3769000640 1 -- 172.21.15.121:0/189863375 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f374006c9c0 msgr2=0x7f374006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.886 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.886+0000 7f3769000640 1 --2- 172.21.15.121:0/189863375 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f374006c9c0 0x7f374006ee80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3754001d70 tx=0x7f3754001d00 comp rx=0 tx=0).stop 2024-04-20T10:21:34.886 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.886+0000 7f3769000640 1 -- 172.21.15.121:0/189863375 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37640b3330 msgr2=0x7f37640c9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.887 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.886+0000 7f3769000640 1 --2- 172.21.15.121:0/189863375 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37640b3330 0x7f37640c9530 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f374c00ca70 tx=0x7f374c00cf30 comp rx=0 tx=0).stop 2024-04-20T10:21:34.887 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.887+0000 7f3769000640 1 -- 172.21.15.121:0/189863375 shutdown_connections 2024-04-20T10:21:34.887 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.887+0000 7f3769000640 1 --2- 172.21.15.121:0/189863375 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f374006c9c0 0x7f374006ee80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.887 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.887+0000 7f3769000640 1 --2- 172.21.15.121:0/189863375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3764161d50 0x7f37641547c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.887 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.887+0000 7f3769000640 1 --2- 172.21.15.121:0/189863375 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f37640b3330 0x7f37640c9530 secure :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f374c00ca70 tx=0x7f374c00cf30 comp rx=0 tx=0).stop 2024-04-20T10:21:34.887 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.887+0000 7f3769000640 1 --2- 172.21.15.121:0/189863375 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f376406cde0 0x7f37640cd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.887 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.887+0000 7f3769000640 1 -- 172.21.15.121:0/189863375 >> 172.21.15.121:0/189863375 conn(0x7f37640d4450 msgr2=0x7f3764160ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:34.887 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.887+0000 7f3769000640 1 -- 172.21.15.121:0/189863375 shutdown_connections 2024-04-20T10:21:34.887 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.887+0000 7f3769000640 1 -- 172.21.15.121:0/189863375 wait complete. 2024-04-20T10:21:34.888 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.888+0000 7fdc56600640 1 -- 172.21.15.121:0/4015925061 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fdc3006c9c0 msgr2=0x7fdc3006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.888 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.888+0000 7fdc56600640 1 --2- 172.21.15.121:0/4015925061 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fdc3006c9c0 0x7fdc3006ee80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdc40036000 tx=0x7fdc40008030 comp rx=0 tx=0).stop 2024-04-20T10:21:34.888 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.888+0000 7fdc56600640 1 -- 172.21.15.121:0/4015925061 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc500b17d0 msgr2=0x7fdc501f49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.888 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.888+0000 7fdc56600640 1 --2- 172.21.15.121:0/4015925061 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc500b17d0 0x7fdc501f49e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c00c3c0 tx=0x7fdc3c00c880 comp rx=0 tx=0).stop 2024-04-20T10:21:34.889 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.888+0000 7fdc56600640 1 -- 172.21.15.121:0/4015925061 shutdown_connections 2024-04-20T10:21:34.889 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.888+0000 7fdc56600640 1 --2- 172.21.15.121:0/4015925061 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fdc3006c9c0 0x7fdc3006ee80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.889 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.888+0000 7fdc56600640 1 --2- 172.21.15.121:0/4015925061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc500d4270 0x7fdc50158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.889 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.888+0000 7fdc56600640 1 --2- 172.21.15.121:0/4015925061 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdc500d1a00 0x7fdc500b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.889 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.888+0000 7fdc56600640 1 --2- 172.21.15.121:0/4015925061 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdc500b17d0 0x7fdc501f49e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.889 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.888+0000 7fdc56600640 1 -- 172.21.15.121:0/4015925061 >> 172.21.15.121:0/4015925061 conn(0x7fdc500b99c0 msgr2=0x7fdc50160f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:34.889 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.888+0000 7fdc56600640 1 -- 172.21.15.121:0/4015925061 shutdown_connections 2024-04-20T10:21:34.889 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.889+0000 7fdc56600640 1 -- 172.21.15.121:0/4015925061 wait complete. 2024-04-20T10:21:34.900 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.901+0000 7f29a0600640 1 -- 172.21.15.121:0/3291776127 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f2974005850 con 0x7f29980d4270 2024-04-20T10:21:34.901 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.902+0000 7f2995600640 1 -- 172.21.15.121:0/3291776127 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2984058d60 con 0x7f29980d4270 2024-04-20T10:21:34.901 INFO:teuthology.orchestra.run.smithi121.stdout:60129542153 2024-04-20T10:21:34.903 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.904+0000 7f29a0600640 1 -- 172.21.15.121:0/3291776127 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f297806c9f0 msgr2=0x7f297806eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.904 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.904+0000 7f29a0600640 1 --2- 172.21.15.121:0/3291776127 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f297806c9f0 0x7f297806eeb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f298c036000 tx=0x7f298c03c040 comp rx=0 tx=0).stop 2024-04-20T10:21:34.904 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.904+0000 7f29a0600640 1 -- 172.21.15.121:0/3291776127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29980d4270 msgr2=0x7f299806ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.904 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.904+0000 7f29a0600640 1 --2- 172.21.15.121:0/3291776127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29980d4270 0x7f299806ba20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f298400c410 tx=0x7f298400c8d0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.904 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.904+0000 7f29a0600640 1 -- 172.21.15.121:0/3291776127 shutdown_connections 2024-04-20T10:21:34.904 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.904+0000 7f29a0600640 1 --2- 172.21.15.121:0/3291776127 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f29980d4650 0x7f299815d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.904 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.904+0000 7f29a0600640 1 --2- 172.21.15.121:0/3291776127 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f297806c9f0 0x7f297806eeb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.904 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.904+0000 7f29a0600640 1 --2- 172.21.15.121:0/3291776127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29980d4270 0x7f299806ba20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.905 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.904+0000 7f29a0600640 1 --2- 172.21.15.121:0/3291776127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f29980d0380 0x7f29980c7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.905 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.904+0000 7f29a0600640 1 -- 172.21.15.121:0/3291776127 >> 172.21.15.121:0/3291776127 conn(0x7f29980b99c0 msgr2=0x7f299815f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:34.905 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.905+0000 7f29a0600640 1 -- 172.21.15.121:0/3291776127 shutdown_connections 2024-04-20T10:21:34.905 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.905+0000 7f29a0600640 1 -- 172.21.15.121:0/3291776127 wait complete. 2024-04-20T10:21:34.905 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.906+0000 7f98be000640 1 -- 172.21.15.121:0/1519791427 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f98b815dca0 con 0x7f98b80cb8e0 2024-04-20T10:21:34.905 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.906+0000 7f98af400640 1 -- 172.21.15.121:0/1519791427 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f98b0058f70 con 0x7f98b80cb8e0 2024-04-20T10:21:34.908 INFO:teuthology.orchestra.run.smithi121.stdout:60129542153 2024-04-20T10:21:34.909 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.909+0000 7fe996400640 1 -- 172.21.15.121:0/3266943418 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fe9900b0ce0 con 0x7fe974005010 2024-04-20T10:21:34.909 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.910+0000 7f98be000640 1 -- 172.21.15.121:0/1519791427 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f989806c970 msgr2=0x7f989806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.909 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.910+0000 7f98be000640 1 --2- 172.21.15.121:0/1519791427 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f989806c970 0x7f989806ee30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f98a8036000 tx=0x7f98a8030000 comp rx=0 tx=0).stop 2024-04-20T10:21:34.910 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.910+0000 7f98be000640 1 -- 172.21.15.121:0/1519791427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98b80cb8e0 msgr2=0x7f98b80cbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.910 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.910+0000 7f98be000640 1 --2- 172.21.15.121:0/1519791427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98b80cb8e0 0x7f98b80cbd60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f98b80b74e0 tx=0x7f98b000cb10 comp rx=0 tx=0).stop 2024-04-20T10:21:34.910 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.910+0000 7fe987e00640 1 -- 172.21.15.121:0/3266943418 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe98005a1e0 con 0x7fe974005010 2024-04-20T10:21:34.910 INFO:teuthology.orchestra.run.smithi121.stdout:60129542153 2024-04-20T10:21:34.913 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.913+0000 7fe996400640 1 -- 172.21.15.121:0/3266943418 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe96c06c920 msgr2=0x7fe96c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.913 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.913+0000 7fe996400640 1 --2- 172.21.15.121:0/3266943418 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe96c06c920 0x7fe96c06ede0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe978007fd0 tx=0x7fe978007c80 comp rx=0 tx=0).stop 2024-04-20T10:21:34.913 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.913+0000 7fe996400640 1 -- 172.21.15.121:0/3266943418 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe974005010 msgr2=0x7fe99009af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.913 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.913+0000 7fe996400640 1 --2- 172.21.15.121:0/3266943418 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe974005010 0x7fe99009af70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe9900b5700 tx=0x7fe980011e30 comp rx=0 tx=0).stop 2024-04-20T10:21:34.913 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.913+0000 7fe996400640 1 -- 172.21.15.121:0/3266943418 shutdown_connections 2024-04-20T10:21:34.913 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.913+0000 7fe996400640 1 --2- 172.21.15.121:0/3266943418 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe99009ded0 0x7fe990147190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.913 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.913+0000 7fe996400640 1 --2- 172.21.15.121:0/3266943418 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe9901ef650 0x7fe99009d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.913 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.913+0000 7fe996400640 1 --2- 172.21.15.121:0/3266943418 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe96c06c920 0x7fe96c06ede0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.913 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.913+0000 7fe996400640 1 --2- 172.21.15.121:0/3266943418 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe974005010 0x7fe99009af70 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe9900b5700 tx=0x7fe980011e30 comp rx=0 tx=0).stop 2024-04-20T10:21:34.913 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.913+0000 7fe996400640 1 -- 172.21.15.121:0/3266943418 >> 172.21.15.121:0/3266943418 conn(0x7fe990072090 msgr2=0x7fe990158e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:34.914 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.911+0000 7f98be000640 1 -- 172.21.15.121:0/1519791427 shutdown_connections 2024-04-20T10:21:34.914 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.911+0000 7f98be000640 1 --2- 172.21.15.121:0/1519791427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f98b80cb8e0 0x7f98b80cbd60 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.914 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.911+0000 7f98be000640 1 --2- 172.21.15.121:0/1519791427 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f989806c970 0x7f989806ee30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.914 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.911+0000 7f98be000640 1 --2- 172.21.15.121:0/1519791427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98b814f7c0 0x7f98b814fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.914 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.911+0000 7f98be000640 1 --2- 172.21.15.121:0/1519791427 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f989c003a00 0x7f98b80cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.914 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.911+0000 7f98be000640 1 -- 172.21.15.121:0/1519791427 >> 172.21.15.121:0/1519791427 conn(0x7f98b80b99c0 msgr2=0x7f98b815e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:34.914 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.912+0000 7f98be000640 1 -- 172.21.15.121:0/1519791427 shutdown_connections 2024-04-20T10:21:34.914 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.912+0000 7f98be000640 1 -- 172.21.15.121:0/1519791427 wait complete. 2024-04-20T10:21:34.915 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.914+0000 7fe996400640 1 -- 172.21.15.121:0/3266943418 shutdown_connections 2024-04-20T10:21:34.915 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.914+0000 7fe996400640 1 -- 172.21.15.121:0/3266943418 wait complete. 2024-04-20T10:21:34.916 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542156 got 60129542155 for osd.1 2024-04-20T10:21:34.916 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.915+0000 7fe691600640 1 -- 172.21.15.121:0/1915970240 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fe670005850 con 0x7fe69c0d0380 2024-04-20T10:21:34.917 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542154 for osd.6 2024-04-20T10:21:34.917 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.916+0000 7fe693e00640 1 -- 172.21.15.121:0/1915970240 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe68c07c070 con 0x7fe69c0d0380 2024-04-20T10:21:34.917 INFO:teuthology.orchestra.run.smithi121.stdout:60129542155 2024-04-20T10:21:34.919 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.919+0000 7fe691600640 1 -- 172.21.15.121:0/1915970240 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe67c06c920 msgr2=0x7fe67c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.919 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.919+0000 7fe691600640 1 --2- 172.21.15.121:0/1915970240 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe67c06c920 0x7fe67c06ede0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe69c0b6750 tx=0x7fe688007f10 comp rx=0 tx=0).stop 2024-04-20T10:21:34.919 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.919+0000 7fe691600640 1 -- 172.21.15.121:0/1915970240 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe69c0d0380 msgr2=0x7fe69c0cb6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.919 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.919+0000 7fe691600640 1 --2- 172.21.15.121:0/1915970240 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe69c0d0380 0x7fe69c0cb6b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe68c033000 tx=0x7fe68c004730 comp rx=0 tx=0).stop 2024-04-20T10:21:34.920 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.920+0000 7fe691600640 1 -- 172.21.15.121:0/1915970240 shutdown_connections 2024-04-20T10:21:34.920 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.920+0000 7fe691600640 1 --2- 172.21.15.121:0/1915970240 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe69c06e5d0 0x7fe69c0b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.920 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.920+0000 7fe691600640 1 --2- 172.21.15.121:0/1915970240 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe67c06c920 0x7fe67c06ede0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.920 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.920+0000 7fe691600640 1 --2- 172.21.15.121:0/1915970240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe69c0d4650 0x7fe69c200960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.920 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.920+0000 7fe691600640 1 --2- 172.21.15.121:0/1915970240 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe69c0d0380 0x7fe69c0cb6b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.920 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.920+0000 7fe691600640 1 -- 172.21.15.121:0/1915970240 >> 172.21.15.121:0/1915970240 conn(0x7fe69c0b99c0 msgr2=0x7fe69c15eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:34.920 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.920+0000 7fe691600640 1 -- 172.21.15.121:0/1915970240 shutdown_connections 2024-04-20T10:21:34.920 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.920+0000 7fe691600640 1 -- 172.21.15.121:0/1915970240 wait complete. 2024-04-20T10:21:34.922 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.923+0000 7f9e65600640 1 -- 172.21.15.121:0/69392825 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f9e600caee0 con 0x7f9e600d19e0 2024-04-20T10:21:34.924 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.925+0000 7f9e57e00640 1 -- 172.21.15.121:0/69392825 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9e5007b490 con 0x7f9e600d19e0 2024-04-20T10:21:34.924 INFO:teuthology.orchestra.run.smithi121.stdout:60129542153 2024-04-20T10:21:34.927 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542153 for osd.5 2024-04-20T10:21:34.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.928+0000 7f9e65600640 1 -- 172.21.15.121:0/69392825 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9e4006c9c0 msgr2=0x7f9e4006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.928+0000 7f9e65600640 1 --2- 172.21.15.121:0/69392825 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9e4006c9c0 0x7f9e4006ee80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9e6006e600 tx=0x7f9e4c009040 comp rx=0 tx=0).stop 2024-04-20T10:21:34.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.928+0000 7f9e65600640 1 -- 172.21.15.121:0/69392825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e600d19e0 msgr2=0x7f9e60201170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:34.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.928+0000 7f9e65600640 1 --2- 172.21.15.121:0/69392825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e600d19e0 0x7f9e60201170 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009890 tx=0x7f9e500046c0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.928+0000 7f9e65600640 1 -- 172.21.15.121:0/69392825 shutdown_connections 2024-04-20T10:21:34.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.928+0000 7f9e65600640 1 --2- 172.21.15.121:0/69392825 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e60162b20 0x7f9e600705f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.928+0000 7f9e65600640 1 --2- 172.21.15.121:0/69392825 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e600d4fe0 0x7f9e600cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.928+0000 7f9e65600640 1 --2- 172.21.15.121:0/69392825 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9e4006c9c0 0x7f9e4006ee80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.928+0000 7f9e65600640 1 --2- 172.21.15.121:0/69392825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e600d19e0 0x7f9e60201170 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:34.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.928+0000 7f9e65600640 1 -- 172.21.15.121:0/69392825 >> 172.21.15.121:0/69392825 conn(0x7f9e6009d6f0 msgr2=0x7f9e600b6310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:34.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.928+0000 7f9e65600640 1 -- 172.21.15.121:0/69392825 shutdown_connections 2024-04-20T10:21:34.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:34.928+0000 7f9e65600640 1 -- 172.21.15.121:0/69392825 wait complete. 2024-04-20T10:21:34.934 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542153 for osd.7 2024-04-20T10:21:34.936 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542153 for osd.4 2024-04-20T10:21:34.939 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542156 got 60129542155 for osd.0 2024-04-20T10:21:34.947 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542153 for osd.2 2024-04-20T10:21:35.873 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-20T10:21:35.917 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-20T10:21:35.917 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-20T10:21:35.927 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-20T10:21:35.935 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-20T10:21:35.937 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-20T10:21:35.940 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-20T10:21:35.947 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-20T10:21:36.134 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.135+0000 7fbd6b200640 1 Processor -- start 2024-04-20T10:21:36.137 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.135+0000 7fbd6b200640 1 -- start start 2024-04-20T10:21:36.137 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.137+0000 7fbd6b200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fbd6c15ea00 0x7fbd6c15ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.137 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.138+0000 7fbd69e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fbd6c15ea00 0x7fbd6c15ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.138 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.138+0000 7fbd69e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fbd6c15ea00 0x7fbd6c15ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54504/0 (socket says 172.21.15.121:54504) 2024-04-20T10:21:36.138 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.139+0000 7fbd69e00640 1 -- 172.21.15.121:0/1502303145 learned_addr learned my addr 172.21.15.121:0/1502303145 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:36.139 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.139+0000 7fbd6b200640 1 -- 172.21.15.121:0/1502303145 --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd6c0acff0 con 0x7fbd6c15b490 2024-04-20T10:21:36.139 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.139+0000 7fbd6b200640 1 -- 172.21.15.121:0/1502303145 --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd6c0ad2d0 con 0x7fbd6c0ccfe0 2024-04-20T10:21:36.139 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.139+0000 7fbd6b200640 1 -- 172.21.15.121:0/1502303145 --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fbd6c0ad5b0 con 0x7fbd6c15ea00 2024-04-20T10:21:36.140 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.141+0000 7fbd68a00640 1 -- 172.21.15.121:0/1502303145 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3572170724 0 0) 0x7fbd6c0ad2d0 con 0x7fbd6c0ccfe0 2024-04-20T10:21:36.140 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.141+0000 7fbd68a00640 1 -- 172.21.15.121:0/1502303145 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd50003520 con 0x7fbd6c0ccfe0 2024-04-20T10:21:36.145 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.141+0000 7fbd68a00640 1 -- 172.21.15.121:0/1502303145 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2414758644 0 0) 0x7fbd6c0acff0 con 0x7fbd6c15b490 2024-04-20T10:21:36.145 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.142+0000 7fbd69e00640 1 -- 172.21.15.121:0/1502303145 >> v1:172.21.15.123:6789/0 conn(0x7fbd6c0ccfe0 legacy=0x7fbd6c06c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.146 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.142+0000 7fbd69e00640 1 -- 172.21.15.121:0/1502303145 >> v1:172.21.15.121:6789/0 conn(0x7fbd6c15b490 legacy=0x7fbd6c0cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.146 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.142+0000 7fbd69e00640 1 -- 172.21.15.121:0/1502303145 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd6c0ad890 con 0x7fbd6c15ea00 2024-04-20T10:21:36.146 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.144+0000 7fbd69e00640 1 --2- 172.21.15.121:0/1502303145 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd6c15ea00 0x7fbd6c15ede0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c0097f0 tx=0x7fbd5c02b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d497f09735b0cb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:36.146 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.145+0000 7fbd68a00640 1 -- 172.21.15.121:0/1502303145 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c038070 con 0x7fbd6c15ea00 2024-04-20T10:21:36.146 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.145+0000 7fbd68a00640 1 -- 172.21.15.121:0/1502303145 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd5c033420 con 0x7fbd6c15ea00 2024-04-20T10:21:36.146 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.146+0000 7fbd6b200640 1 -- 172.21.15.121:0/1502303145 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd6c15ea00 msgr2=0x7fbd6c15ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.146 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.146+0000 7fbd68a00640 1 -- 172.21.15.121:0/1502303145 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c038070 con 0x7fbd6c15ea00 2024-04-20T10:21:36.146 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.146+0000 7fbd6b200640 1 --2- 172.21.15.121:0/1502303145 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd6c15ea00 0x7fbd6c15ede0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c0097f0 tx=0x7fbd5c02b7d0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.147 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.147+0000 7fbd6b200640 1 -- 172.21.15.121:0/1502303145 shutdown_connections 2024-04-20T10:21:36.147 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.147+0000 7fbd6b200640 1 --2- 172.21.15.121:0/1502303145 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd6c15ea00 0x7fbd6c15ede0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.147 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.147+0000 7fbd6b200640 1 -- 172.21.15.121:0/1502303145 >> 172.21.15.121:0/1502303145 conn(0x7fbd6c0af6f0 msgr2=0x7fbd6c0b1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:36.149 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.147+0000 7fbd6b200640 1 -- 172.21.15.121:0/1502303145 shutdown_connections 2024-04-20T10:21:36.150 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.147+0000 7fbd6b200640 1 -- 172.21.15.121:0/1502303145 wait complete. 2024-04-20T10:21:36.150 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.150+0000 7fbd6b200640 1 Processor -- start 2024-04-20T10:21:36.153 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.151+0000 7fbd6b200640 1 -- start start 2024-04-20T10:21:36.153 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.153+0000 7fbd6b200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd6c15b490 0x7fbd6c200680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.153 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.153+0000 7fbd6b200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd6c15ea00 0x7fbd6c09c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.153 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.153+0000 7fbd6b200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd6c09b3d0 0x7fbd6c09c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.153 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.154+0000 7fbd69400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd6c15ea00 0x7fbd6c09c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.154 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.154+0000 7fbd69400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd6c15ea00 0x7fbd6c09c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54514/0 (socket says 172.21.15.121:54514) 2024-04-20T10:21:36.154 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.155+0000 7fbd69400640 1 -- 172.21.15.121:0/336578120 learned_addr learned my addr 172.21.15.121:0/336578120 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:36.154 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.155+0000 7fbd6b200640 1 -- 172.21.15.121:0/336578120 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6c06c710 con 0x7fbd6c15ea00 2024-04-20T10:21:36.155 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.155+0000 7fbd6b200640 1 -- 172.21.15.121:0/336578120 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbd6c15bb20 con 0x7fbd6c15b490 2024-04-20T10:21:36.155 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.155+0000 7fbd6a800640 1 --2- 172.21.15.121:0/336578120 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd6c09b3d0 0x7fbd6c09c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.155 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.155+0000 7fbd69e00640 1 --2- 172.21.15.121:0/336578120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd6c15b490 0x7fbd6c200680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.155 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.156+0000 7fbd6b200640 1 -- 172.21.15.121:0/336578120 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fbd6c06c890 con 0x7fbd6c09b3d0 2024-04-20T10:21:36.157 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.157+0000 7fbd6a800640 1 -- 172.21.15.121:0/336578120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd6c15b490 msgr2=0x7fbd6c200680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.157 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.157+0000 7fbd6a800640 1 --2- 172.21.15.121:0/336578120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd6c15b490 0x7fbd6c200680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.157 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.157+0000 7fbd6a800640 1 -- 172.21.15.121:0/336578120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd6c15ea00 msgr2=0x7fbd6c09c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.157 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.158+0000 7fbd6a800640 1 --2- 172.21.15.121:0/336578120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd6c15ea00 0x7fbd6c09c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.157 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.158+0000 7fbd6a800640 1 -- 172.21.15.121:0/336578120 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd6c09b950 con 0x7fbd6c09b3d0 2024-04-20T10:21:36.158 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.158+0000 7fbd69400640 1 --2- 172.21.15.121:0/336578120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd6c15ea00 0x7fbd6c09c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:21:36.159 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.159+0000 7fbd6a800640 1 --2- 172.21.15.121:0/336578120 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd6c09b3d0 0x7fbd6c09c9d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbd5800c7e0 tx=0x7fbd5800cca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:36.159 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.160+0000 7fbd62000640 1 -- 172.21.15.121:0/336578120 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd58017070 con 0x7fbd6c09b3d0 2024-04-20T10:21:36.160 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.160+0000 7fbd62000640 1 -- 172.21.15.121:0/336578120 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd5800a640 con 0x7fbd6c09b3d0 2024-04-20T10:21:36.160 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.161+0000 7fbd6b200640 1 -- 172.21.15.121:0/336578120 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd6c09bc30 con 0x7fbd6c09b3d0 2024-04-20T10:21:36.160 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.161+0000 7fbd6b200640 1 -- 172.21.15.121:0/336578120 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd6c0b2ed0 con 0x7fbd6c09b3d0 2024-04-20T10:21:36.160 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.161+0000 7fbd62000640 1 -- 172.21.15.121:0/336578120 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd5800a9f0 con 0x7fbd6c09b3d0 2024-04-20T10:21:36.162 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.163+0000 7fbd6b200640 1 -- 172.21.15.121:0/336578120 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd6c0721e0 con 0x7fbd6c09b3d0 2024-04-20T10:21:36.165 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.165+0000 7fbd62000640 1 -- 172.21.15.121:0/336578120 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbd5801a070 con 0x7fbd6c09b3d0 2024-04-20T10:21:36.166 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.167+0000 7fbd62000640 1 --2- 172.21.15.121:0/336578120 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fbd4c06c9f0 0x7fbd4c06eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.167 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.168+0000 7fbd69e00640 1 --2- 172.21.15.121:0/336578120 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fbd4c06c9f0 0x7fbd4c06eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.169 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.169+0000 7fbd69e00640 1 --2- 172.21.15.121:0/336578120 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fbd4c06c9f0 0x7fbd4c06eeb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c035000 tx=0x7fbd5c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:36.169 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.170+0000 7fbd62000640 1 -- 172.21.15.121:0/336578120 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbd5808b520 con 0x7fbd6c09b3d0 2024-04-20T10:21:36.179 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.179+0000 7fbd62000640 1 -- 172.21.15.121:0/336578120 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbd6c0721e0 con 0x7fbd6c09b3d0 2024-04-20T10:21:36.180 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.181+0000 7f593c800640 1 Processor -- start 2024-04-20T10:21:36.180 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.181+0000 7f593c800640 1 -- start start 2024-04-20T10:21:36.182 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.183+0000 7f0bb3a00640 1 Processor -- start 2024-04-20T10:21:36.183 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.184+0000 7f593c800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f59340d0380 0x7f59340d07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.183 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.183+0000 7f0bb3a00640 1 -- start start 2024-04-20T10:21:36.184 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.184+0000 7f593c800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59340d1e90 con 0x7f59340d4270 2024-04-20T10:21:36.184 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.184+0000 7f593c800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59340d0d10 con 0x7f59340d1a00 2024-04-20T10:21:36.184 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.184+0000 7f593c800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f59340d40c0 con 0x7f59340d0380 2024-04-20T10:21:36.184 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.184+0000 7f5939e00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f59340d4270 0x7f59340d4650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:47576/0 (socket says 172.21.15.121:47576) 2024-04-20T10:21:36.184 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.184+0000 7f5939e00640 1 -- 172.21.15.121:0/2648799108 learned_addr learned my addr 172.21.15.121:0/2648799108 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:36.185 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.185+0000 7f593a800640 1 --2- 172.21.15.121:0/2648799108 >> v2:172.21.15.123:3300/0 conn(0x7f59340d0380 0x7f59340d07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.185 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.185+0000 7f5938a00640 1 -- 172.21.15.121:0/2648799108 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4219475994 0 0) 0x7f5928001110 con 0x7f59340d4270 2024-04-20T10:21:36.185 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.186+0000 7f0bb3a00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0bac0d1a00 0x7f0bac0b1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.185 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.186+0000 7f0bb3a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0bac0b17d0 0x7f0bac0d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.186 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.185+0000 7f5938a00640 1 -- 172.21.15.121:0/2648799108 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f591c003520 con 0x7f59340d4270 2024-04-20T10:21:36.186 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.186+0000 7f5938a00640 1 -- 172.21.15.121:0/2648799108 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 898372503 0 0) 0x7f591c003520 con 0x7f59340d4270 2024-04-20T10:21:36.186 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.187+0000 7f0bb3a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bac0d1e60 con 0x7f0bac0d4270 2024-04-20T10:21:36.187 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.187+0000 7f0bb3a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f0bac0d40c0 con 0x7f0bac0b17d0 2024-04-20T10:21:36.187 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.187+0000 7f0bb3a00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f0bac0d3820 con 0x7f0bac0d1a00 2024-04-20T10:21:36.187 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.186+0000 7f0bb1000640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f0bac0d4270 0x7f0bac0d4650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.121:37094/0 (socket says 172.21.15.121:37094) 2024-04-20T10:21:36.187 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.187+0000 7f0bb1000640 1 -- 172.21.15.121:0/3625819677 learned_addr learned my addr 172.21.15.121:0/3625819677 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:36.187 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.187+0000 7f0babe00640 1 --2- 172.21.15.121:0/3625819677 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0bac0d1a00 0x7f0bac0b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.187 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.186+0000 7f5938a00640 1 -- 172.21.15.121:0/2648799108 >> v2:172.21.15.123:3300/0 conn(0x7f59340d0380 msgr2=0x7f59340d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:36.187 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.186+0000 7f5938a00640 1 --2- 172.21.15.121:0/2648799108 >> v2:172.21.15.123:3300/0 conn(0x7f59340d0380 0x7f59340d07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.188 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.186+0000 7f5938a00640 1 -- 172.21.15.121:0/2648799108 >> v1:172.21.15.123:6789/0 conn(0x7f59340d1a00 legacy=0x7f59340b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.188 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.186+0000 7f5938a00640 1 -- 172.21.15.121:0/2648799108 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59340d3820 con 0x7f59340d4270 2024-04-20T10:21:36.188 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.187+0000 7f0bb1a00640 1 --2- 172.21.15.121:0/3625819677 >> v2:172.21.15.121:3300/0 conn(0x7f0bac0b17d0 0x7f0bac0d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.189 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.186+0000 7f5938a00640 1 -- 172.21.15.121:0/2648799108 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f5928001820 con 0x7f59340d4270 2024-04-20T10:21:36.189 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.186+0000 7f5938a00640 1 -- 172.21.15.121:0/2648799108 >> v1:172.21.15.121:6789/0 conn(0x7f59340d4270 legacy=0x7f59340d4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.189 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.186+0000 7f5938a00640 1 --2- 172.21.15.121:0/2648799108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f591c0039d0 0x7f591c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.189 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.186+0000 7f5938a00640 1 --2- 172.21.15.121:0/2648799108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f591c004800 0x7f591c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.189 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.186+0000 7f5938a00640 1 --2- 172.21.15.121:0/2648799108 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f591c005210 0x7f591c0141d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.189 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.188+0000 7f0bab400640 1 -- 172.21.15.121:0/3625819677 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3756526377 0 0) 0x7f0bac0d1e60 con 0x7f0bac0d4270 2024-04-20T10:21:36.189 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.188+0000 7f0bab400640 1 -- 172.21.15.121:0/3625819677 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b94003520 con 0x7f0bac0d4270 2024-04-20T10:21:36.190 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.186+0000 7f5938a00640 1 -- 172.21.15.121:0/2648799108 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f59340d40c0 con 0x7f591c004800 2024-04-20T10:21:36.191 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.186+0000 7f5938a00640 1 -- 172.21.15.121:0/2648799108 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f591c014a00 con 0x7f591c0039d0 2024-04-20T10:21:36.191 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.186+0000 7f5938a00640 1 -- 172.21.15.121:0/2648799108 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f591c014c00 con 0x7f591c005210 2024-04-20T10:21:36.191 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.187+0000 7f5939400640 1 --2- 172.21.15.121:0/2648799108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f591c004800 0x7f591c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.191 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.188+0000 7f593a800640 1 --2- 172.21.15.121:0/2648799108 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f591c005210 0x7f591c0141d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.191 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.188+0000 7f5939e00640 1 --2- 172.21.15.121:0/2648799108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f591c0039d0 0x7f591c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.191 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.189+0000 7f5939400640 1 -- 172.21.15.121:0/2648799108 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f591c005210 msgr2=0x7f591c0141d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:36.191 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.189+0000 7f5939400640 1 --2- 172.21.15.121:0/2648799108 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f591c005210 0x7f591c0141d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.191 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.189+0000 7f5939400640 1 -- 172.21.15.121:0/2648799108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f591c0039d0 msgr2=0x7f591c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:36.191 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.189+0000 7f5939400640 1 --2- 172.21.15.121:0/2648799108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f591c0039d0 0x7f591c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.191 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.189+0000 7f5939400640 1 -- 172.21.15.121:0/2648799108 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59340d3820 con 0x7f591c004800 2024-04-20T10:21:36.191 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.190+0000 7f1099200640 1 Processor -- start 2024-04-20T10:21:36.192 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.190+0000 7f593a800640 1 -- 172.21.15.121:0/2648799108 reap_dead start 2024-04-20T10:21:36.192 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.189+0000 7f0bb1a00640 1 -- 172.21.15.121:0/3625819677 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0bac0d1a00 msgr2=0x7f0bac0b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:36.192 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.189+0000 7f0bb1a00640 1 --2- 172.21.15.121:0/3625819677 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0bac0d1a00 0x7f0bac0b1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.193 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.189+0000 7f0bb1a00640 1 -- 172.21.15.121:0/3625819677 >> v1:172.21.15.123:6789/0 conn(0x7f0bac0d4270 legacy=0x7f0bac0d4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.193 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.189+0000 7f0bb1a00640 1 -- 172.21.15.121:0/3625819677 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bac0d3b00 con 0x7f0bac0b17d0 2024-04-20T10:21:36.193 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.191+0000 7f5939400640 1 --2- 172.21.15.121:0/2648799108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f591c004800 0x7f591c004c80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5924001700 tx=0x7f591c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ace4b39d044b0c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:36.193 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.191+0000 7f0bb1a00640 1 --2- 172.21.15.121:0/3625819677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bac0b17d0 0x7f0bac0d0380 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4009d90 tx=0x7f0ba402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6459ee5006f3824e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:36.194 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.191+0000 7f5938a00640 1 -- 172.21.15.121:0/2648799108 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5924001350 con 0x7f591c004800 2024-04-20T10:21:36.194 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.191+0000 7f5938a00640 1 -- 172.21.15.121:0/2648799108 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59240052f0 con 0x7f591c004800 2024-04-20T10:21:36.194 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.192+0000 7f5938a00640 1 -- 172.21.15.121:0/2648799108 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5924003830 con 0x7f591c004800 2024-04-20T10:21:36.194 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.192+0000 7f0bab400640 1 -- 172.21.15.121:0/3625819677 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba4039070 con 0x7f0bac0b17d0 2024-04-20T10:21:36.195 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.190+0000 7f1099200640 1 -- start start 2024-04-20T10:21:36.195 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.192+0000 7f1099200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f108c0ead00 0x7f108c04fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.195 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.192+0000 7f1099200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f108c04ffd0 0x7f108c037bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.195 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.192+0000 7f1099200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f108c053720 con 0x7f108c038110 2024-04-20T10:21:36.195 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.192+0000 7f1099200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f108c054330 con 0x7f108c04ffd0 2024-04-20T10:21:36.195 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.192+0000 7f1099200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f108c053a00 con 0x7f108c0ead00 2024-04-20T10:21:36.195 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.193+0000 7f0bab400640 1 -- 172.21.15.121:0/3625819677 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ba402cc80 con 0x7f0bac0b17d0 2024-04-20T10:21:36.195 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.193+0000 7f0bab400640 1 -- 172.21.15.121:0/3625819677 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba4034660 con 0x7f0bac0b17d0 2024-04-20T10:21:36.196 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.193+0000 7f1093400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f108c0ead00 0x7f108c04fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.196 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.193+0000 7f593c800640 1 -- 172.21.15.121:0/2648799108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f591c004800 msgr2=0x7f591c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.196 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.193+0000 7f593c800640 1 --2- 172.21.15.121:0/2648799108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f591c004800 0x7f591c004c80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5924001700 tx=0x7f591c005830 comp rx=0 tx=0).stop 2024-04-20T10:21:36.196 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.193+0000 7f1093e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f108c038110 0x7f108c0385a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.121:37106/0 (socket says 172.21.15.121:37106) 2024-04-20T10:21:36.197 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.193+0000 7f1093e00640 1 -- 172.21.15.121:0/1125795255 learned_addr learned my addr 172.21.15.121:0/1125795255 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:36.197 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.193+0000 7f1092a00640 1 --2- 172.21.15.121:0/1125795255 >> v2:172.21.15.123:3300/0 conn(0x7f108c04ffd0 0x7f108c037bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.197 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.194+0000 7f593c800640 1 -- 172.21.15.121:0/2648799108 shutdown_connections 2024-04-20T10:21:36.197 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.194+0000 7f593c800640 1 --2- 172.21.15.121:0/2648799108 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f591c004800 0x7f591c004c80 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.197 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.194+0000 7f593c800640 1 -- 172.21.15.121:0/2648799108 >> 172.21.15.121:0/2648799108 conn(0x7f59340b99c0 msgr2=0x7f59340b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:36.197 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.194+0000 7f1093400640 1 -- 172.21.15.121:0/1125795255 >> v2:172.21.15.123:3300/0 conn(0x7f108c04ffd0 msgr2=0x7f108c037bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:36.198 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.194+0000 7f1093400640 1 --2- 172.21.15.121:0/1125795255 >> v2:172.21.15.123:3300/0 conn(0x7f108c04ffd0 0x7f108c037bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.198 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.194+0000 7f1093400640 1 -- 172.21.15.121:0/1125795255 >> v1:172.21.15.123:6789/0 conn(0x7f108c038110 legacy=0x7f108c0385a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.198 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.194+0000 7f1093400640 1 -- 172.21.15.121:0/1125795255 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f108c053ce0 con 0x7f108c0ead00 2024-04-20T10:21:36.198 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.195+0000 7f0bb3a00640 1 -- 172.21.15.121:0/3625819677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bac0b17d0 msgr2=0x7f0bac0d0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.198 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.195+0000 7f0bb3a00640 1 --2- 172.21.15.121:0/3625819677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bac0b17d0 0x7f0bac0d0380 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4009d90 tx=0x7f0ba402bed0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.199 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.195+0000 7f593c800640 1 -- 172.21.15.121:0/2648799108 shutdown_connections 2024-04-20T10:21:36.199 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.195+0000 7f593c800640 1 -- 172.21.15.121:0/2648799108 wait complete. 2024-04-20T10:21:36.199 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.195+0000 7f0bb3a00640 1 -- 172.21.15.121:0/3625819677 shutdown_connections 2024-04-20T10:21:36.199 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.195+0000 7f0bb3a00640 1 --2- 172.21.15.121:0/3625819677 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bac0b17d0 0x7f0bac0d0380 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.199 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.195+0000 7f0bb3a00640 1 --2- 172.21.15.121:0/3625819677 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0bac0d1a00 0x7f0bac0b1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.200 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.195+0000 7f0bb3a00640 1 -- 172.21.15.121:0/3625819677 >> 172.21.15.121:0/3625819677 conn(0x7f0bac0b99c0 msgr2=0x7f0bac0b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:36.200 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.195+0000 7f0bb3a00640 1 -- 172.21.15.121:0/3625819677 shutdown_connections 2024-04-20T10:21:36.200 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.195+0000 7f0bb3a00640 1 -- 172.21.15.121:0/3625819677 wait complete. 2024-04-20T10:21:36.200 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.196+0000 7f1093400640 1 --2- 172.21.15.121:0/1125795255 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f108c0ead00 0x7f108c04fa90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f109406a770 tx=0x7f10940a6fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=849169babb2552ef server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:36.200 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.197+0000 7f1092000640 1 -- 172.21.15.121:0/1125795255 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10940b4070 con 0x7f108c0ead00 2024-04-20T10:21:36.200 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.197+0000 7f1092000640 1 -- 172.21.15.121:0/1125795255 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10940a7dd0 con 0x7f108c0ead00 2024-04-20T10:21:36.201 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.197+0000 7f1092000640 1 -- 172.21.15.121:0/1125795255 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10940af640 con 0x7f108c0ead00 2024-04-20T10:21:36.201 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.198+0000 7f593c800640 1 Processor -- start 2024-04-20T10:21:36.201 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.198+0000 7f593c800640 1 -- start start 2024-04-20T10:21:36.202 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.198+0000 7f1099200640 1 -- 172.21.15.121:0/1125795255 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f108c0ead00 msgr2=0x7f108c04fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.202 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.198+0000 7f1099200640 1 --2- 172.21.15.121:0/1125795255 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f108c0ead00 0x7f108c04fa90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f109406a770 tx=0x7f10940a6fa0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.202 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.198+0000 7f1099200640 1 -- 172.21.15.121:0/1125795255 shutdown_connections 2024-04-20T10:21:36.202 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.198+0000 7f1099200640 1 --2- 172.21.15.121:0/1125795255 >> v2:172.21.15.123:3300/0 conn(0x7f108c04ffd0 0x7f108c037bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.202 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.198+0000 7f1099200640 1 --2- 172.21.15.121:0/1125795255 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f108c0ead00 0x7f108c04fa90 secure :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f109406a770 tx=0x7f10940a6fa0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.202 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.198+0000 7f1099200640 1 -- 172.21.15.121:0/1125795255 >> 172.21.15.121:0/1125795255 conn(0x7f108c009fb0 msgr2=0x7f108c00a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:36.202 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.199+0000 7f1099200640 1 -- 172.21.15.121:0/1125795255 shutdown_connections 2024-04-20T10:21:36.202 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.199+0000 7f0bb3a00640 1 Processor -- start 2024-04-20T10:21:36.203 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.199+0000 7f1099200640 1 -- 172.21.15.121:0/1125795255 wait complete. 2024-04-20T10:21:36.203 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.200+0000 7f0bb3a00640 1 -- start start 2024-04-20T10:21:36.204 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.200+0000 7f593c800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f591c004800 0x7f5934159110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.204 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.200+0000 7f593c800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5934159810 0x7f59340cc280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.204 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.200+0000 7f593c800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59340cd170 0x7f59340b5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.204 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.201+0000 7f593c800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f59340cfe60 con 0x7f5934159810 2024-04-20T10:21:36.204 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.201+0000 7f593c800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f59340d3de0 con 0x7f59340cd170 2024-04-20T10:21:36.204 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.201+0000 7f593c800640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f59340d53b0 con 0x7f591c004800 2024-04-20T10:21:36.204 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.201+0000 7f5939400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5934159810 0x7f59340cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.204 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.201+0000 7f5939400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5934159810 0x7f59340cc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58468/0 (socket says 172.21.15.121:58468) 2024-04-20T10:21:36.204 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.201+0000 7f5939400640 1 -- 172.21.15.121:0/1072898731 learned_addr learned my addr 172.21.15.121:0/1072898731 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:36.204 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.201+0000 7f5939e00640 1 --2- 172.21.15.121:0/1072898731 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f591c004800 0x7f5934159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.204 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.201+0000 7f593a800640 1 --2- 172.21.15.121:0/1072898731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59340cd170 0x7f59340b5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.205 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.201+0000 7f0bb3a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bac0b17d0 0x7f0bac0c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.205 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.201+0000 7f0bb3a00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0bac0d1a00 0x7f0bac1551a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.205 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.201+0000 7f0bb3a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0bac0d4270 0x7f0bac0b64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.206 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.201+0000 7f0bb3a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0bac0d53b0 con 0x7f0bac0b17d0 2024-04-20T10:21:36.206 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.201+0000 7f0bb3a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0bac0cfe60 con 0x7f0bac0d4270 2024-04-20T10:21:36.206 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.201+0000 7f0bb3a00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f0bac161f70 con 0x7f0bac0d1a00 2024-04-20T10:21:36.206 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.202+0000 7f0bb1000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bac0b17d0 0x7f0bac0c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.206 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.202+0000 7f0bb1000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bac0b17d0 0x7f0bac0c8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58482/0 (socket says 172.21.15.121:58482) 2024-04-20T10:21:36.206 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.202+0000 7f0bb1000640 1 -- 172.21.15.121:0/2659132409 learned_addr learned my addr 172.21.15.121:0/2659132409 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:36.206 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.202+0000 7f593a800640 1 -- 172.21.15.121:0/1072898731 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f591c004800 msgr2=0x7f5934159110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.206 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.202+0000 7f593a800640 1 --2- 172.21.15.121:0/1072898731 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f591c004800 0x7f5934159110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.207 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.202+0000 7f593a800640 1 -- 172.21.15.121:0/1072898731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5934159810 msgr2=0x7f59340cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.209 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.202+0000 7f0babe00640 1 --2- 172.21.15.121:0/2659132409 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0bac0d1a00 0x7f0bac1551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.210 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.204+0000 7f593a800640 1 --2- 172.21.15.121:0/1072898731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5934159810 0x7f59340cc280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.210 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.204+0000 7f593a800640 1 -- 172.21.15.121:0/1072898731 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59340cb7e0 con 0x7f59340cd170 2024-04-20T10:21:36.210 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.202+0000 7f0bb1a00640 1 --2- 172.21.15.121:0/2659132409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0bac0d4270 0x7f0bac0b64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.210 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.204+0000 7f0bb1000640 1 -- 172.21.15.121:0/2659132409 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0bac0d1a00 msgr2=0x7f0bac1551a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.210 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.204+0000 7f0bb1000640 1 --2- 172.21.15.121:0/2659132409 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0bac0d1a00 0x7f0bac1551a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.210 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.204+0000 7f0bb1000640 1 -- 172.21.15.121:0/2659132409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0bac0d4270 msgr2=0x7f0bac0b64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.210 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.204+0000 7f0bb1000640 1 --2- 172.21.15.121:0/2659132409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0bac0d4270 0x7f0bac0b64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.210 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.204+0000 7f0bb1000640 1 -- 172.21.15.121:0/2659132409 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bac0b6a30 con 0x7f0bac0b17d0 2024-04-20T10:21:36.211 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.204+0000 7f1099200640 1 Processor -- start 2024-04-20T10:21:36.211 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.204+0000 7f0bb1a00640 1 --2- 172.21.15.121:0/2659132409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0bac0d4270 0x7f0bac0b64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:36.211 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.204+0000 7f5939400640 1 --2- 172.21.15.121:0/1072898731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5934159810 0x7f59340cc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:21:36.211 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.204+0000 7f5939e00640 1 --2- 172.21.15.121:0/1072898731 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f591c004800 0x7f5934159110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:21:36.212 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.205+0000 7f1099200640 1 -- start start 2024-04-20T10:21:36.212 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.205+0000 7f0bb1000640 1 --2- 172.21.15.121:0/2659132409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bac0b17d0 0x7f0bac0c8d00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0ba000c390 tx=0x7f0ba000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:36.212 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.205+0000 7f593a800640 1 --2- 172.21.15.121:0/1072898731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59340cd170 0x7f59340b5b00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f593408d680 tx=0x7f5930007700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:36.213 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.206+0000 7f0ba8c00640 1 -- 172.21.15.121:0/2659132409 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0017070 con 0x7f0bac0b17d0 2024-04-20T10:21:36.213 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.206+0000 7f0ba8c00640 1 -- 172.21.15.121:0/2659132409 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ba0005ce0 con 0x7f0bac0b17d0 2024-04-20T10:21:36.213 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.206+0000 7f0ba8c00640 1 -- 172.21.15.121:0/2659132409 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba00072f0 con 0x7f0bac0b17d0 2024-04-20T10:21:36.213 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.206+0000 7f0bb3a00640 1 -- 172.21.15.121:0/2659132409 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bac06e320 con 0x7f0bac0b17d0 2024-04-20T10:21:36.213 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.206+0000 7f0bb3a00640 1 -- 172.21.15.121:0/2659132409 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bac0b5cf0 con 0x7f0bac0b17d0 2024-04-20T10:21:36.213 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.207+0000 7f1099200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f108c038110 0x7f108c0d4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.214 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.207+0000 7f1099200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f108c04ffd0 0x7f108c0ed680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.214 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.207+0000 7f1099200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f108c0edbc0 0x7f108c0dcb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.214 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.208+0000 7f1099200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f108c0547c0 con 0x7f108c038110 2024-04-20T10:21:36.214 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.208+0000 7f1099200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f108c038cb0 con 0x7f108c0edbc0 2024-04-20T10:21:36.214 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.208+0000 7f1099200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f108c054940 con 0x7f108c04ffd0 2024-04-20T10:21:36.214 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.207+0000 7f0bb3a00640 1 -- 172.21.15.121:0/2659132409 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bac068980 con 0x7f0bac0b17d0 2024-04-20T10:21:36.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.208+0000 7f1092a00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f108c04ffd0 0x7f108c0ed680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.208+0000 7f1093e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f108c0edbc0 0x7f108c0dcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.208+0000 7f1093e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f108c0edbc0 0x7f108c0dcb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.121:43094/0 (socket says 172.21.15.121:43094) 2024-04-20T10:21:36.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.208+0000 7f1093e00640 1 -- 172.21.15.121:0/3391878896 learned_addr learned my addr 172.21.15.121:0/3391878896 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:36.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.207+0000 7f592e000640 1 -- 172.21.15.121:0/1072898731 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5930007c80 con 0x7f59340cd170 2024-04-20T10:21:36.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.208+0000 7f592e000640 1 -- 172.21.15.121:0/1072898731 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f593001a070 con 0x7f59340cd170 2024-04-20T10:21:36.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.208+0000 7f592e000640 1 -- 172.21.15.121:0/1072898731 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5930015660 con 0x7f59340cd170 2024-04-20T10:21:36.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.208+0000 7f593c800640 1 -- 172.21.15.121:0/1072898731 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59340cdae0 con 0x7f59340cd170 2024-04-20T10:21:36.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.208+0000 7f593c800640 1 -- 172.21.15.121:0/1072898731 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59340cbd60 con 0x7f59340cd170 2024-04-20T10:21:36.217 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.209+0000 7f1093400640 1 --2- 172.21.15.121:0/3391878896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f108c038110 0x7f108c0d4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.217 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.209+0000 7f1093e00640 1 -- 172.21.15.121:0/3391878896 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f108c04ffd0 msgr2=0x7f108c0ed680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.217 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.209+0000 7f1093e00640 1 --2- 172.21.15.121:0/3391878896 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f108c04ffd0 0x7f108c0ed680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.217 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.209+0000 7f1093e00640 1 -- 172.21.15.121:0/3391878896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f108c038110 msgr2=0x7f108c0d4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.217 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.209+0000 7f1093e00640 1 --2- 172.21.15.121:0/3391878896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f108c038110 0x7f108c0d4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.217 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.209+0000 7f1093e00640 1 -- 172.21.15.121:0/3391878896 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f108c0ee0b0 con 0x7f108c0edbc0 2024-04-20T10:21:36.217 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.210+0000 7f1093e00640 1 --2- 172.21.15.121:0/3391878896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f108c0edbc0 0x7f108c0dcb00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f108400c750 tx=0x7f108400cc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:36.217 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.210+0000 7f1083400640 1 -- 172.21.15.121:0/3391878896 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1084018070 con 0x7f108c0edbc0 2024-04-20T10:21:36.217 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.210+0000 7f1083400640 1 -- 172.21.15.121:0/3391878896 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10840072d0 con 0x7f108c0edbc0 2024-04-20T10:21:36.217 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.210+0000 7f1083400640 1 -- 172.21.15.121:0/3391878896 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10840047e0 con 0x7f108c0edbc0 2024-04-20T10:21:36.217 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.211+0000 7f1099200640 1 -- 172.21.15.121:0/3391878896 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f108c0ee390 con 0x7f108c0edbc0 2024-04-20T10:21:36.218 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.211+0000 7f1099200640 1 -- 172.21.15.121:0/3391878896 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f108c0db730 con 0x7f108c0edbc0 2024-04-20T10:21:36.218 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.215+0000 7f1099200640 1 -- 172.21.15.121:0/3391878896 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f108c005390 con 0x7f108c0edbc0 2024-04-20T10:21:36.218 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.216+0000 7f1083400640 1 -- 172.21.15.121:0/3391878896 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f108401b070 con 0x7f108c0edbc0 2024-04-20T10:21:36.218 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.217+0000 7f1083400640 1 --2- 172.21.15.121:0/3391878896 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f107806c9c0 0x7f107806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.218 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.218+0000 7f1083400640 1 -- 172.21.15.121:0/3391878896 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f108408b2e0 con 0x7f108c0edbc0 2024-04-20T10:21:36.218 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.218+0000 7f1093400640 1 --2- 172.21.15.121:0/3391878896 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f107806c9c0 0x7f107806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.219 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.210+0000 7f593c800640 1 -- 172.21.15.121:0/1072898731 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5934068980 con 0x7f59340cd170 2024-04-20T10:21:36.219 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.212+0000 7f0ba8c00640 1 -- 172.21.15.121:0/2659132409 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ba001a070 con 0x7f0bac0b17d0 2024-04-20T10:21:36.219 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.213+0000 7f0ba8c00640 1 --2- 172.21.15.121:0/2659132409 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f0b8c06c940 0x7f0b8c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.219 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.214+0000 7f0ba8c00640 1 -- 172.21.15.121:0/2659132409 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0ba008b1c0 con 0x7f0bac0b17d0 2024-04-20T10:21:36.219 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.214+0000 7f0babe00640 1 --2- 172.21.15.121:0/2659132409 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f0b8c06c940 0x7f0b8c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.219 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.216+0000 7f0babe00640 1 --2- 172.21.15.121:0/2659132409 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f0b8c06c940 0x7f0b8c06ee00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0bac0cae90 tx=0x7f0b98009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:36.219 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.219+0000 7f1093400640 1 --2- 172.21.15.121:0/3391878896 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f107806c9c0 0x7f107806ee80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f108c0544b0 tx=0x7f10940ab000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:36.220 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.221+0000 7f592e000640 1 -- 172.21.15.121:0/1072898731 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5930005d10 con 0x7f59340cd170 2024-04-20T10:21:36.221 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.222+0000 7f592e000640 1 --2- 172.21.15.121:0/1072898731 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f591806c8f0 0x7f591806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.222 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.223+0000 7f0ba8c00640 1 -- 172.21.15.121:0/2659132409 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0ba0058cd0 con 0x7f0bac0b17d0 2024-04-20T10:21:36.222 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.223+0000 7f5939e00640 1 --2- 172.21.15.121:0/1072898731 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f591806c8f0 0x7f591806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.223 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.223+0000 7f592e000640 1 -- 172.21.15.121:0/1072898731 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5930057af0 con 0x7f59340cd170 2024-04-20T10:21:36.223 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.224+0000 7f5939e00640 1 --2- 172.21.15.121:0/1072898731 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f591806c8f0 0x7f591806edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5928007fd0 tx=0x7f5928000ed0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:36.230 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.231+0000 7f1083400640 1 -- 172.21.15.121:0/3391878896 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1084058df0 con 0x7f108c0edbc0 2024-04-20T10:21:36.244 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.244+0000 7f592e000640 1 -- 172.21.15.121:0/1072898731 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5930007e20 con 0x7f59340cd170 2024-04-20T10:21:36.248 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.249+0000 7fea0b000640 1 Processor -- start 2024-04-20T10:21:36.251 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.249+0000 7fea0b000640 1 -- start start 2024-04-20T10:21:36.251 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.251+0000 7fea0b000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fea040b3330 0x7fea040d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.251 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.251+0000 7fea0b000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea0406c190 con 0x7fea0406cde0 2024-04-20T10:21:36.251 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.251+0000 7fea0b000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea0406c470 con 0x7fea04161d50 2024-04-20T10:21:36.251 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.251+0000 7fea0b000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fea04162130 con 0x7fea040b3330 2024-04-20T10:21:36.251 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.252+0000 7fea0a600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fea040b3330 0x7fea040d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.252 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.252+0000 7fea09200640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fea0406cde0 0x7fea040b7660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:47580/0 (socket says 172.21.15.121:47580) 2024-04-20T10:21:36.252 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.252+0000 7fea09c00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fea04161d50 0x7fea0406c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.121:37114/0 (socket says 172.21.15.121:37114) 2024-04-20T10:21:36.252 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.252+0000 7fea09200640 1 -- 172.21.15.121:0/1996523288 learned_addr learned my addr 172.21.15.121:0/1996523288 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:36.252 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.252+0000 7fea03e00640 1 -- 172.21.15.121:0/1996523288 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 725825143 0 0) 0x7fea0406c470 con 0x7fea04161d50 2024-04-20T10:21:36.252 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.253+0000 7fea03e00640 1 -- 172.21.15.121:0/1996523288 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe9e8003520 con 0x7fea04161d50 2024-04-20T10:21:36.253 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.253+0000 7fea0a600640 1 -- 172.21.15.121:0/1996523288 >> v1:172.21.15.123:6789/0 conn(0x7fea04161d50 legacy=0x7fea0406c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.253 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.253+0000 7fea0a600640 1 -- 172.21.15.121:0/1996523288 >> v1:172.21.15.121:6789/0 conn(0x7fea0406cde0 legacy=0x7fea040b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.253 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.253+0000 7fea0a600640 1 -- 172.21.15.121:0/1996523288 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea040d2e00 con 0x7fea040b3330 2024-04-20T10:21:36.255 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.255+0000 7fea0a600640 1 --2- 172.21.15.121:0/1996523288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea040b3330 0x7fea040d38e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe9f00097a0 tx=0x7fe9f002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a26ada33a08e62e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:36.255 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.256+0000 7fea03e00640 1 -- 172.21.15.121:0/1996523288 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0034070 con 0x7fea040b3330 2024-04-20T10:21:36.255 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.256+0000 7fea03e00640 1 -- 172.21.15.121:0/1996523288 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9f0030840 con 0x7fea040b3330 2024-04-20T10:21:36.255 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.256+0000 7fea03e00640 1 -- 172.21.15.121:0/1996523288 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0034070 con 0x7fea040b3330 2024-04-20T10:21:36.256 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.257+0000 7f4837200640 1 Processor -- start 2024-04-20T10:21:36.258 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.257+0000 7fea0b000640 1 -- 172.21.15.121:0/1996523288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea040b3330 msgr2=0x7fea040d38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.258 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.257+0000 7fea0b000640 1 --2- 172.21.15.121:0/1996523288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea040b3330 0x7fea040d38e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe9f00097a0 tx=0x7fe9f002b980 comp rx=0 tx=0).stop 2024-04-20T10:21:36.258 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.258+0000 7fea0b000640 1 -- 172.21.15.121:0/1996523288 shutdown_connections 2024-04-20T10:21:36.258 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.258+0000 7fea0b000640 1 --2- 172.21.15.121:0/1996523288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea040b3330 0x7fea040d38e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.258 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.258+0000 7fea0b000640 1 -- 172.21.15.121:0/1996523288 >> 172.21.15.121:0/1996523288 conn(0x7fea040d4450 msgr2=0x7fea040d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:36.259 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.258+0000 7fea0b000640 1 -- 172.21.15.121:0/1996523288 shutdown_connections 2024-04-20T10:21:36.259 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.258+0000 7fea0b000640 1 -- 172.21.15.121:0/1996523288 wait complete. 2024-04-20T10:21:36.260 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.257+0000 7f4837200640 1 -- start start 2024-04-20T10:21:36.260 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.259+0000 7f4837200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f483015a7d0 0x7f48300ad210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.259+0000 7f4837200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f483006bdd0 0x7f483015abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.259+0000 7f4837200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f483006de00 0x7f483006e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.259+0000 7f4837200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f483015b0f0 con 0x7f483006de00 2024-04-20T10:21:36.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.259+0000 7f4837200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f48300ad810 con 0x7f483015a7d0 2024-04-20T10:21:36.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.259+0000 7f4837200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f48300adaf0 con 0x7f483006bdd0 2024-04-20T10:21:36.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.259+0000 7f482fe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f483015a7d0 0x7f48300ad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.260+0000 7f482fe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f483015a7d0 0x7f48300ad210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.121:43108/0 (socket says 172.21.15.121:43108) 2024-04-20T10:21:36.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.260+0000 7f482fe00640 1 -- 172.21.15.121:0/982908439 learned_addr learned my addr 172.21.15.121:0/982908439 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:36.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.260+0000 7f482fe00640 1 -- 172.21.15.121:0/982908439 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f483006bdd0 msgr2=0x7f483015abb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T10:21:36.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.260+0000 7f482fe00640 1 --2- 172.21.15.121:0/982908439 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f483006bdd0 0x7f483015abb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.260+0000 7f482fe00640 1 -- 172.21.15.121:0/982908439 >> v2:172.21.15.121:3300/0 conn(0x7f483006de00 msgr2=0x7f483006e290 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T10:21:36.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.261+0000 7f4835200640 1 --2- 172.21.15.121:0/982908439 >> v2:172.21.15.121:3300/0 conn(0x7f483006de00 0x7f483006e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.261+0000 7f482fe00640 1 --2- 172.21.15.121:0/982908439 >> v2:172.21.15.121:3300/0 conn(0x7f483006de00 0x7f483006e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.262 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.261+0000 7f482fe00640 1 -- 172.21.15.121:0/982908439 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48300addd0 con 0x7f483015a7d0 2024-04-20T10:21:36.262 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.261+0000 7f4835200640 1 --2- 172.21.15.121:0/982908439 >> v2:172.21.15.121:3300/0 conn(0x7f483006de00 0x7f483006e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:36.262 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.262+0000 7fea0b000640 1 Processor -- start 2024-04-20T10:21:36.262 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.262+0000 7fea0b000640 1 -- start start 2024-04-20T10:21:36.263 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.263+0000 7f482fe00640 1 --2- 172.21.15.121:0/982908439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f483015a7d0 0x7f48300ad210 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f48300b56d0 tx=0x7f482802e330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9299cf1715a5124 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:36.263 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.264+0000 7f7ca2600640 1 Processor -- start 2024-04-20T10:21:36.263 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.264+0000 7f7ca2600640 1 -- start start 2024-04-20T10:21:36.264 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.264+0000 7fea0b000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea0406cde0 0x7fea041f51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.264 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.265+0000 7f1770a00640 1 Processor -- start 2024-04-20T10:21:36.265 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.265+0000 7fea0b000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea040b3330 0x7fea04158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.265 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.265+0000 7fea09c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea0406cde0 0x7fea041f51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.265 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.265+0000 7fea09c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea0406cde0 0x7fea041f51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58524/0 (socket says 172.21.15.121:58524) 2024-04-20T10:21:36.265 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.265+0000 7fea0b000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea04161d50 0x7fea040cbed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.265 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.265+0000 7fea0b000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fea040b3880 con 0x7fea0406cde0 2024-04-20T10:21:36.265 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.265+0000 7fea0b000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fea040d33c0 con 0x7fea04161d50 2024-04-20T10:21:36.266 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.265+0000 7fea0b000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fea040b3a00 con 0x7fea040b3330 2024-04-20T10:21:36.266 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.266+0000 7fea09c00640 1 -- 172.21.15.121:0/1692037745 learned_addr learned my addr 172.21.15.121:0/1692037745 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:36.266 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.266+0000 7f1770a00640 1 -- start start 2024-04-20T10:21:36.266 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.266+0000 7fea09200640 1 --2- 172.21.15.121:0/1692037745 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea040b3330 0x7fea04158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.267 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.266+0000 7f482ea00640 1 -- 172.21.15.121:0/982908439 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4828034070 con 0x7f483015a7d0 2024-04-20T10:21:36.267 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.266+0000 7f482ea00640 1 -- 172.21.15.121:0/982908439 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4828038600 con 0x7f483015a7d0 2024-04-20T10:21:36.267 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.266+0000 7f482ea00640 1 -- 172.21.15.121:0/982908439 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482800be60 con 0x7f483015a7d0 2024-04-20T10:21:36.267 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.266+0000 7f7ca2600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7c9c15e550 0x7f7c9c0d06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.267 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.266+0000 7f7ca2600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c9c0cf950 con 0x7f7c9c0d4380 2024-04-20T10:21:36.267 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.266+0000 7f7ca2600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c9c162e90 con 0x7f7c9c1628e0 2024-04-20T10:21:36.267 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.266+0000 7f7ca2600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f7c9c0cfbf0 con 0x7f7c9c15e550 2024-04-20T10:21:36.268 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.266+0000 7fea0a600640 1 --2- 172.21.15.121:0/1692037745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea04161d50 0x7fea040cbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.268 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.267+0000 7fea09200640 1 -- 172.21.15.121:0/1692037745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea04161d50 msgr2=0x7fea040cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.268 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.267+0000 7fea09200640 1 --2- 172.21.15.121:0/1692037745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea04161d50 0x7fea040cbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.268 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.267+0000 7fea09200640 1 -- 172.21.15.121:0/1692037745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea0406cde0 msgr2=0x7fea041f51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.268 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.267+0000 7fea09200640 1 --2- 172.21.15.121:0/1692037745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea0406cde0 0x7fea041f51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.268 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.267+0000 7fea09200640 1 -- 172.21.15.121:0/1692037745 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea040c7f60 con 0x7fea040b3330 2024-04-20T10:21:36.268 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.267+0000 7fea0a600640 1 --2- 172.21.15.121:0/1692037745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea04161d50 0x7fea040cbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:21:36.268 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.267+0000 7f7ca1200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7c9c15e550 0x7f7c9c0d06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.269 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.268+0000 7f4837200640 1 -- 172.21.15.121:0/982908439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f483015a7d0 msgr2=0x7f48300ad210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.269 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.268+0000 7f4837200640 1 --2- 172.21.15.121:0/982908439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f483015a7d0 0x7f48300ad210 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f48300b56d0 tx=0x7f482802e330 comp rx=0 tx=0).stop 2024-04-20T10:21:36.269 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.268+0000 7f7c9be00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f7c9c0d4380 0x7f7c9c1621d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:47584/0 (socket says 172.21.15.121:47584) 2024-04-20T10:21:36.269 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.268+0000 7f7c9be00640 1 -- 172.21.15.121:0/4178747439 learned_addr learned my addr 172.21.15.121:0/4178747439 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:36.269 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.268+0000 7f1770a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f176c0d0380 0x7f176c0d07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.269 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.268+0000 7f1770a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f176c0d1e90 con 0x7f176c0d1a00 2024-04-20T10:21:36.270 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.268+0000 7f1770a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f176c0d0d10 con 0x7f176c0d4270 2024-04-20T10:21:36.270 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.268+0000 7f1770a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f176c0d40c0 con 0x7f176c0d0380 2024-04-20T10:21:36.270 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.267+0000 7fea09c00640 1 --2- 172.21.15.121:0/1692037745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea0406cde0 0x7fea041f51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:21:36.270 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.268+0000 7f176a800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f176c0d0380 0x7f176c0d07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.270 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.268+0000 7f176a800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f176c0d0380 0x7f176c0d07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58542/0 (socket says 172.21.15.121:58542) 2024-04-20T10:21:36.270 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.268+0000 7f176a800640 1 -- 172.21.15.121:0/2504152193 learned_addr learned my addr 172.21.15.121:0/2504152193 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:36.271 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.268+0000 7f4837200640 1 -- 172.21.15.121:0/982908439 shutdown_connections 2024-04-20T10:21:36.271 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.268+0000 7f4837200640 1 --2- 172.21.15.121:0/982908439 >> v2:172.21.15.121:3300/0 conn(0x7f483006de00 0x7f483006e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.271 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.268+0000 7f4837200640 1 --2- 172.21.15.121:0/982908439 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f483006bdd0 0x7f483015abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.271 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.268+0000 7f4837200640 1 --2- 172.21.15.121:0/982908439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f483015a7d0 0x7f48300ad210 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.271 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.268+0000 7f4837200640 1 -- 172.21.15.121:0/982908439 >> 172.21.15.121:0/982908439 conn(0x7f4830072090 msgr2=0x7f48300b4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:36.271 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.268+0000 7f7ca1200640 1 -- 172.21.15.121:0/4178747439 >> v1:172.21.15.123:6789/0 conn(0x7f7c9c1628e0 legacy=0x7f7c9c0d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:36.272 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.268+0000 7fea09200640 1 --2- 172.21.15.121:0/1692037745 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea040b3330 0x7fea04158a30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc00cae0 tx=0x7fe9fc00a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:36.272 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.269+0000 7fea01600640 1 -- 172.21.15.121:0/1692037745 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc017070 con 0x7fea040b3330 2024-04-20T10:21:36.272 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.269+0000 7fea0b000640 1 -- 172.21.15.121:0/1692037745 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea040c8240 con 0x7fea040b3330 2024-04-20T10:21:36.272 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.269+0000 7fea0b000640 1 -- 172.21.15.121:0/1692037745 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fea040c8800 con 0x7fea040b3330 2024-04-20T10:21:36.272 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.269+0000 7f1768a00640 1 -- 172.21.15.121:0/2504152193 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3758917461 0 0) 0x7f1758001110 con 0x7f176c0d1a00 2024-04-20T10:21:36.272 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.270+0000 7f1768a00640 1 -- 172.21.15.121:0/2504152193 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1750003520 con 0x7f176c0d1a00 2024-04-20T10:21:36.272 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.270+0000 7f4837200640 1 -- 172.21.15.121:0/982908439 shutdown_connections 2024-04-20T10:21:36.273 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.269+0000 7fea01600640 1 -- 172.21.15.121:0/1692037745 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9fc00ac40 con 0x7fea040b3330 2024-04-20T10:21:36.273 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.269+0000 7fea01600640 1 -- 172.21.15.121:0/1692037745 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc012600 con 0x7fea040b3330 2024-04-20T10:21:36.273 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.268+0000 7f7ca1200640 1 -- 172.21.15.121:0/4178747439 >> v1:172.21.15.121:6789/0 conn(0x7f7c9c0d4380 legacy=0x7f7c9c1621d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:36.273 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.268+0000 7f7ca1200640 1 -- 172.21.15.121:0/4178747439 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c9c0cfed0 con 0x7f7c9c15e550 2024-04-20T10:21:36.273 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.270+0000 7f7ca1200640 1 --2- 172.21.15.121:0/4178747439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c9c15e550 0x7f7c9c0d06b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c08e370 tx=0x7f7c9002b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86ecbbeca49aadce server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:36.274 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.270+0000 7f176a800640 1 -- 172.21.15.121:0/2504152193 >> v1:172.21.15.123:6789/0 conn(0x7f176c0d4270 legacy=0x7f176c0d4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.274 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.270+0000 7f176a800640 1 -- 172.21.15.121:0/2504152193 >> v1:172.21.15.121:6789/0 conn(0x7f176c0d1a00 legacy=0x7f176c0b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.274 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.270+0000 7f176a800640 1 -- 172.21.15.121:0/2504152193 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f176c0d3820 con 0x7f176c0d0380 2024-04-20T10:21:36.274 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.272+0000 7f7c9b400640 1 -- 172.21.15.121:0/4178747439 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c90037070 con 0x7f7c9c15e550 2024-04-20T10:21:36.274 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.272+0000 7f7c9b400640 1 -- 172.21.15.121:0/4178747439 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c90030710 con 0x7f7c9c15e550 2024-04-20T10:21:36.274 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.272+0000 7f7c9b400640 1 -- 172.21.15.121:0/4178747439 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c90030ae0 con 0x7f7c9c15e550 2024-04-20T10:21:36.275 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.271+0000 7f4837200640 1 -- 172.21.15.121:0/982908439 wait complete. 2024-04-20T10:21:36.275 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.272+0000 7fea0b000640 1 -- 172.21.15.121:0/1692037745 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9dc005290 con 0x7fea040b3330 2024-04-20T10:21:36.275 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.273+0000 7f7ca2600640 1 -- 172.21.15.121:0/4178747439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c9c15e550 msgr2=0x7f7c9c0d06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.275 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.273+0000 7f7ca2600640 1 --2- 172.21.15.121:0/4178747439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c9c15e550 0x7f7c9c0d06b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c08e370 tx=0x7f7c9002b780 comp rx=0 tx=0).stop 2024-04-20T10:21:36.276 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.273+0000 7f176a800640 1 --2- 172.21.15.121:0/2504152193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f176c0d0380 0x7f176c0d07d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1764009e50 tx=0x7f176402be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98fdfd2031f5355f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:36.276 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.273+0000 7f4837200640 1 Processor -- start 2024-04-20T10:21:36.276 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.273+0000 7f7ca2600640 1 -- 172.21.15.121:0/4178747439 shutdown_connections 2024-04-20T10:21:36.276 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.273+0000 7f7ca2600640 1 --2- 172.21.15.121:0/4178747439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c9c15e550 0x7f7c9c0d06b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.276 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.273+0000 7f7ca2600640 1 -- 172.21.15.121:0/4178747439 >> 172.21.15.121:0/4178747439 conn(0x7f7c9c08bf80 msgr2=0x7f7c9c09d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:36.277 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.273+0000 7f7ca2600640 1 -- 172.21.15.121:0/4178747439 shutdown_connections 2024-04-20T10:21:36.277 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.274+0000 7f1768a00640 1 -- 172.21.15.121:0/2504152193 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176402c990 con 0x7f176c0d0380 2024-04-20T10:21:36.277 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.274+0000 7f1768a00640 1 -- 172.21.15.121:0/2504152193 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f176402cb30 con 0x7f176c0d0380 2024-04-20T10:21:36.277 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.274+0000 7f1768a00640 1 -- 172.21.15.121:0/2504152193 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17640343f0 con 0x7f176c0d0380 2024-04-20T10:21:36.277 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.274+0000 7f7ca2600640 1 -- 172.21.15.121:0/4178747439 wait complete. 2024-04-20T10:21:36.278 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.275+0000 7f4837200640 1 -- start start 2024-04-20T10:21:36.278 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.274+0000 7fea01600640 1 -- 172.21.15.121:0/1692037745 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe9fc01a070 con 0x7fea040b3330 2024-04-20T10:21:36.278 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.275+0000 7f1770a00640 1 -- 172.21.15.121:0/2504152193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f176c0d0380 msgr2=0x7f176c0d07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.278 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.275+0000 7f1770a00640 1 --2- 172.21.15.121:0/2504152193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f176c0d0380 0x7f176c0d07d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1764009e50 tx=0x7f176402be70 comp rx=0 tx=0).stop 2024-04-20T10:21:36.278 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.276+0000 7f1770a00640 1 -- 172.21.15.121:0/2504152193 shutdown_connections 2024-04-20T10:21:36.279 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.276+0000 7f4837200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f483006bdd0 0x7f483009ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.279 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.276+0000 7f1770a00640 1 --2- 172.21.15.121:0/2504152193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f176c0d0380 0x7f176c0d07d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.279 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.276+0000 7f1770a00640 1 -- 172.21.15.121:0/2504152193 >> 172.21.15.121:0/2504152193 conn(0x7f176c0b99c0 msgr2=0x7f176c0b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:36.279 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.276+0000 7f4837200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f483006de00 0x7f48301f0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.280 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.276+0000 7f4837200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f483015a7d0 0x7f48300aeca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.280 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.276+0000 7f4837200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f483006b970 con 0x7f483006de00 2024-04-20T10:21:36.280 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.276+0000 7f4837200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f48300ae390 con 0x7f483006bdd0 2024-04-20T10:21:36.280 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.276+0000 7f4837200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f483006baf0 con 0x7f483015a7d0 2024-04-20T10:21:36.280 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.276+0000 7f1770a00640 1 -- 172.21.15.121:0/2504152193 shutdown_connections 2024-04-20T10:21:36.280 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.276+0000 7f1770a00640 1 -- 172.21.15.121:0/2504152193 wait complete. 2024-04-20T10:21:36.281 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.276+0000 7fea01600640 1 --2- 172.21.15.121:0/1692037745 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe9e406c8f0 0x7fe9e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.281 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.276+0000 7fea01600640 1 -- 172.21.15.121:0/1692037745 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe9fc08bae0 con 0x7fea040b3330 2024-04-20T10:21:36.281 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.277+0000 7f482f400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f483006de00 0x7f48301f0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.281 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.277+0000 7f7ca2600640 1 Processor -- start 2024-04-20T10:21:36.281 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.277+0000 7f482f400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f483006de00 0x7f48301f0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58546/0 (socket says 172.21.15.121:58546) 2024-04-20T10:21:36.282 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.277+0000 7f482f400640 1 -- 172.21.15.121:0/869156069 learned_addr learned my addr 172.21.15.121:0/869156069 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:36.282 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.278+0000 7fea09c00640 1 --2- 172.21.15.121:0/1692037745 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe9e406c8f0 0x7fe9e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.282 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.277+0000 7f482fe00640 1 --2- 172.21.15.121:0/869156069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f483006bdd0 0x7f483009ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.282 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.277+0000 7f4835200640 1 --2- 172.21.15.121:0/869156069 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f483015a7d0 0x7f48300aeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.282 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.278+0000 7f482fe00640 1 -- 172.21.15.121:0/869156069 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f483015a7d0 msgr2=0x7f48300aeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.282 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.278+0000 7f482fe00640 1 --2- 172.21.15.121:0/869156069 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f483015a7d0 0x7f48300aeca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.282 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.278+0000 7f482fe00640 1 -- 172.21.15.121:0/869156069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f483006de00 msgr2=0x7f48301f0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.282 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.278+0000 7f482fe00640 1 --2- 172.21.15.121:0/869156069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f483006de00 0x7f48301f0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.283 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.278+0000 7f482fe00640 1 -- 172.21.15.121:0/869156069 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48300cfbd0 con 0x7f483006bdd0 2024-04-20T10:21:36.283 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.277+0000 7f7ca2600640 1 -- start start 2024-04-20T10:21:36.283 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.279+0000 7f482fe00640 1 --2- 172.21.15.121:0/869156069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f483006bdd0 0x7f483009ca80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4828036000 tx=0x7f4828004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:36.284 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.279+0000 7f7ca2600640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7c9c0d4380 0x7f7c9c15d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.284 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.279+0000 7f7ca2600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c9c1628e0 0x7f7c9c0c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.284 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.279+0000 7f7ca2600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c9c0c9a80 0x7f7c9c0cd020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.284 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.279+0000 7f7ca2600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c15eab0 con 0x7f7c9c1628e0 2024-04-20T10:21:36.284 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.279+0000 7f7ca2600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9c15e930 con 0x7f7c9c0c9a80 2024-04-20T10:21:36.284 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.279+0000 7f7ca2600640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f7c9c0d26d0 con 0x7f7c9c0d4380 2024-04-20T10:21:36.284 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.279+0000 7f1770a00640 1 Processor -- start 2024-04-20T10:21:36.285 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.280+0000 7f7c9be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c9c1628e0 0x7f7c9c0c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.285 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.280+0000 7f4827e00640 1 -- 172.21.15.121:0/869156069 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4828008240 con 0x7f483006bdd0 2024-04-20T10:21:36.285 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.280+0000 7f4827e00640 1 -- 172.21.15.121:0/869156069 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4828045070 con 0x7f483006bdd0 2024-04-20T10:21:36.285 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.280+0000 7f4827e00640 1 -- 172.21.15.121:0/869156069 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4828008240 con 0x7f483006bdd0 2024-04-20T10:21:36.285 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.280+0000 7f4837200640 1 -- 172.21.15.121:0/869156069 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f483009a290 con 0x7f483006bdd0 2024-04-20T10:21:36.286 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.280+0000 7f7c9be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c9c1628e0 0x7f7c9c0c7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58562/0 (socket says 172.21.15.121:58562) 2024-04-20T10:21:36.286 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.280+0000 7f7c9be00640 1 -- 172.21.15.121:0/1515278238 learned_addr learned my addr 172.21.15.121:0/1515278238 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:36.286 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.280+0000 7f7ca1200640 1 --2- 172.21.15.121:0/1515278238 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7c9c0d4380 0x7f7c9c15d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.286 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.280+0000 7f1770a00640 1 -- start start 2024-04-20T10:21:36.286 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.280+0000 7f7ca1c00640 1 --2- 172.21.15.121:0/1515278238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c9c0c9a80 0x7f7c9c0cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.287 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.281+0000 7fea09c00640 1 --2- 172.21.15.121:0/1692037745 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe9e406c8f0 0x7fe9e406edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe9f40011e0 tx=0x7fe9f4007df0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:36.287 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.281+0000 7f7ca1200640 1 -- 172.21.15.121:0/1515278238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c9c0c9a80 msgr2=0x7f7c9c0cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.287 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.281+0000 7f7ca1200640 1 --2- 172.21.15.121:0/1515278238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c9c0c9a80 0x7f7c9c0cd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.287 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.281+0000 7f7ca1200640 1 -- 172.21.15.121:0/1515278238 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c9c1628e0 msgr2=0x7f7c9c0c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.287 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.281+0000 7f7ca1200640 1 --2- 172.21.15.121:0/1515278238 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c9c1628e0 0x7f7c9c0c7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.287 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.281+0000 7f7ca1200640 1 -- 172.21.15.121:0/1515278238 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c9c0c89e0 con 0x7f7c9c0d4380 2024-04-20T10:21:36.287 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.281+0000 7f4837200640 1 -- 172.21.15.121:0/869156069 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f483009a530 con 0x7f483006bdd0 2024-04-20T10:21:36.288 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.282+0000 7f7ca1200640 1 --2- 172.21.15.121:0/1515278238 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7c9c0d4380 0x7f7c9c15d7d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7c90032000 tx=0x7f7c900046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:36.288 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.283+0000 7f4837200640 1 -- 172.21.15.121:0/869156069 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4830071de0 con 0x7f483006bdd0 2024-04-20T10:21:36.289 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.283+0000 7f1770a00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f176c0d0380 0x7f176c1f63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.289 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.283+0000 7f1770a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f176c0d1a00 0x7f176c0c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.289 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.283+0000 7f1770a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f176c0d4270 0x7f176c0c9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.289 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.283+0000 7f1770a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f176c0cfe60 con 0x7f176c0d4270 2024-04-20T10:21:36.289 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.283+0000 7f1770a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f176c0d3de0 con 0x7f176c0d1a00 2024-04-20T10:21:36.289 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.283+0000 7f1770a00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f176c0b19a0 con 0x7f176c0d0380 2024-04-20T10:21:36.290 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.284+0000 7f176a800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f176c0d4270 0x7f176c0c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.290 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.284+0000 7f176a800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f176c0d4270 0x7f176c0c9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58570/0 (socket says 172.21.15.121:58570) 2024-04-20T10:21:36.290 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.284+0000 7f176a800640 1 -- 172.21.15.121:0/2851701038 learned_addr learned my addr 172.21.15.121:0/2851701038 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:36.290 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.284+0000 7f1769400640 1 --2- 172.21.15.121:0/2851701038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f176c0d1a00 0x7f176c0c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.290 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.285+0000 7f176a800640 1 -- 172.21.15.121:0/2851701038 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f176c0d0380 msgr2=0x7f176c1f63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.290 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.285+0000 7f176a800640 1 --2- 172.21.15.121:0/2851701038 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f176c0d0380 0x7f176c1f63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.290 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.285+0000 7f176a800640 1 -- 172.21.15.121:0/2851701038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f176c0d1a00 msgr2=0x7f176c0c7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.290 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.285+0000 7f176a800640 1 --2- 172.21.15.121:0/2851701038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f176c0d1a00 0x7f176c0c7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.290 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.285+0000 7f176a800640 1 -- 172.21.15.121:0/2851701038 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f176c0c8790 con 0x7f176c0d4270 2024-04-20T10:21:36.290 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.285+0000 7f1769400640 1 --2- 172.21.15.121:0/2851701038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f176c0d1a00 0x7f176c0c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:21:36.291 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.285+0000 7f4827e00640 1 -- 172.21.15.121:0/869156069 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f482803f070 con 0x7f483006bdd0 2024-04-20T10:21:36.291 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.286+0000 7f4827e00640 1 --2- 172.21.15.121:0/869156069 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f481006caf0 0x7f481006efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.291 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.287+0000 7f176a800640 1 --2- 172.21.15.121:0/2851701038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f176c0d4270 0x7f176c0c9180 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1764009e50 tx=0x7f1764004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:36.291 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.287+0000 7f4827e00640 1 -- 172.21.15.121:0/869156069 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f48280adcd0 con 0x7f483006bdd0 2024-04-20T10:21:36.291 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.287+0000 7f482f400640 1 --2- 172.21.15.121:0/869156069 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f481006caf0 0x7f481006efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.292 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.287+0000 7f1762000640 1 -- 172.21.15.121:0/2851701038 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1764043020 con 0x7f176c0d4270 2024-04-20T10:21:36.292 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.287+0000 7f1770a00640 1 -- 172.21.15.121:0/2851701038 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f176c0c8a70 con 0x7f176c0d4270 2024-04-20T10:21:36.292 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.287+0000 7f1762000640 1 -- 172.21.15.121:0/2851701038 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f176403e450 con 0x7f176c0d4270 2024-04-20T10:21:36.292 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.287+0000 7f1762000640 1 -- 172.21.15.121:0/2851701038 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1764009510 con 0x7f176c0d4270 2024-04-20T10:21:36.292 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.287+0000 7f1770a00640 1 -- 172.21.15.121:0/2851701038 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f176c1f6d30 con 0x7f176c0d4270 2024-04-20T10:21:36.292 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.288+0000 7f482f400640 1 --2- 172.21.15.121:0/869156069 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f481006caf0 0x7f481006efb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f483009bad0 tx=0x7f481c009190 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:36.293 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.289+0000 7f1757400640 1 -- 172.21.15.121:0/2851701038 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1740005290 con 0x7f176c0d4270 2024-04-20T10:21:36.293 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.291+0000 7fea01600640 1 -- 172.21.15.121:0/1692037745 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe9fc059570 con 0x7fea040b3330 2024-04-20T10:21:36.293 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.290+0000 7f7c98c00640 1 -- 172.21.15.121:0/1515278238 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c90042070 con 0x7f7c9c0d4380 2024-04-20T10:21:36.293 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.290+0000 7f7c98c00640 1 -- 172.21.15.121:0/1515278238 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c90032bc0 con 0x7f7c9c0d4380 2024-04-20T10:21:36.293 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.290+0000 7f7c98c00640 1 -- 172.21.15.121:0/1515278238 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c90037040 con 0x7f7c9c0d4380 2024-04-20T10:21:36.294 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.291+0000 7f7ca2600640 1 -- 172.21.15.121:0/1515278238 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c9c0c8cc0 con 0x7f7c9c0d4380 2024-04-20T10:21:36.294 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.291+0000 7f7ca2600640 1 -- 172.21.15.121:0/1515278238 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c9c0c9280 con 0x7f7c9c0d4380 2024-04-20T10:21:36.294 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.291+0000 7f1762000640 1 -- 172.21.15.121:0/2851701038 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1764046020 con 0x7f176c0d4270 2024-04-20T10:21:36.294 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.293+0000 7f1762000640 1 --2- 172.21.15.121:0/2851701038 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f174c06c8f0 0x7f174c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.294 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.293+0000 7f1762000640 1 -- 172.21.15.121:0/2851701038 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f17640ae720 con 0x7f176c0d4270 2024-04-20T10:21:36.294 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.294+0000 7f1769e00640 1 --2- 172.21.15.121:0/2851701038 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f174c06c8f0 0x7f174c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.295 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.295+0000 7f7ca2600640 1 -- 172.21.15.121:0/1515278238 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c9c075b30 con 0x7f7c9c0d4380 2024-04-20T10:21:36.295 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.296+0000 7f1769e00640 1 --2- 172.21.15.121:0/2851701038 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f174c06c8f0 0x7f174c06edb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f175c001790 tx=0x7f175c0079b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:36.295 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.295+0000 7f7c98c00640 1 -- 172.21.15.121:0/1515278238 <== mon.2 v2:172.21.15.123:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7c90045070 con 0x7f7c9c0d4380 2024-04-20T10:21:36.296 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.296+0000 7f7c98c00640 1 --2- 172.21.15.121:0/1515278238 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7c7c06c8f0 0x7f7c7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.296 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.296+0000 7f7c98c00640 1 -- 172.21.15.121:0/1515278238 <== mon.2 v2:172.21.15.123:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7c900ada40 con 0x7f7c9c0d4380 2024-04-20T10:21:36.296 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.297+0000 7f7c9be00640 1 --2- 172.21.15.121:0/1515278238 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7c7c06c8f0 0x7f7c7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.297 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.298+0000 7f7c9be00640 1 --2- 172.21.15.121:0/1515278238 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7c7c06c8f0 0x7f7c7c06edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7c88007d50 tx=0x7f7c88007ca0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:36.299 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.300+0000 7f1762000640 1 -- 172.21.15.121:0/2851701038 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f176404d030 con 0x7f176c0d4270 2024-04-20T10:21:36.301 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.301+0000 7f4827e00640 1 -- 172.21.15.121:0/869156069 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4830071de0 con 0x7f483006bdd0 2024-04-20T10:21:36.309 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.310+0000 7f7c98c00640 1 -- 172.21.15.121:0/1515278238 <== mon.2 v2:172.21.15.123:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7c900afe20 con 0x7f7c9c0d4380 2024-04-20T10:21:36.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.604+0000 7fbd6b200640 1 -- 172.21.15.121:0/336578120 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fbd6c0b30d0 con 0x7fbd6c09b3d0 2024-04-20T10:21:36.604 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.605+0000 7fbd62000640 1 -- 172.21.15.121:0/336578120 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbd6c0b30d0 con 0x7fbd6c09b3d0 2024-04-20T10:21:36.605 INFO:teuthology.orchestra.run.smithi121.stdout:60129542154 2024-04-20T10:21:36.608 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.609+0000 7fbd6b200640 1 -- 172.21.15.121:0/336578120 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fbd4c06c9f0 msgr2=0x7fbd4c06eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.608 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.609+0000 7fbd6b200640 1 --2- 172.21.15.121:0/336578120 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fbd4c06c9f0 0x7fbd4c06eeb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c035000 tx=0x7fbd5c008030 comp rx=0 tx=0).stop 2024-04-20T10:21:36.609 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.609+0000 7fbd6b200640 1 -- 172.21.15.121:0/336578120 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd6c09b3d0 msgr2=0x7fbd6c09c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.609 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.609+0000 7fbd6b200640 1 --2- 172.21.15.121:0/336578120 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd6c09b3d0 0x7fbd6c09c9d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbd5800c7e0 tx=0x7fbd5800cca0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.610 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.610+0000 7fbd6b200640 1 -- 172.21.15.121:0/336578120 shutdown_connections 2024-04-20T10:21:36.610 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.610+0000 7fbd6b200640 1 --2- 172.21.15.121:0/336578120 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd6c09b3d0 0x7fbd6c09c9d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.610 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.610+0000 7fbd6b200640 1 --2- 172.21.15.121:0/336578120 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd6c15ea00 0x7fbd6c09c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.610 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.610+0000 7fbd6b200640 1 --2- 172.21.15.121:0/336578120 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fbd4c06c9f0 0x7fbd4c06eeb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.610 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.610+0000 7fbd6b200640 1 --2- 172.21.15.121:0/336578120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd6c15b490 0x7fbd6c200680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.610 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.610+0000 7fbd6b200640 1 -- 172.21.15.121:0/336578120 >> 172.21.15.121:0/336578120 conn(0x7fbd6c0ccfe0 msgr2=0x7fbd6c06cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:36.611 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.611+0000 7fbd6b200640 1 -- 172.21.15.121:0/336578120 shutdown_connections 2024-04-20T10:21:36.611 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.611+0000 7fbd6b200640 1 -- 172.21.15.121:0/336578120 wait complete. 2024-04-20T10:21:36.636 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542154 for osd.3 2024-04-20T10:21:36.637 DEBUG:teuthology.parallel:result is None 2024-04-20T10:21:36.643 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.643+0000 7f0bb3a00640 1 -- 172.21.15.121:0/2659132409 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f0bac0b2410 con 0x7f0bac0b17d0 2024-04-20T10:21:36.643 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.644+0000 7f0ba8c00640 1 -- 172.21.15.121:0/2659132409 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0ba005cfd0 con 0x7f0bac0b17d0 2024-04-20T10:21:36.644 INFO:teuthology.orchestra.run.smithi121.stdout:60129542156 2024-04-20T10:21:36.647 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.647+0000 7f0bb3a00640 1 -- 172.21.15.121:0/2659132409 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f0b8c06c940 msgr2=0x7f0b8c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.647 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.648+0000 7f0bb3a00640 1 --2- 172.21.15.121:0/2659132409 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f0b8c06c940 0x7f0b8c06ee00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0bac0cae90 tx=0x7f0b98009190 comp rx=0 tx=0).stop 2024-04-20T10:21:36.647 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.648+0000 7f0bb3a00640 1 -- 172.21.15.121:0/2659132409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bac0b17d0 msgr2=0x7f0bac0c8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.648 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.648+0000 7f0bb3a00640 1 --2- 172.21.15.121:0/2659132409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bac0b17d0 0x7f0bac0c8d00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f0ba000c390 tx=0x7f0ba000c850 comp rx=0 tx=0).stop 2024-04-20T10:21:36.648 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.649+0000 7f0bb3a00640 1 -- 172.21.15.121:0/2659132409 shutdown_connections 2024-04-20T10:21:36.648 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.649+0000 7f0bb3a00640 1 --2- 172.21.15.121:0/2659132409 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f0b8c06c940 0x7f0b8c06ee00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.648 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.649+0000 7f0bb3a00640 1 --2- 172.21.15.121:0/2659132409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0bac0d4270 0x7f0bac0b64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.648 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.649+0000 7f0bb3a00640 1 --2- 172.21.15.121:0/2659132409 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0bac0d1a00 0x7f0bac1551a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.648 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.649+0000 7f0bb3a00640 1 --2- 172.21.15.121:0/2659132409 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bac0b17d0 0x7f0bac0c8d00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.648 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.649+0000 7f0bb3a00640 1 -- 172.21.15.121:0/2659132409 >> 172.21.15.121:0/2659132409 conn(0x7f0bac0b99c0 msgr2=0x7f0bac0b34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:36.649 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.649+0000 7f0bb3a00640 1 -- 172.21.15.121:0/2659132409 shutdown_connections 2024-04-20T10:21:36.649 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.649+0000 7f0bb3a00640 1 -- 172.21.15.121:0/2659132409 wait complete. 2024-04-20T10:21:36.672 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.672+0000 7f593c800640 1 -- 172.21.15.121:0/1072898731 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f593415e900 con 0x7f59340cd170 2024-04-20T10:21:36.673 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.673+0000 7f592e000640 1 -- 172.21.15.121:0/1072898731 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f593005a150 con 0x7f59340cd170 2024-04-20T10:21:36.674 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542156 got 60129542156 for osd.1 2024-04-20T10:21:36.674 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.674+0000 7f1099200640 1 -- 172.21.15.121:0/3391878896 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f108c037be0 con 0x7f108c0edbc0 2024-04-20T10:21:36.674 DEBUG:teuthology.parallel:result is None 2024-04-20T10:21:36.675 INFO:teuthology.orchestra.run.smithi121.stdout:60129542154 2024-04-20T10:21:36.675 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.675+0000 7f1083400640 1 -- 172.21.15.121:0/3391878896 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f108405d0f0 con 0x7f108c0edbc0 2024-04-20T10:21:36.675 INFO:teuthology.orchestra.run.smithi121.stdout:60129542155 2024-04-20T10:21:36.676 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.677+0000 7f593c800640 1 -- 172.21.15.121:0/1072898731 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f591806c8f0 msgr2=0x7f591806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.677 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.677+0000 7f593c800640 1 --2- 172.21.15.121:0/1072898731 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f591806c8f0 0x7f591806edb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5928007fd0 tx=0x7f5928000ed0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.677 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.677+0000 7f593c800640 1 -- 172.21.15.121:0/1072898731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59340cd170 msgr2=0x7f59340b5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.677 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.677+0000 7f593c800640 1 --2- 172.21.15.121:0/1072898731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59340cd170 0x7f59340b5b00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f593408d680 tx=0x7f5930007700 comp rx=0 tx=0).stop 2024-04-20T10:21:36.677 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.677+0000 7f593c800640 1 -- 172.21.15.121:0/1072898731 shutdown_connections 2024-04-20T10:21:36.677 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.677+0000 7f593c800640 1 --2- 172.21.15.121:0/1072898731 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f591806c8f0 0x7f591806edb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.677 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.677+0000 7f593c800640 1 --2- 172.21.15.121:0/1072898731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59340cd170 0x7f59340b5b00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.677 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.677+0000 7f593c800640 1 --2- 172.21.15.121:0/1072898731 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5934159810 0x7f59340cc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.677 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.677+0000 7f593c800640 1 --2- 172.21.15.121:0/1072898731 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f591c004800 0x7f5934159110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.677 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.677+0000 7f593c800640 1 -- 172.21.15.121:0/1072898731 >> 172.21.15.121:0/1072898731 conn(0x7f59340b99c0 msgr2=0x7f59340676c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:36.677 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.677+0000 7f593c800640 1 -- 172.21.15.121:0/1072898731 shutdown_connections 2024-04-20T10:21:36.678 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.678+0000 7f593c800640 1 -- 172.21.15.121:0/1072898731 wait complete. 2024-04-20T10:21:36.679 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.679+0000 7f1099200640 1 -- 172.21.15.121:0/3391878896 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f107806c9c0 msgr2=0x7f107806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.679 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.679+0000 7f1099200640 1 --2- 172.21.15.121:0/3391878896 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f107806c9c0 0x7f107806ee80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f108c0544b0 tx=0x7f10940ab000 comp rx=0 tx=0).stop 2024-04-20T10:21:36.679 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.679+0000 7f1099200640 1 -- 172.21.15.121:0/3391878896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f108c0edbc0 msgr2=0x7f108c0dcb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.680 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.679+0000 7f1099200640 1 --2- 172.21.15.121:0/3391878896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f108c0edbc0 0x7f108c0dcb00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f108400c750 tx=0x7f108400cc10 comp rx=0 tx=0).stop 2024-04-20T10:21:36.680 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.679+0000 7f1099200640 1 -- 172.21.15.121:0/3391878896 shutdown_connections 2024-04-20T10:21:36.680 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.679+0000 7f1099200640 1 --2- 172.21.15.121:0/3391878896 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f107806c9c0 0x7f107806ee80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.680 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.679+0000 7f1099200640 1 --2- 172.21.15.121:0/3391878896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f108c0edbc0 0x7f108c0dcb00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.680 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.679+0000 7f1099200640 1 --2- 172.21.15.121:0/3391878896 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f108c04ffd0 0x7f108c0ed680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.680 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.679+0000 7f1099200640 1 --2- 172.21.15.121:0/3391878896 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f108c038110 0x7f108c0d4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.680 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.679+0000 7f1099200640 1 -- 172.21.15.121:0/3391878896 >> 172.21.15.121:0/3391878896 conn(0x7f108c009fb0 msgr2=0x7f108c005dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:36.680 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.679+0000 7f1099200640 1 -- 172.21.15.121:0/3391878896 shutdown_connections 2024-04-20T10:21:36.680 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.679+0000 7f1099200640 1 -- 172.21.15.121:0/3391878896 wait complete. 2024-04-20T10:21:36.685 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.686+0000 7f1757400640 1 -- 172.21.15.121:0/2851701038 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f1740005580 con 0x7f176c0d4270 2024-04-20T10:21:36.686 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.686+0000 7f1762000640 1 -- 172.21.15.121:0/2851701038 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f176407c780 con 0x7f176c0d4270 2024-04-20T10:21:36.686 INFO:teuthology.orchestra.run.smithi121.stdout:60129542154 2024-04-20T10:21:36.688 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.689+0000 7f1757400640 1 -- 172.21.15.121:0/2851701038 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f174c06c8f0 msgr2=0x7f174c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.688 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.689+0000 7f1757400640 1 --2- 172.21.15.121:0/2851701038 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f174c06c8f0 0x7f174c06edb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f175c001790 tx=0x7f175c0079b0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.688 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.689+0000 7f1757400640 1 -- 172.21.15.121:0/2851701038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f176c0d4270 msgr2=0x7f176c0c9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.688 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.689+0000 7f1757400640 1 --2- 172.21.15.121:0/2851701038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f176c0d4270 0x7f176c0c9180 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1764009e50 tx=0x7f1764004a40 comp rx=0 tx=0).stop 2024-04-20T10:21:36.689 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.689+0000 7f1757400640 1 -- 172.21.15.121:0/2851701038 shutdown_connections 2024-04-20T10:21:36.689 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.689+0000 7f1757400640 1 --2- 172.21.15.121:0/2851701038 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f176c0d4270 0x7f176c0c9180 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.689 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.689+0000 7f1757400640 1 --2- 172.21.15.121:0/2851701038 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f174c06c8f0 0x7f174c06edb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.689 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.689+0000 7f1757400640 1 --2- 172.21.15.121:0/2851701038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f176c0d1a00 0x7f176c0c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.689 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.689+0000 7f1757400640 1 --2- 172.21.15.121:0/2851701038 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f176c0d0380 0x7f176c1f63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.689 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.689+0000 7f1757400640 1 -- 172.21.15.121:0/2851701038 >> 172.21.15.121:0/2851701038 conn(0x7f176c0b99c0 msgr2=0x7f176c0b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:36.689 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.689+0000 7f1757400640 1 -- 172.21.15.121:0/2851701038 shutdown_connections 2024-04-20T10:21:36.690 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.689+0000 7f1757400640 1 -- 172.21.15.121:0/2851701038 wait complete. 2024-04-20T10:21:36.696 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.696+0000 7fea0b000640 1 -- 172.21.15.121:0/1692037745 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fe9dc005580 con 0x7fea040b3330 2024-04-20T10:21:36.697 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.697+0000 7fea01600640 1 -- 172.21.15.121:0/1692037745 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe9fc05d870 con 0x7fea040b3330 2024-04-20T10:21:36.697 INFO:teuthology.orchestra.run.smithi121.stdout:60129542154 2024-04-20T10:21:36.700 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.701+0000 7fea0b000640 1 -- 172.21.15.121:0/1692037745 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe9e406c8f0 msgr2=0x7fe9e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.700 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.701+0000 7fea0b000640 1 --2- 172.21.15.121:0/1692037745 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe9e406c8f0 0x7fe9e406edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe9f40011e0 tx=0x7fe9f4007df0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.700 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.701+0000 7fea0b000640 1 -- 172.21.15.121:0/1692037745 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea040b3330 msgr2=0x7fea04158a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.701 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.701+0000 7fea0b000640 1 --2- 172.21.15.121:0/1692037745 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea040b3330 0x7fea04158a30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc00cae0 tx=0x7fe9fc00a060 comp rx=0 tx=0).stop 2024-04-20T10:21:36.701 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.701+0000 7fea0b000640 1 -- 172.21.15.121:0/1692037745 shutdown_connections 2024-04-20T10:21:36.701 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.701+0000 7fea0b000640 1 --2- 172.21.15.121:0/1692037745 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe9e406c8f0 0x7fe9e406edb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.701 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.701+0000 7fea0b000640 1 --2- 172.21.15.121:0/1692037745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea04161d50 0x7fea040cbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.701 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.701+0000 7fea0b000640 1 --2- 172.21.15.121:0/1692037745 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea040b3330 0x7fea04158a30 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.701 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.701+0000 7fea0b000640 1 --2- 172.21.15.121:0/1692037745 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea0406cde0 0x7fea041f51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.701 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.701+0000 7fea0b000640 1 -- 172.21.15.121:0/1692037745 >> 172.21.15.121:0/1692037745 conn(0x7fea040d4450 msgr2=0x7fea04160a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:36.702 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.702+0000 7fea0b000640 1 -- 172.21.15.121:0/1692037745 shutdown_connections 2024-04-20T10:21:36.702 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.702+0000 7fea0b000640 1 -- 172.21.15.121:0/1692037745 wait complete. 2024-04-20T10:21:36.704 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542155 for osd.6 2024-04-20T10:21:36.705 DEBUG:teuthology.parallel:result is None 2024-04-20T10:21:36.705 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542154 for osd.5 2024-04-20T10:21:36.705 DEBUG:teuthology.parallel:result is None 2024-04-20T10:21:36.709 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.709+0000 7f4837200640 1 -- 172.21.15.121:0/869156069 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f48300690f0 con 0x7f483006bdd0 2024-04-20T10:21:36.710 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.710+0000 7f4827e00640 1 -- 172.21.15.121:0/869156069 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f48280b3020 con 0x7f483006bdd0 2024-04-20T10:21:36.710 INFO:teuthology.orchestra.run.smithi121.stdout:60129542154 2024-04-20T10:21:36.713 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542154 for osd.2 2024-04-20T10:21:36.714 DEBUG:teuthology.parallel:result is None 2024-04-20T10:21:36.714 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.714+0000 7f4837200640 1 -- 172.21.15.121:0/869156069 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f481006caf0 msgr2=0x7f481006efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.714 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.714+0000 7f4837200640 1 --2- 172.21.15.121:0/869156069 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f481006caf0 0x7f481006efb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f483009bad0 tx=0x7f481c009190 comp rx=0 tx=0).stop 2024-04-20T10:21:36.714 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.714+0000 7f4837200640 1 -- 172.21.15.121:0/869156069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f483006bdd0 msgr2=0x7f483009ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.714 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.714+0000 7f4837200640 1 --2- 172.21.15.121:0/869156069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f483006bdd0 0x7f483009ca80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4828036000 tx=0x7f4828004a40 comp rx=0 tx=0).stop 2024-04-20T10:21:36.714 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.714+0000 7f4837200640 1 -- 172.21.15.121:0/869156069 shutdown_connections 2024-04-20T10:21:36.714 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.714+0000 7f4837200640 1 --2- 172.21.15.121:0/869156069 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f483015a7d0 0x7f48300aeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.715 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.714+0000 7f4837200640 1 --2- 172.21.15.121:0/869156069 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f483006de00 0x7f48301f0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.715 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.714+0000 7f4837200640 1 --2- 172.21.15.121:0/869156069 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f481006caf0 0x7f481006efb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.715 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.714+0000 7f4837200640 1 --2- 172.21.15.121:0/869156069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f483006bdd0 0x7f483009ca80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.715 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.714+0000 7f4837200640 1 -- 172.21.15.121:0/869156069 >> 172.21.15.121:0/869156069 conn(0x7f4830072090 msgr2=0x7f4830151780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:36.715 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.714+0000 7f4837200640 1 -- 172.21.15.121:0/869156069 shutdown_connections 2024-04-20T10:21:36.715 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.714+0000 7f4837200640 1 -- 172.21.15.121:0/869156069 wait complete. 2024-04-20T10:21:36.720 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542154 for osd.4 2024-04-20T10:21:36.720 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.721+0000 7f7ca2600640 1 -- 172.21.15.121:0/1515278238 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f7c9c0b2470 con 0x7f7c9c0d4380 2024-04-20T10:21:36.721 DEBUG:teuthology.parallel:result is None 2024-04-20T10:21:36.721 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.721+0000 7f7c98c00640 1 -- 172.21.15.121:0/1515278238 <== mon.2 v2:172.21.15.123:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7c9007b4d0 con 0x7f7c9c0d4380 2024-04-20T10:21:36.721 INFO:teuthology.orchestra.run.smithi121.stdout:60129542156 2024-04-20T10:21:36.723 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.724+0000 7f7ca2600640 1 -- 172.21.15.121:0/1515278238 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7c7c06c8f0 msgr2=0x7f7c7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.724 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.724+0000 7f7ca2600640 1 --2- 172.21.15.121:0/1515278238 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7c7c06c8f0 0x7f7c7c06edb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7c88007d50 tx=0x7f7c88007ca0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.724 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.724+0000 7f7ca2600640 1 -- 172.21.15.121:0/1515278238 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7c9c0d4380 msgr2=0x7f7c9c15d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.724 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.724+0000 7f7ca2600640 1 --2- 172.21.15.121:0/1515278238 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7c9c0d4380 0x7f7c9c15d7d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7c90032000 tx=0x7f7c900046c0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.724 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.724+0000 7f7ca2600640 1 -- 172.21.15.121:0/1515278238 shutdown_connections 2024-04-20T10:21:36.724 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.724+0000 7f7ca2600640 1 --2- 172.21.15.121:0/1515278238 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7c7c06c8f0 0x7f7c7c06edb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.724 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.724+0000 7f7ca2600640 1 --2- 172.21.15.121:0/1515278238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c9c0c9a80 0x7f7c9c0cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.724 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.724+0000 7f7ca2600640 1 --2- 172.21.15.121:0/1515278238 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c9c1628e0 0x7f7c9c0c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.724 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.724+0000 7f7ca2600640 1 --2- 172.21.15.121:0/1515278238 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7c9c0d4380 0x7f7c9c15d7d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.724 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.724+0000 7f7ca2600640 1 -- 172.21.15.121:0/1515278238 >> 172.21.15.121:0/1515278238 conn(0x7f7c9c08bf80 msgr2=0x7f7c9c0c7330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:36.724 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.724+0000 7f7ca2600640 1 -- 172.21.15.121:0/1515278238 shutdown_connections 2024-04-20T10:21:36.724 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.724+0000 7f7ca2600640 1 -- 172.21.15.121:0/1515278238 wait complete. 2024-04-20T10:21:36.732 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542154 for osd.7 2024-04-20T10:21:36.733 DEBUG:teuthology.parallel:result is None 2024-04-20T10:21:36.740 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542156 got 60129542156 for osd.0 2024-04-20T10:21:36.740 DEBUG:teuthology.parallel:result is None 2024-04-20T10:21:36.740 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-20T10:21:36.740 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T10:21:36.969 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.969+0000 7f596ee00640 1 Processor -- start 2024-04-20T10:21:36.969 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.970+0000 7f596ee00640 1 -- start start 2024-04-20T10:21:36.971 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.971+0000 7f596ee00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f59680b2f70 0x7f59680b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.972 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.971+0000 7f596ee00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59680adf60 0x7f596815f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.972 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.971+0000 7f596ee00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5968071a80 con 0x7f596815faf0 2024-04-20T10:21:36.972 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.971+0000 7f596ee00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f59680ae3c0 con 0x7f59680b2f70 2024-04-20T10:21:36.972 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.971+0000 7f596ee00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f5968071cf0 con 0x7f59680adf60 2024-04-20T10:21:36.972 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.971+0000 7f596da00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f59680b2f70 0x7f59680b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.972 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.972+0000 7f596e400640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f596815faf0 0x7f596806b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:47604/0 (socket says 172.21.15.121:47604) 2024-04-20T10:21:36.972 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.972+0000 7f596e400640 1 -- 172.21.15.121:0/1520147200 learned_addr learned my addr 172.21.15.121:0/1520147200 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:36.972 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.972+0000 7f596d000640 1 --2- 172.21.15.121:0/1520147200 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59680adf60 0x7f596815f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.972 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.972+0000 7f5967e00640 1 -- 172.21.15.121:0/1520147200 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1778838073 0 0) 0x7f5968071a80 con 0x7f596815faf0 2024-04-20T10:21:36.972 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.972+0000 7f5967e00640 1 -- 172.21.15.121:0/1520147200 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5950003520 con 0x7f596815faf0 2024-04-20T10:21:36.972 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.972+0000 7f596da00640 1 -- 172.21.15.121:0/1520147200 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59680adf60 msgr2=0x7f596815f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:36.972 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.972+0000 7f596da00640 1 --2- 172.21.15.121:0/1520147200 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59680adf60 0x7f596815f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.972+0000 7f596da00640 1 -- 172.21.15.121:0/1520147200 >> v1:172.21.15.121:6789/0 conn(0x7f596815faf0 legacy=0x7f596806b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.972+0000 7f596da00640 1 -- 172.21.15.121:0/1520147200 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5968071fd0 con 0x7f59680b2f70 2024-04-20T10:21:36.974 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.973+0000 7f596da00640 1 --2- 172.21.15.121:0/1520147200 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59680b2f70 0x7f59680b4320 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f595c0097a0 tx=0x7f595c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f52df836e551b287 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:36.974 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.973+0000 7f5967e00640 1 -- 172.21.15.121:0/1520147200 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c035020 con 0x7f59680b2f70 2024-04-20T10:21:36.974 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.973+0000 7f5967e00640 1 -- 172.21.15.121:0/1520147200 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f595c031810 con 0x7f59680b2f70 2024-04-20T10:21:36.974 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.973+0000 7f5967e00640 1 -- 172.21.15.121:0/1520147200 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c031be0 con 0x7f59680b2f70 2024-04-20T10:21:36.974 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.974+0000 7f596ee00640 1 -- 172.21.15.121:0/1520147200 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59680b2f70 msgr2=0x7f59680b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.974 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.974+0000 7f596ee00640 1 --2- 172.21.15.121:0/1520147200 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59680b2f70 0x7f59680b4320 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f595c0097a0 tx=0x7f595c02b8b0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.974 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.974+0000 7f596ee00640 1 -- 172.21.15.121:0/1520147200 shutdown_connections 2024-04-20T10:21:36.974 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.974+0000 7f596ee00640 1 --2- 172.21.15.121:0/1520147200 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f59680adf60 0x7f596815f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.974 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.974+0000 7f596ee00640 1 --2- 172.21.15.121:0/1520147200 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59680b2f70 0x7f59680b4320 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.975 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.974+0000 7f596ee00640 1 -- 172.21.15.121:0/1520147200 >> 172.21.15.121:0/1520147200 conn(0x7f59680d0040 msgr2=0x7f59680d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:36.975 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.974+0000 7f596ee00640 1 -- 172.21.15.121:0/1520147200 shutdown_connections 2024-04-20T10:21:36.975 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.974+0000 7f596ee00640 1 -- 172.21.15.121:0/1520147200 wait complete. 2024-04-20T10:21:36.975 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.976+0000 7f596ee00640 1 Processor -- start 2024-04-20T10:21:36.975 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.976+0000 7f596ee00640 1 -- start start 2024-04-20T10:21:36.977 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.977+0000 7f596ee00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59680adf60 0x7f59680b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.977 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.977+0000 7f596ee00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59680b2f70 0x7f59681e4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.977 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.977+0000 7f596ee00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f596815faf0 0x7f596814c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.977 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.977+0000 7f596ee00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f596806c010 con 0x7f59680adf60 2024-04-20T10:21:36.977 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.977+0000 7f596ee00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f59680736c0 con 0x7f59680b2f70 2024-04-20T10:21:36.977 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.977+0000 7f596ee00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f596806c190 con 0x7f596815faf0 2024-04-20T10:21:36.977 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.977+0000 7f596da00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59680adf60 0x7f59680b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.977 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.977+0000 7f596da00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59680adf60 0x7f59680b35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58580/0 (socket says 172.21.15.121:58580) 2024-04-20T10:21:36.978 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.977+0000 7f596da00640 1 -- 172.21.15.121:0/3025673800 learned_addr learned my addr 172.21.15.121:0/3025673800 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:36.978 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.977+0000 7f596da00640 1 -- 172.21.15.121:0/3025673800 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f596815faf0 msgr2=0x7f596814c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.978 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.977+0000 7f596da00640 1 --2- 172.21.15.121:0/3025673800 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f596815faf0 0x7f596814c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.978 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.977+0000 7f596da00640 1 -- 172.21.15.121:0/3025673800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59680b2f70 msgr2=0x7f59681e4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:36.978 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.977+0000 7f596da00640 1 --2- 172.21.15.121:0/3025673800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59680b2f70 0x7f59681e4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:36.978 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.977+0000 7f596da00640 1 -- 172.21.15.121:0/3025673800 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5968159ea0 con 0x7f59680adf60 2024-04-20T10:21:36.978 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.978+0000 7f596da00640 1 --2- 172.21.15.121:0/3025673800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59680adf60 0x7f59680b35c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f595c009890 tx=0x7f595c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:36.979 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.978+0000 7f5965600640 1 -- 172.21.15.121:0/3025673800 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c035040 con 0x7f59680adf60 2024-04-20T10:21:36.979 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.978+0000 7f5965600640 1 -- 172.21.15.121:0/3025673800 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f595c03ad30 con 0x7f59680adf60 2024-04-20T10:21:36.979 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.978+0000 7f5965600640 1 -- 172.21.15.121:0/3025673800 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c0466a0 con 0x7f59680adf60 2024-04-20T10:21:36.979 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.978+0000 7f596ee00640 1 -- 172.21.15.121:0/3025673800 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f596815a180 con 0x7f59680adf60 2024-04-20T10:21:36.979 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.978+0000 7f596ee00640 1 -- 172.21.15.121:0/3025673800 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f596815d430 con 0x7f59680adf60 2024-04-20T10:21:36.979 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.979+0000 7f596ee00640 1 -- 172.21.15.121:0/3025673800 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59680742b0 con 0x7f59680adf60 2024-04-20T10:21:36.980 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.980+0000 7f5965600640 1 -- 172.21.15.121:0/3025673800 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f595c038020 con 0x7f59680adf60 2024-04-20T10:21:36.981 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.981+0000 7f5965600640 1 --2- 172.21.15.121:0/3025673800 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f594c06c8f0 0x7f594c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:36.981 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.981+0000 7f5965600640 1 -- 172.21.15.121:0/3025673800 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f595c0adfe0 con 0x7f59680adf60 2024-04-20T10:21:36.981 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.982+0000 7f596d000640 1 --2- 172.21.15.121:0/3025673800 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f594c06c8f0 0x7f594c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:36.982 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.982+0000 7f596d000640 1 --2- 172.21.15.121:0/3025673800 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f594c06c8f0 0x7f594c06edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f59680b3f80 tx=0x7f5958008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:36.987 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:36.988+0000 7f5965600640 1 -- 172.21.15.121:0/3025673800 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f59680742b0 con 0x7f59680adf60 2024-04-20T10:21:37.169 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.169+0000 7f596ee00640 1 -- 172.21.15.121:0/3025673800 --> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5968159c50 con 0x7f594c06c8f0 2024-04-20T10:21:37.190 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.190+0000 7f5965600640 1 -- 172.21.15.121:0/3025673800 <== mgr.4103 v2:172.21.15.121:6832/1408233272 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332537 (secure 0 0 0) 0x7f5968159c50 con 0x7f594c06c8f0 2024-04-20T10:21:37.193 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:21:37.196 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-04-20T10:21:37.198 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.197+0000 7f596ee00640 1 -- 172.21.15.121:0/3025673800 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f594c06c8f0 msgr2=0x7f594c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:37.198 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.197+0000 7f596ee00640 1 --2- 172.21.15.121:0/3025673800 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f594c06c8f0 0x7f594c06edb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f59680b3f80 tx=0x7f5958008040 comp rx=0 tx=0).stop 2024-04-20T10:21:37.198 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.197+0000 7f596ee00640 1 -- 172.21.15.121:0/3025673800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59680adf60 msgr2=0x7f59680b35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:37.198 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.197+0000 7f596ee00640 1 --2- 172.21.15.121:0/3025673800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59680adf60 0x7f59680b35c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f595c009890 tx=0x7f595c0046c0 comp rx=0 tx=0).stop 2024-04-20T10:21:37.198 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.198+0000 7f596ee00640 1 -- 172.21.15.121:0/3025673800 shutdown_connections 2024-04-20T10:21:37.198 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.198+0000 7f596ee00640 1 --2- 172.21.15.121:0/3025673800 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f596815faf0 0x7f596814c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:37.198 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.198+0000 7f596ee00640 1 --2- 172.21.15.121:0/3025673800 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f594c06c8f0 0x7f594c06edb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:37.198 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.198+0000 7f596ee00640 1 --2- 172.21.15.121:0/3025673800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59680b2f70 0x7f59681e4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:37.199 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.198+0000 7f596ee00640 1 --2- 172.21.15.121:0/3025673800 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f59680adf60 0x7f59680b35c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:37.199 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.198+0000 7f596ee00640 1 -- 172.21.15.121:0/3025673800 >> 172.21.15.121:0/3025673800 conn(0x7f59680d0040 msgr2=0x7f596806f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:37.199 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.198+0000 7f596ee00640 1 -- 172.21.15.121:0/3025673800 shutdown_connections 2024-04-20T10:21:37.199 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.198+0000 7f596ee00640 1 -- 172.21.15.121:0/3025673800 wait complete. 2024-04-20T10:21:37.215 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2024-04-20T10:21:37.121076+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219436,"kb_used_data":3272,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755284,"statfs":{"total":773094113280,"available":772869410816,"internally_reserved":0,"allocated":3350528,"data_stored":2018488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005504"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.544361+0000","last_change":"2024-04-20T10:21:01.620269+0000","last_active":"2024-04-20T10:21:08.544361+0000","last_peered":"2024-04-20T10:21:08.544361+0000","last_clean":"2024-04-20T10:21:08.544361+0000","last_became_active":"2024-04-20T10:21:01.619751+0000","last_became_peered":"2024-04-20T10:21:01.619751+0000","last_unstale":"2024-04-20T10:21:08.544361+0000","last_undegraded":"2024-04-20T10:21:08.544361+0000","last_fullsized":"2024-04-20T10:21:08.544361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:28:01.100842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.544942+0000","last_change":"2024-04-20T10:21:02.642147+0000","last_active":"2024-04-20T10:21:08.544942+0000","last_peered":"2024-04-20T10:21:08.544942+0000","last_clean":"2024-04-20T10:21:08.544942+0000","last_became_active":"2024-04-20T10:21:02.641636+0000","last_became_peered":"2024-04-20T10:21:02.641636+0000","last_unstale":"2024-04-20T10:21:08.544942+0000","last_undegraded":"2024-04-20T10:21:08.544942+0000","last_fullsized":"2024-04-20T10:21:08.544942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:53:57.418208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540582+0000","last_change":"2024-04-20T10:21:06.909318+0000","last_active":"2024-04-20T10:21:08.540582+0000","last_peered":"2024-04-20T10:21:08.540582+0000","last_clean":"2024-04-20T10:21:08.540582+0000","last_became_active":"2024-04-20T10:21:06.907900+0000","last_became_peered":"2024-04-20T10:21:06.907900+0000","last_unstale":"2024-04-20T10:21:08.540582+0000","last_undegraded":"2024-04-20T10:21:08.540582+0000","last_fullsized":"2024-04-20T10:21:08.540582+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:35:58.510637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.545148+0000","last_change":"2024-04-20T10:21:01.609344+0000","last_active":"2024-04-20T10:21:08.545148+0000","last_peered":"2024-04-20T10:21:08.545148+0000","last_clean":"2024-04-20T10:21:08.545148+0000","last_became_active":"2024-04-20T10:21:01.608863+0000","last_became_peered":"2024-04-20T10:21:01.608863+0000","last_unstale":"2024-04-20T10:21:08.545148+0000","last_undegraded":"2024-04-20T10:21:08.545148+0000","last_fullsized":"2024-04-20T10:21:08.545148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:32:02.981747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.543880+0000","last_change":"2024-04-20T10:21:02.649607+0000","last_active":"2024-04-20T10:21:08.543880+0000","last_peered":"2024-04-20T10:21:08.543880+0000","last_clean":"2024-04-20T10:21:08.543880+0000","last_became_active":"2024-04-20T10:21:02.649283+0000","last_became_peered":"2024-04-20T10:21:02.649283+0000","last_unstale":"2024-04-20T10:21:08.543880+0000","last_undegraded":"2024-04-20T10:21:08.543880+0000","last_fullsized":"2024-04-20T10:21:08.543880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:06:35.349863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.885288+0000","last_change":"2024-04-20T10:21:06.910300+0000","last_active":"2024-04-20T10:21:07.885288+0000","last_peered":"2024-04-20T10:21:07.885288+0000","last_clean":"2024-04-20T10:21:07.885288+0000","last_became_active":"2024-04-20T10:21:06.909341+0000","last_became_peered":"2024-04-20T10:21:06.909341+0000","last_unstale":"2024-04-20T10:21:07.885288+0000","last_undegraded":"2024-04-20T10:21:07.885288+0000","last_fullsized":"2024-04-20T10:21:07.885288+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:06:39.624891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.391510+0000","last_change":"2024-04-20T10:21:01.623714+0000","last_active":"2024-04-20T10:21:08.391510+0000","last_peered":"2024-04-20T10:21:08.391510+0000","last_clean":"2024-04-20T10:21:08.391510+0000","last_became_active":"2024-04-20T10:21:01.623427+0000","last_became_peered":"2024-04-20T10:21:01.623427+0000","last_unstale":"2024-04-20T10:21:08.391510+0000","last_undegraded":"2024-04-20T10:21:08.391510+0000","last_fullsized":"2024-04-20T10:21:08.391510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:25:59.939855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541788+0000","last_change":"2024-04-20T10:21:02.636522+0000","last_active":"2024-04-20T10:21:08.541788+0000","last_peered":"2024-04-20T10:21:08.541788+0000","last_clean":"2024-04-20T10:21:08.541788+0000","last_became_active":"2024-04-20T10:21:02.633235+0000","last_became_peered":"2024-04-20T10:21:02.633235+0000","last_unstale":"2024-04-20T10:21:08.541788+0000","last_undegraded":"2024-04-20T10:21:08.541788+0000","last_fullsized":"2024-04-20T10:21:08.541788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:55:00.905038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.886695+0000","last_change":"2024-04-20T10:21:06.911285+0000","last_active":"2024-04-20T10:21:07.886695+0000","last_peered":"2024-04-20T10:21:07.886695+0000","last_clean":"2024-04-20T10:21:07.886695+0000","last_became_active":"2024-04-20T10:21:06.908548+0000","last_became_peered":"2024-04-20T10:21:06.908548+0000","last_unstale":"2024-04-20T10:21:07.886695+0000","last_undegraded":"2024-04-20T10:21:07.886695+0000","last_fullsized":"2024-04-20T10:21:07.886695+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:09:46.241216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.391163+0000","last_change":"2024-04-20T10:21:01.626685+0000","last_active":"2024-04-20T10:21:08.391163+0000","last_peered":"2024-04-20T10:21:08.391163+0000","last_clean":"2024-04-20T10:21:08.391163+0000","last_became_active":"2024-04-20T10:21:01.626354+0000","last_became_peered":"2024-04-20T10:21:01.626354+0000","last_unstale":"2024-04-20T10:21:08.391163+0000","last_undegraded":"2024-04-20T10:21:08.391163+0000","last_fullsized":"2024-04-20T10:21:08.391163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:07:50.284372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542097+0000","last_change":"2024-04-20T10:21:02.636800+0000","last_active":"2024-04-20T10:21:08.542097+0000","last_peered":"2024-04-20T10:21:08.542097+0000","last_clean":"2024-04-20T10:21:08.542097+0000","last_became_active":"2024-04-20T10:21:02.636523+0000","last_became_peered":"2024-04-20T10:21:02.636523+0000","last_unstale":"2024-04-20T10:21:08.542097+0000","last_undegraded":"2024-04-20T10:21:08.542097+0000","last_fullsized":"2024-04-20T10:21:08.542097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:31:07.363373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.885879+0000","last_change":"2024-04-20T10:21:06.893126+0000","last_active":"2024-04-20T10:21:07.885879+0000","last_peered":"2024-04-20T10:21:07.885879+0000","last_clean":"2024-04-20T10:21:07.885879+0000","last_became_active":"2024-04-20T10:21:06.892722+0000","last_became_peered":"2024-04-20T10:21:06.892722+0000","last_unstale":"2024-04-20T10:21:07.885879+0000","last_undegraded":"2024-04-20T10:21:07.885879+0000","last_fullsized":"2024-04-20T10:21:07.885879+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:10:19.481713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.038387+0000","last_change":"2024-04-20T10:21:02.633548+0000","last_active":"2024-04-20T10:21:08.038387+0000","last_peered":"2024-04-20T10:21:08.038387+0000","last_clean":"2024-04-20T10:21:08.038387+0000","last_became_active":"2024-04-20T10:21:02.631142+0000","last_became_peered":"2024-04-20T10:21:02.631142+0000","last_unstale":"2024-04-20T10:21:08.038387+0000","last_undegraded":"2024-04-20T10:21:08.038387+0000","last_fullsized":"2024-04-20T10:21:08.038387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:14:16.934710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.390972+0000","last_change":"2024-04-20T10:21:01.622399+0000","last_active":"2024-04-20T10:21:08.390972+0000","last_peered":"2024-04-20T10:21:08.390972+0000","last_clean":"2024-04-20T10:21:08.390972+0000","last_became_active":"2024-04-20T10:21:01.622072+0000","last_became_peered":"2024-04-20T10:21:01.622072+0000","last_unstale":"2024-04-20T10:21:08.390972+0000","last_undegraded":"2024-04-20T10:21:08.390972+0000","last_fullsized":"2024-04-20T10:21:08.390972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:35:02.394948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.887355+0000","last_change":"2024-04-20T10:21:06.897024+0000","last_active":"2024-04-20T10:21:07.887355+0000","last_peered":"2024-04-20T10:21:07.887355+0000","last_clean":"2024-04-20T10:21:07.887355+0000","last_became_active":"2024-04-20T10:21:06.896315+0000","last_became_peered":"2024-04-20T10:21:06.896315+0000","last_unstale":"2024-04-20T10:21:07.887355+0000","last_undegraded":"2024-04-20T10:21:07.887355+0000","last_fullsized":"2024-04-20T10:21:07.887355+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:21:29.812831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.544646+0000","last_change":"2024-04-20T10:21:02.646217+0000","last_active":"2024-04-20T10:21:08.544646+0000","last_peered":"2024-04-20T10:21:08.544646+0000","last_clean":"2024-04-20T10:21:08.544646+0000","last_became_active":"2024-04-20T10:21:02.645926+0000","last_became_peered":"2024-04-20T10:21:02.645926+0000","last_unstale":"2024-04-20T10:21:08.544646+0000","last_undegraded":"2024-04-20T10:21:08.544646+0000","last_fullsized":"2024-04-20T10:21:08.544646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:10:15.206686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441134+0000","last_change":"2024-04-20T10:21:01.617948+0000","last_active":"2024-04-20T10:21:08.441134+0000","last_peered":"2024-04-20T10:21:08.441134+0000","last_clean":"2024-04-20T10:21:08.441134+0000","last_became_active":"2024-04-20T10:21:01.617628+0000","last_became_peered":"2024-04-20T10:21:01.617628+0000","last_unstale":"2024-04-20T10:21:08.441134+0000","last_undegraded":"2024-04-20T10:21:08.441134+0000","last_fullsized":"2024-04-20T10:21:08.441134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:55:41.208744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.886801+0000","last_change":"2024-04-20T10:21:06.900164+0000","last_active":"2024-04-20T10:21:07.886801+0000","last_peered":"2024-04-20T10:21:07.886801+0000","last_clean":"2024-04-20T10:21:07.886801+0000","last_became_active":"2024-04-20T10:21:06.899783+0000","last_became_peered":"2024-04-20T10:21:06.899783+0000","last_unstale":"2024-04-20T10:21:07.886801+0000","last_undegraded":"2024-04-20T10:21:07.886801+0000","last_fullsized":"2024-04-20T10:21:07.886801+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:54:01.693236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541211+0000","last_change":"2024-04-20T10:21:02.629982+0000","last_active":"2024-04-20T10:21:08.541211+0000","last_peered":"2024-04-20T10:21:08.541211+0000","last_clean":"2024-04-20T10:21:08.541211+0000","last_became_active":"2024-04-20T10:21:02.629729+0000","last_became_peered":"2024-04-20T10:21:02.629729+0000","last_unstale":"2024-04-20T10:21:08.541211+0000","last_undegraded":"2024-04-20T10:21:08.541211+0000","last_fullsized":"2024-04-20T10:21:08.541211+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:31:53.442042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.440779+0000","last_change":"2024-04-20T10:21:01.604313+0000","last_active":"2024-04-20T10:21:08.440779+0000","last_peered":"2024-04-20T10:21:08.440779+0000","last_clean":"2024-04-20T10:21:08.440779+0000","last_became_active":"2024-04-20T10:21:01.603629+0000","last_became_peered":"2024-04-20T10:21:01.603629+0000","last_unstale":"2024-04-20T10:21:08.440779+0000","last_undegraded":"2024-04-20T10:21:08.440779+0000","last_fullsized":"2024-04-20T10:21:08.440779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:01:19.566351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441622+0000","last_change":"2024-04-20T10:21:06.903477+0000","last_active":"2024-04-20T10:21:08.441622+0000","last_peered":"2024-04-20T10:21:08.441622+0000","last_clean":"2024-04-20T10:21:08.441622+0000","last_became_active":"2024-04-20T10:21:06.903010+0000","last_became_peered":"2024-04-20T10:21:06.903010+0000","last_unstale":"2024-04-20T10:21:08.441622+0000","last_undegraded":"2024-04-20T10:21:08.441622+0000","last_fullsized":"2024-04-20T10:21:08.441622+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:46:45.448591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.885935+0000","last_change":"2024-04-20T10:21:02.614476+0000","last_active":"2024-04-20T10:21:07.885935+0000","last_peered":"2024-04-20T10:21:07.885935+0000","last_clean":"2024-04-20T10:21:07.885935+0000","last_became_active":"2024-04-20T10:21:02.614140+0000","last_became_peered":"2024-04-20T10:21:02.614140+0000","last_unstale":"2024-04-20T10:21:07.885935+0000","last_undegraded":"2024-04-20T10:21:07.885935+0000","last_fullsized":"2024-04-20T10:21:07.885935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:01:20.579875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.389864+0000","last_change":"2024-04-20T10:21:01.619958+0000","last_active":"2024-04-20T10:21:08.389864+0000","last_peered":"2024-04-20T10:21:08.389864+0000","last_clean":"2024-04-20T10:21:08.389864+0000","last_became_active":"2024-04-20T10:21:01.619688+0000","last_became_peered":"2024-04-20T10:21:01.619688+0000","last_unstale":"2024-04-20T10:21:08.389864+0000","last_undegraded":"2024-04-20T10:21:08.389864+0000","last_fullsized":"2024-04-20T10:21:08.389864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:45:51.165353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.887140+0000","last_change":"2024-04-20T10:21:06.908789+0000","last_active":"2024-04-20T10:21:07.887140+0000","last_peered":"2024-04-20T10:21:07.887140+0000","last_clean":"2024-04-20T10:21:07.887140+0000","last_became_active":"2024-04-20T10:21:06.908405+0000","last_became_peered":"2024-04-20T10:21:06.908405+0000","last_unstale":"2024-04-20T10:21:07.887140+0000","last_undegraded":"2024-04-20T10:21:07.887140+0000","last_fullsized":"2024-04-20T10:21:07.887140+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:02:30.991758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.544406+0000","last_change":"2024-04-20T10:21:02.649177+0000","last_active":"2024-04-20T10:21:08.544406+0000","last_peered":"2024-04-20T10:21:08.544406+0000","last_clean":"2024-04-20T10:21:08.544406+0000","last_became_active":"2024-04-20T10:21:02.648861+0000","last_became_peered":"2024-04-20T10:21:02.648861+0000","last_unstale":"2024-04-20T10:21:08.544406+0000","last_undegraded":"2024-04-20T10:21:08.544406+0000","last_fullsized":"2024-04-20T10:21:08.544406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:31:53.442042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.039602+0000","last_change":"2024-04-20T10:21:01.621228+0000","last_active":"2024-04-20T10:21:08.039602+0000","last_peered":"2024-04-20T10:21:08.039602+0000","last_clean":"2024-04-20T10:21:08.039602+0000","last_became_active":"2024-04-20T10:21:01.620525+0000","last_became_peered":"2024-04-20T10:21:01.620525+0000","last_unstale":"2024-04-20T10:21:08.039602+0000","last_undegraded":"2024-04-20T10:21:08.039602+0000","last_fullsized":"2024-04-20T10:21:08.039602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:50:27.716971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.885713+0000","last_change":"2024-04-20T10:21:06.903693+0000","last_active":"2024-04-20T10:21:07.885713+0000","last_peered":"2024-04-20T10:21:07.885713+0000","last_clean":"2024-04-20T10:21:07.885713+0000","last_became_active":"2024-04-20T10:21:06.903087+0000","last_became_peered":"2024-04-20T10:21:06.903087+0000","last_unstale":"2024-04-20T10:21:07.885713+0000","last_undegraded":"2024-04-20T10:21:07.885713+0000","last_fullsized":"2024-04-20T10:21:07.885713+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:22:40.880752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542691+0000","last_change":"2024-04-20T10:21:02.632360+0000","last_active":"2024-04-20T10:21:08.542691+0000","last_peered":"2024-04-20T10:21:08.542691+0000","last_clean":"2024-04-20T10:21:08.542691+0000","last_became_active":"2024-04-20T10:21:02.632062+0000","last_became_peered":"2024-04-20T10:21:02.632062+0000","last_unstale":"2024-04-20T10:21:08.542691+0000","last_undegraded":"2024-04-20T10:21:08.542691+0000","last_fullsized":"2024-04-20T10:21:08.542691+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:06:11.427561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441791+0000","last_change":"2024-04-20T10:21:01.622049+0000","last_active":"2024-04-20T10:21:08.441791+0000","last_peered":"2024-04-20T10:21:08.441791+0000","last_clean":"2024-04-20T10:21:08.441791+0000","last_became_active":"2024-04-20T10:21:01.621779+0000","last_became_peered":"2024-04-20T10:21:01.621779+0000","last_unstale":"2024-04-20T10:21:08.441791+0000","last_undegraded":"2024-04-20T10:21:08.441791+0000","last_fullsized":"2024-04-20T10:21:08.441791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:17:23.552169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540642+0000","last_change":"2024-04-20T10:21:06.908686+0000","last_active":"2024-04-20T10:21:08.540642+0000","last_peered":"2024-04-20T10:21:08.540642+0000","last_clean":"2024-04-20T10:21:08.540642+0000","last_became_active":"2024-04-20T10:21:06.907525+0000","last_became_peered":"2024-04-20T10:21:06.907525+0000","last_unstale":"2024-04-20T10:21:08.540642+0000","last_undegraded":"2024-04-20T10:21:08.540642+0000","last_fullsized":"2024-04-20T10:21:08.540642+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:38:05.651135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441452+0000","last_change":"2024-04-20T10:21:02.633193+0000","last_active":"2024-04-20T10:21:08.441452+0000","last_peered":"2024-04-20T10:21:08.441452+0000","last_clean":"2024-04-20T10:21:08.441452+0000","last_became_active":"2024-04-20T10:21:02.632896+0000","last_became_peered":"2024-04-20T10:21:02.632896+0000","last_unstale":"2024-04-20T10:21:08.441452+0000","last_undegraded":"2024-04-20T10:21:08.441452+0000","last_fullsized":"2024-04-20T10:21:08.441452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:06:35.349863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.391636+0000","last_change":"2024-04-20T10:21:01.624712+0000","last_active":"2024-04-20T10:21:08.391636+0000","last_peered":"2024-04-20T10:21:08.391636+0000","last_clean":"2024-04-20T10:21:08.391636+0000","last_became_active":"2024-04-20T10:21:01.624381+0000","last_became_peered":"2024-04-20T10:21:01.624381+0000","last_unstale":"2024-04-20T10:21:08.391636+0000","last_undegraded":"2024-04-20T10:21:08.391636+0000","last_fullsized":"2024-04-20T10:21:08.391636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:17:23.552169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.543354+0000","last_change":"2024-04-20T10:21:06.903399+0000","last_active":"2024-04-20T10:21:08.543354+0000","last_peered":"2024-04-20T10:21:08.543354+0000","last_clean":"2024-04-20T10:21:08.543354+0000","last_became_active":"2024-04-20T10:21:06.902529+0000","last_became_peered":"2024-04-20T10:21:06.902529+0000","last_unstale":"2024-04-20T10:21:08.543354+0000","last_undegraded":"2024-04-20T10:21:08.543354+0000","last_fullsized":"2024-04-20T10:21:08.543354+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:52:28.298927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542324+0000","last_change":"2024-04-20T10:21:02.637139+0000","last_active":"2024-04-20T10:21:08.542324+0000","last_peered":"2024-04-20T10:21:08.542324+0000","last_clean":"2024-04-20T10:21:08.542324+0000","last_became_active":"2024-04-20T10:21:02.636849+0000","last_became_peered":"2024-04-20T10:21:02.636849+0000","last_unstale":"2024-04-20T10:21:08.542324+0000","last_undegraded":"2024-04-20T10:21:08.542324+0000","last_fullsized":"2024-04-20T10:21:08.542324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:44:52.759993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441617+0000","last_change":"2024-04-20T10:21:01.606832+0000","last_active":"2024-04-20T10:21:08.441617+0000","last_peered":"2024-04-20T10:21:08.441617+0000","last_clean":"2024-04-20T10:21:08.441617+0000","last_became_active":"2024-04-20T10:21:01.606562+0000","last_became_peered":"2024-04-20T10:21:01.606562+0000","last_unstale":"2024-04-20T10:21:08.441617+0000","last_undegraded":"2024-04-20T10:21:08.441617+0000","last_fullsized":"2024-04-20T10:21:08.441617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:50:27.716971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.440686+0000","last_change":"2024-04-20T10:21:06.894216+0000","last_active":"2024-04-20T10:21:08.440686+0000","last_peered":"2024-04-20T10:21:08.440686+0000","last_clean":"2024-04-20T10:21:08.440686+0000","last_became_active":"2024-04-20T10:21:06.893762+0000","last_became_peered":"2024-04-20T10:21:06.893762+0000","last_unstale":"2024-04-20T10:21:08.440686+0000","last_undegraded":"2024-04-20T10:21:08.440686+0000","last_fullsized":"2024-04-20T10:21:08.440686+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:31:30.034892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.887359+0000","last_change":"2024-04-20T10:21:02.648574+0000","last_active":"2024-04-20T10:21:07.887359+0000","last_peered":"2024-04-20T10:21:07.887359+0000","last_clean":"2024-04-20T10:21:07.887359+0000","last_became_active":"2024-04-20T10:21:02.648275+0000","last_became_peered":"2024-04-20T10:21:02.648275+0000","last_unstale":"2024-04-20T10:21:07.887359+0000","last_undegraded":"2024-04-20T10:21:07.887359+0000","last_fullsized":"2024-04-20T10:21:07.887359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:55:42.222268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541444+0000","last_change":"2024-04-20T10:21:01.615460+0000","last_active":"2024-04-20T10:21:08.541444+0000","last_peered":"2024-04-20T10:21:08.541444+0000","last_clean":"2024-04-20T10:21:08.541444+0000","last_became_active":"2024-04-20T10:21:01.614405+0000","last_became_peered":"2024-04-20T10:21:01.614405+0000","last_unstale":"2024-04-20T10:21:08.541444+0000","last_undegraded":"2024-04-20T10:21:08.541444+0000","last_fullsized":"2024-04-20T10:21:08.541444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:51:46.790176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441597+0000","last_change":"2024-04-20T10:21:06.899961+0000","last_active":"2024-04-20T10:21:08.441597+0000","last_peered":"2024-04-20T10:21:08.441597+0000","last_clean":"2024-04-20T10:21:08.441597+0000","last_became_active":"2024-04-20T10:21:06.899574+0000","last_became_peered":"2024-04-20T10:21:06.899574+0000","last_unstale":"2024-04-20T10:21:08.441597+0000","last_undegraded":"2024-04-20T10:21:08.441597+0000","last_fullsized":"2024-04-20T10:21:08.441597+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:59:28.847178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.440949+0000","last_change":"2024-04-20T10:21:02.627320+0000","last_active":"2024-04-20T10:21:08.440949+0000","last_peered":"2024-04-20T10:21:08.440949+0000","last_clean":"2024-04-20T10:21:08.440949+0000","last_became_active":"2024-04-20T10:21:02.627008+0000","last_became_peered":"2024-04-20T10:21:02.627008+0000","last_unstale":"2024-04-20T10:21:08.440949+0000","last_undegraded":"2024-04-20T10:21:08.440949+0000","last_fullsized":"2024-04-20T10:21:08.440949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:01:55.363202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.881645+0000","last_change":"2024-04-20T10:21:01.611725+0000","last_active":"2024-04-20T10:21:07.881645+0000","last_peered":"2024-04-20T10:21:07.881645+0000","last_clean":"2024-04-20T10:21:07.881645+0000","last_became_active":"2024-04-20T10:21:01.609982+0000","last_became_peered":"2024-04-20T10:21:01.609982+0000","last_unstale":"2024-04-20T10:21:07.881645+0000","last_undegraded":"2024-04-20T10:21:07.881645+0000","last_fullsized":"2024-04-20T10:21:07.881645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:32:02.981747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441859+0000","last_change":"2024-04-20T10:21:06.897904+0000","last_active":"2024-04-20T10:21:08.441859+0000","last_peered":"2024-04-20T10:21:08.441859+0000","last_clean":"2024-04-20T10:21:08.441859+0000","last_became_active":"2024-04-20T10:21:06.896628+0000","last_became_peered":"2024-04-20T10:21:06.896628+0000","last_unstale":"2024-04-20T10:21:08.441859+0000","last_undegraded":"2024-04-20T10:21:08.441859+0000","last_fullsized":"2024-04-20T10:21:08.441859+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:35:58.510637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542234+0000","last_change":"2024-04-20T10:21:02.651679+0000","last_active":"2024-04-20T10:21:08.542234+0000","last_peered":"2024-04-20T10:21:08.542234+0000","last_clean":"2024-04-20T10:21:08.542234+0000","last_became_active":"2024-04-20T10:21:02.651336+0000","last_became_peered":"2024-04-20T10:21:02.651336+0000","last_unstale":"2024-04-20T10:21:08.542234+0000","last_undegraded":"2024-04-20T10:21:08.542234+0000","last_fullsized":"2024-04-20T10:21:08.542234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:17:24.565693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.389935+0000","last_change":"2024-04-20T10:21:01.620091+0000","last_active":"2024-04-20T10:21:08.389935+0000","last_peered":"2024-04-20T10:21:08.389935+0000","last_clean":"2024-04-20T10:21:08.389935+0000","last_became_active":"2024-04-20T10:21:01.619747+0000","last_became_peered":"2024-04-20T10:21:01.619747+0000","last_unstale":"2024-04-20T10:21:08.389935+0000","last_undegraded":"2024-04-20T10:21:08.389935+0000","last_fullsized":"2024-04-20T10:21:08.389935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:38:10.686751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.544184+0000","last_change":"2024-04-20T10:21:06.902502+0000","last_active":"2024-04-20T10:21:08.544184+0000","last_peered":"2024-04-20T10:21:08.544184+0000","last_clean":"2024-04-20T10:21:08.544184+0000","last_became_active":"2024-04-20T10:21:06.899513+0000","last_became_peered":"2024-04-20T10:21:06.899513+0000","last_unstale":"2024-04-20T10:21:08.544184+0000","last_undegraded":"2024-04-20T10:21:08.544184+0000","last_fullsized":"2024-04-20T10:21:08.544184+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:46:45.448591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541620+0000","last_change":"2024-04-20T10:21:02.628521+0000","last_active":"2024-04-20T10:21:08.541620+0000","last_peered":"2024-04-20T10:21:08.541620+0000","last_clean":"2024-04-20T10:21:08.541620+0000","last_became_active":"2024-04-20T10:21:02.626414+0000","last_became_peered":"2024-04-20T10:21:02.626414+0000","last_unstale":"2024-04-20T10:21:08.541620+0000","last_undegraded":"2024-04-20T10:21:08.541620+0000","last_fullsized":"2024-04-20T10:21:08.541620+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:01:55.363202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541447+0000","last_change":"2024-04-20T10:21:01.610399+0000","last_active":"2024-04-20T10:21:08.541447+0000","last_peered":"2024-04-20T10:21:08.541447+0000","last_clean":"2024-04-20T10:21:08.541447+0000","last_became_active":"2024-04-20T10:21:01.610116+0000","last_became_peered":"2024-04-20T10:21:01.610116+0000","last_unstale":"2024-04-20T10:21:08.541447+0000","last_undegraded":"2024-04-20T10:21:08.541447+0000","last_fullsized":"2024-04-20T10:21:08.541447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:01:18.578396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.390168+0000","last_change":"2024-04-20T10:20:52.916129+0000","last_active":"2024-04-20T10:21:08.390168+0000","last_peered":"2024-04-20T10:21:08.390168+0000","last_clean":"2024-04-20T10:21:08.390168+0000","last_became_active":"2024-04-20T10:20:52.913922+0000","last_became_peered":"2024-04-20T10:20:52.913922+0000","last_unstale":"2024-04-20T10:21:08.390168+0000","last_undegraded":"2024-04-20T10:21:08.390168+0000","last_fullsized":"2024-04-20T10:21:08.390168+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:20:51.267588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:20:51.267588+0000","last_clean_scrub_stamp":"2024-04-20T10:20:51.267588+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:30:44.465836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.039062+0000","last_change":"2024-04-20T10:21:06.913314+0000","last_active":"2024-04-20T10:21:08.039062+0000","last_peered":"2024-04-20T10:21:08.039062+0000","last_clean":"2024-04-20T10:21:08.039062+0000","last_became_active":"2024-04-20T10:21:06.912896+0000","last_became_peered":"2024-04-20T10:21:06.912896+0000","last_unstale":"2024-04-20T10:21:08.039062+0000","last_undegraded":"2024-04-20T10:21:08.039062+0000","last_fullsized":"2024-04-20T10:21:08.039062+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:16:08.900837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.391138+0000","last_change":"2024-04-20T10:21:02.643414+0000","last_active":"2024-04-20T10:21:08.391138+0000","last_peered":"2024-04-20T10:21:08.391138+0000","last_clean":"2024-04-20T10:21:08.391138+0000","last_became_active":"2024-04-20T10:21:02.641460+0000","last_became_peered":"2024-04-20T10:21:02.641460+0000","last_unstale":"2024-04-20T10:21:08.391138+0000","last_undegraded":"2024-04-20T10:21:08.391138+0000","last_fullsized":"2024-04-20T10:21:08.391138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:24:51.806454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.882750+0000","last_change":"2024-04-20T10:21:01.612256+0000","last_active":"2024-04-20T10:21:07.882750+0000","last_peered":"2024-04-20T10:21:07.882750+0000","last_clean":"2024-04-20T10:21:07.882750+0000","last_became_active":"2024-04-20T10:21:01.611856+0000","last_became_peered":"2024-04-20T10:21:01.611856+0000","last_unstale":"2024-04-20T10:21:07.882750+0000","last_undegraded":"2024-04-20T10:21:07.882750+0000","last_fullsized":"2024-04-20T10:21:07.882750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:59:37.356548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.038723+0000","last_change":"2024-04-20T10:21:06.913797+0000","last_active":"2024-04-20T10:21:08.038723+0000","last_peered":"2024-04-20T10:21:08.038723+0000","last_clean":"2024-04-20T10:21:08.038723+0000","last_became_active":"2024-04-20T10:21:06.912928+0000","last_became_peered":"2024-04-20T10:21:06.912928+0000","last_unstale":"2024-04-20T10:21:08.038723+0000","last_undegraded":"2024-04-20T10:21:08.038723+0000","last_fullsized":"2024-04-20T10:21:08.038723+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:21:29.812831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542069+0000","last_change":"2024-04-20T10:21:02.630629+0000","last_active":"2024-04-20T10:21:08.542069+0000","last_peered":"2024-04-20T10:21:08.542069+0000","last_clean":"2024-04-20T10:21:08.542069+0000","last_became_active":"2024-04-20T10:21:02.623112+0000","last_became_peered":"2024-04-20T10:21:02.623112+0000","last_unstale":"2024-04-20T10:21:08.542069+0000","last_undegraded":"2024-04-20T10:21:08.542069+0000","last_fullsized":"2024-04-20T10:21:08.542069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:37:12.899052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441939+0000","last_change":"2024-04-20T10:21:01.598214+0000","last_active":"2024-04-20T10:21:08.441939+0000","last_peered":"2024-04-20T10:21:08.441939+0000","last_clean":"2024-04-20T10:21:08.441939+0000","last_became_active":"2024-04-20T10:21:01.597260+0000","last_became_peered":"2024-04-20T10:21:01.597260+0000","last_unstale":"2024-04-20T10:21:08.441939+0000","last_undegraded":"2024-04-20T10:21:08.441939+0000","last_fullsized":"2024-04-20T10:21:08.441939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:25:59.939855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.543790+0000","last_change":"2024-04-20T10:21:06.894202+0000","last_active":"2024-04-20T10:21:08.543790+0000","last_peered":"2024-04-20T10:21:08.543790+0000","last_clean":"2024-04-20T10:21:08.543790+0000","last_became_active":"2024-04-20T10:21:06.893620+0000","last_became_peered":"2024-04-20T10:21:06.893620+0000","last_unstale":"2024-04-20T10:21:08.543790+0000","last_undegraded":"2024-04-20T10:21:08.543790+0000","last_fullsized":"2024-04-20T10:21:08.543790+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:31:30.034892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.391863+0000","last_change":"2024-04-20T10:21:02.655710+0000","last_active":"2024-04-20T10:21:08.391863+0000","last_peered":"2024-04-20T10:21:08.391863+0000","last_clean":"2024-04-20T10:21:08.391863+0000","last_became_active":"2024-04-20T10:21:02.655531+0000","last_became_peered":"2024-04-20T10:21:02.655531+0000","last_unstale":"2024-04-20T10:21:08.391863+0000","last_undegraded":"2024-04-20T10:21:08.391863+0000","last_fullsized":"2024-04-20T10:21:08.391863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:27:00.187303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.883115+0000","last_change":"2024-04-20T10:21:01.612100+0000","last_active":"2024-04-20T10:21:07.883115+0000","last_peered":"2024-04-20T10:21:07.883115+0000","last_clean":"2024-04-20T10:21:07.883115+0000","last_became_active":"2024-04-20T10:21:01.610465+0000","last_became_peered":"2024-04-20T10:21:01.610465+0000","last_unstale":"2024-04-20T10:21:07.883115+0000","last_undegraded":"2024-04-20T10:21:07.883115+0000","last_fullsized":"2024-04-20T10:21:07.883115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:01:19.566351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540603+0000","last_change":"2024-04-20T10:21:06.908159+0000","last_active":"2024-04-20T10:21:08.540603+0000","last_peered":"2024-04-20T10:21:08.540603+0000","last_clean":"2024-04-20T10:21:08.540603+0000","last_became_active":"2024-04-20T10:21:06.907687+0000","last_became_peered":"2024-04-20T10:21:06.907687+0000","last_unstale":"2024-04-20T10:21:08.540603+0000","last_undegraded":"2024-04-20T10:21:08.540603+0000","last_fullsized":"2024-04-20T10:21:08.540603+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:17:43.493429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.391503+0000","last_change":"2024-04-20T10:21:02.652617+0000","last_active":"2024-04-20T10:21:08.391503+0000","last_peered":"2024-04-20T10:21:08.391503+0000","last_clean":"2024-04-20T10:21:08.391503+0000","last_became_active":"2024-04-20T10:21:02.652321+0000","last_became_peered":"2024-04-20T10:21:02.652321+0000","last_unstale":"2024-04-20T10:21:08.391503+0000","last_undegraded":"2024-04-20T10:21:08.391503+0000","last_fullsized":"2024-04-20T10:21:08.391503+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:59:24.572150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.887281+0000","last_change":"2024-04-20T10:21:01.608355+0000","last_active":"2024-04-20T10:21:07.887281+0000","last_peered":"2024-04-20T10:21:07.887281+0000","last_clean":"2024-04-20T10:21:07.887281+0000","last_became_active":"2024-04-20T10:21:01.607981+0000","last_became_peered":"2024-04-20T10:21:01.607981+0000","last_unstale":"2024-04-20T10:21:07.887281+0000","last_undegraded":"2024-04-20T10:21:07.887281+0000","last_fullsized":"2024-04-20T10:21:07.887281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:10:57.859781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.885574+0000","last_change":"2024-04-20T10:21:06.900202+0000","last_active":"2024-04-20T10:21:07.885574+0000","last_peered":"2024-04-20T10:21:07.885574+0000","last_clean":"2024-04-20T10:21:07.885574+0000","last_became_active":"2024-04-20T10:21:06.899795+0000","last_became_peered":"2024-04-20T10:21:06.899795+0000","last_unstale":"2024-04-20T10:21:07.885574+0000","last_undegraded":"2024-04-20T10:21:07.885574+0000","last_fullsized":"2024-04-20T10:21:07.885574+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:01:59.638230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.390393+0000","last_change":"2024-04-20T10:21:02.646213+0000","last_active":"2024-04-20T10:21:08.390393+0000","last_peered":"2024-04-20T10:21:08.390393+0000","last_clean":"2024-04-20T10:21:08.390393+0000","last_became_active":"2024-04-20T10:21:02.645881+0000","last_became_peered":"2024-04-20T10:21:02.645881+0000","last_unstale":"2024-04-20T10:21:08.390393+0000","last_undegraded":"2024-04-20T10:21:08.390393+0000","last_fullsized":"2024-04-20T10:21:08.390393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:21:25.537803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.881093+0000","last_change":"2024-04-20T10:21:01.601030+0000","last_active":"2024-04-20T10:21:07.881093+0000","last_peered":"2024-04-20T10:21:07.881093+0000","last_clean":"2024-04-20T10:21:07.881093+0000","last_became_active":"2024-04-20T10:21:01.600702+0000","last_became_peered":"2024-04-20T10:21:01.600702+0000","last_unstale":"2024-04-20T10:21:07.881093+0000","last_undegraded":"2024-04-20T10:21:07.881093+0000","last_fullsized":"2024-04-20T10:21:07.881093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:36:15.966290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.543987+0000","last_change":"2024-04-20T10:21:06.904141+0000","last_active":"2024-04-20T10:21:08.543987+0000","last_peered":"2024-04-20T10:21:08.543987+0000","last_clean":"2024-04-20T10:21:08.543987+0000","last_became_active":"2024-04-20T10:21:06.903212+0000","last_became_peered":"2024-04-20T10:21:06.903212+0000","last_unstale":"2024-04-20T10:21:08.543987+0000","last_undegraded":"2024-04-20T10:21:08.543987+0000","last_fullsized":"2024-04-20T10:21:08.543987+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:59:28.847178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.038508+0000","last_change":"2024-04-20T10:21:02.644636+0000","last_active":"2024-04-20T10:21:08.038508+0000","last_peered":"2024-04-20T10:21:08.038508+0000","last_clean":"2024-04-20T10:21:08.038508+0000","last_became_active":"2024-04-20T10:21:02.644320+0000","last_became_peered":"2024-04-20T10:21:02.644320+0000","last_unstale":"2024-04-20T10:21:08.038508+0000","last_undegraded":"2024-04-20T10:21:08.038508+0000","last_fullsized":"2024-04-20T10:21:08.038508+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:06:59.919488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.038609+0000","last_change":"2024-04-20T10:21:01.616010+0000","last_active":"2024-04-20T10:21:08.038609+0000","last_peered":"2024-04-20T10:21:08.038609+0000","last_clean":"2024-04-20T10:21:08.038609+0000","last_became_active":"2024-04-20T10:21:01.615701+0000","last_became_peered":"2024-04-20T10:21:01.615701+0000","last_unstale":"2024-04-20T10:21:08.038609+0000","last_undegraded":"2024-04-20T10:21:08.038609+0000","last_fullsized":"2024-04-20T10:21:08.038609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:32:02.981747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542308+0000","last_change":"2024-04-20T10:21:06.909393+0000","last_active":"2024-04-20T10:21:08.542308+0000","last_peered":"2024-04-20T10:21:08.542308+0000","last_clean":"2024-04-20T10:21:08.542308+0000","last_became_active":"2024-04-20T10:21:06.907616+0000","last_became_peered":"2024-04-20T10:21:06.907616+0000","last_unstale":"2024-04-20T10:21:08.542308+0000","last_undegraded":"2024-04-20T10:21:08.542308+0000","last_fullsized":"2024-04-20T10:21:08.542308+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:07:32.815495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.390789+0000","last_change":"2024-04-20T10:21:02.654823+0000","last_active":"2024-04-20T10:21:08.390789+0000","last_peered":"2024-04-20T10:21:08.390789+0000","last_clean":"2024-04-20T10:21:08.390789+0000","last_became_active":"2024-04-20T10:21:02.654556+0000","last_became_peered":"2024-04-20T10:21:02.654556+0000","last_unstale":"2024-04-20T10:21:08.390789+0000","last_undegraded":"2024-04-20T10:21:08.390789+0000","last_fullsized":"2024-04-20T10:21:08.390789+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:52:24.023900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541418+0000","last_change":"2024-04-20T10:21:01.618588+0000","last_active":"2024-04-20T10:21:08.541418+0000","last_peered":"2024-04-20T10:21:08.541418+0000","last_clean":"2024-04-20T10:21:08.541418+0000","last_became_active":"2024-04-20T10:21:01.618294+0000","last_became_peered":"2024-04-20T10:21:01.618294+0000","last_unstale":"2024-04-20T10:21:08.541418+0000","last_undegraded":"2024-04-20T10:21:08.541418+0000","last_fullsized":"2024-04-20T10:21:08.541418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:34:27.492302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542602+0000","last_change":"2024-04-20T10:21:06.897913+0000","last_active":"2024-04-20T10:21:08.542602+0000","last_peered":"2024-04-20T10:21:08.542602+0000","last_clean":"2024-04-20T10:21:08.542602+0000","last_became_active":"2024-04-20T10:21:06.897536+0000","last_became_peered":"2024-04-20T10:21:06.897536+0000","last_unstale":"2024-04-20T10:21:08.542602+0000","last_undegraded":"2024-04-20T10:21:08.542602+0000","last_fullsized":"2024-04-20T10:21:08.542602+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:27:49.300931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.543513+0000","last_change":"2024-04-20T10:21:02.644302+0000","last_active":"2024-04-20T10:21:08.543513+0000","last_peered":"2024-04-20T10:21:08.543513+0000","last_clean":"2024-04-20T10:21:08.543513+0000","last_became_active":"2024-04-20T10:21:02.643236+0000","last_became_peered":"2024-04-20T10:21:02.643236+0000","last_unstale":"2024-04-20T10:21:08.543513+0000","last_undegraded":"2024-04-20T10:21:08.543513+0000","last_fullsized":"2024-04-20T10:21:08.543513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:06:59.919488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.544270+0000","last_change":"2024-04-20T10:21:01.604692+0000","last_active":"2024-04-20T10:21:08.544270+0000","last_peered":"2024-04-20T10:21:08.544270+0000","last_clean":"2024-04-20T10:21:08.544270+0000","last_became_active":"2024-04-20T10:21:01.604345+0000","last_became_peered":"2024-04-20T10:21:01.604345+0000","last_unstale":"2024-04-20T10:21:08.544270+0000","last_undegraded":"2024-04-20T10:21:08.544270+0000","last_fullsized":"2024-04-20T10:21:08.544270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:29:47.733236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542165+0000","last_change":"2024-04-20T10:21:06.901480+0000","last_active":"2024-04-20T10:21:08.542165+0000","last_peered":"2024-04-20T10:21:08.542165+0000","last_clean":"2024-04-20T10:21:08.542165+0000","last_became_active":"2024-04-20T10:21:06.900820+0000","last_became_peered":"2024-04-20T10:21:06.900820+0000","last_unstale":"2024-04-20T10:21:08.542165+0000","last_undegraded":"2024-04-20T10:21:08.542165+0000","last_fullsized":"2024-04-20T10:21:08.542165+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:15:30.919284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.391163+0000","last_change":"2024-04-20T10:21:02.642723+0000","last_active":"2024-04-20T10:21:08.391163+0000","last_peered":"2024-04-20T10:21:08.391163+0000","last_clean":"2024-04-20T10:21:08.391163+0000","last_became_active":"2024-04-20T10:21:02.641828+0000","last_became_peered":"2024-04-20T10:21:02.641828+0000","last_unstale":"2024-04-20T10:21:08.391163+0000","last_undegraded":"2024-04-20T10:21:08.391163+0000","last_fullsized":"2024-04-20T10:21:08.391163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:31:25.759865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:21.609415+0000","last_change":"2024-04-20T10:21:01.619387+0000","last_active":"2024-04-20T10:21:21.609415+0000","last_peered":"2024-04-20T10:21:21.609415+0000","last_clean":"2024-04-20T10:21:21.609415+0000","last_became_active":"2024-04-20T10:21:01.619119+0000","last_became_peered":"2024-04-20T10:21:01.619119+0000","last_unstale":"2024-04-20T10:21:21.609415+0000","last_undegraded":"2024-04-20T10:21:21.609415+0000","last_fullsized":"2024-04-20T10:21:21.609415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:55:41.208744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441019+0000","last_change":"2024-04-20T10:21:06.911043+0000","last_active":"2024-04-20T10:21:08.441019+0000","last_peered":"2024-04-20T10:21:08.441019+0000","last_clean":"2024-04-20T10:21:08.441019+0000","last_became_active":"2024-04-20T10:21:06.909896+0000","last_became_peered":"2024-04-20T10:21:06.909896+0000","last_unstale":"2024-04-20T10:21:08.441019+0000","last_undegraded":"2024-04-20T10:21:08.441019+0000","last_fullsized":"2024-04-20T10:21:08.441019+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:24:56.081482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.887153+0000","last_change":"2024-04-20T10:21:02.624575+0000","last_active":"2024-04-20T10:21:07.887153+0000","last_peered":"2024-04-20T10:21:07.887153+0000","last_clean":"2024-04-20T10:21:07.887153+0000","last_became_active":"2024-04-20T10:21:02.624230+0000","last_became_peered":"2024-04-20T10:21:02.624230+0000","last_unstale":"2024-04-20T10:21:07.887153+0000","last_undegraded":"2024-04-20T10:21:07.887153+0000","last_fullsized":"2024-04-20T10:21:07.887153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:07:51.297896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441862+0000","last_change":"2024-04-20T10:21:01.619231+0000","last_active":"2024-04-20T10:21:08.441862+0000","last_peered":"2024-04-20T10:21:08.441862+0000","last_clean":"2024-04-20T10:21:08.441862+0000","last_became_active":"2024-04-20T10:21:01.618872+0000","last_became_peered":"2024-04-20T10:21:01.618872+0000","last_unstale":"2024-04-20T10:21:08.441862+0000","last_undegraded":"2024-04-20T10:21:08.441862+0000","last_fullsized":"2024-04-20T10:21:08.441862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:38:10.686751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.886989+0000","last_change":"2024-04-20T10:21:06.908460+0000","last_active":"2024-04-20T10:21:07.886989+0000","last_peered":"2024-04-20T10:21:07.886989+0000","last_clean":"2024-04-20T10:21:07.886989+0000","last_became_active":"2024-04-20T10:21:06.907722+0000","last_became_peered":"2024-04-20T10:21:06.907722+0000","last_unstale":"2024-04-20T10:21:07.886989+0000","last_undegraded":"2024-04-20T10:21:07.886989+0000","last_fullsized":"2024-04-20T10:21:07.886989+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:06:15.702589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.544085+0000","last_change":"2024-04-20T10:21:02.646092+0000","last_active":"2024-04-20T10:21:08.544085+0000","last_peered":"2024-04-20T10:21:08.544085+0000","last_clean":"2024-04-20T10:21:08.544085+0000","last_became_active":"2024-04-20T10:21:02.645788+0000","last_became_peered":"2024-04-20T10:21:02.645788+0000","last_unstale":"2024-04-20T10:21:08.544085+0000","last_undegraded":"2024-04-20T10:21:08.544085+0000","last_fullsized":"2024-04-20T10:21:08.544085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:22:36.605724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542309+0000","last_change":"2024-04-20T10:21:01.623128+0000","last_active":"2024-04-20T10:21:08.542309+0000","last_peered":"2024-04-20T10:21:08.542309+0000","last_clean":"2024-04-20T10:21:08.542309+0000","last_became_active":"2024-04-20T10:21:01.622867+0000","last_became_peered":"2024-04-20T10:21:01.622867+0000","last_unstale":"2024-04-20T10:21:08.542309+0000","last_undegraded":"2024-04-20T10:21:08.542309+0000","last_fullsized":"2024-04-20T10:21:08.542309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:50:23.957089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541079+0000","last_change":"2024-04-20T10:21:06.915147+0000","last_active":"2024-04-20T10:21:08.541079+0000","last_peered":"2024-04-20T10:21:08.541079+0000","last_clean":"2024-04-20T10:21:08.541079+0000","last_became_active":"2024-04-20T10:21:06.913832+0000","last_became_peered":"2024-04-20T10:21:06.913832+0000","last_unstale":"2024-04-20T10:21:08.541079+0000","last_undegraded":"2024-04-20T10:21:08.541079+0000","last_fullsized":"2024-04-20T10:21:08.541079+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:10:39.166791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542260+0000","last_change":"2024-04-20T10:21:02.638354+0000","last_active":"2024-04-20T10:21:08.542260+0000","last_peered":"2024-04-20T10:21:08.542260+0000","last_clean":"2024-04-20T10:21:08.542260+0000","last_became_active":"2024-04-20T10:21:02.638065+0000","last_became_peered":"2024-04-20T10:21:02.638065+0000","last_unstale":"2024-04-20T10:21:08.542260+0000","last_undegraded":"2024-04-20T10:21:08.542260+0000","last_fullsized":"2024-04-20T10:21:08.542260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:12:12.144216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.039425+0000","last_change":"2024-04-20T10:21:01.621766+0000","last_active":"2024-04-20T10:21:08.039425+0000","last_peered":"2024-04-20T10:21:08.039425+0000","last_clean":"2024-04-20T10:21:08.039425+0000","last_became_active":"2024-04-20T10:21:01.621546+0000","last_became_peered":"2024-04-20T10:21:01.621546+0000","last_unstale":"2024-04-20T10:21:08.039425+0000","last_undegraded":"2024-04-20T10:21:08.039425+0000","last_fullsized":"2024-04-20T10:21:08.039425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:01:19.566351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540739+0000","last_change":"2024-04-20T10:21:06.910220+0000","last_active":"2024-04-20T10:21:08.540739+0000","last_peered":"2024-04-20T10:21:08.540739+0000","last_clean":"2024-04-20T10:21:08.540739+0000","last_became_active":"2024-04-20T10:21:06.909876+0000","last_became_peered":"2024-04-20T10:21:06.909876+0000","last_unstale":"2024-04-20T10:21:08.540739+0000","last_undegraded":"2024-04-20T10:21:08.540739+0000","last_fullsized":"2024-04-20T10:21:08.540739+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:27:04.462330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.543797+0000","last_change":"2024-04-20T10:21:06.903823+0000","last_active":"2024-04-20T10:21:08.543797+0000","last_peered":"2024-04-20T10:21:08.543797+0000","last_clean":"2024-04-20T10:21:08.543797+0000","last_became_active":"2024-04-20T10:21:06.903100+0000","last_became_peered":"2024-04-20T10:21:06.903100+0000","last_unstale":"2024-04-20T10:21:08.543797+0000","last_undegraded":"2024-04-20T10:21:08.543797+0000","last_fullsized":"2024-04-20T10:21:08.543797+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:24:56.081482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.440526+0000","last_change":"2024-04-20T10:21:02.636194+0000","last_active":"2024-04-20T10:21:08.440526+0000","last_peered":"2024-04-20T10:21:08.440526+0000","last_clean":"2024-04-20T10:21:08.440526+0000","last_became_active":"2024-04-20T10:21:02.635907+0000","last_became_peered":"2024-04-20T10:21:02.635907+0000","last_unstale":"2024-04-20T10:21:08.440526+0000","last_undegraded":"2024-04-20T10:21:08.440526+0000","last_fullsized":"2024-04-20T10:21:08.440526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:06:59.919488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540844+0000","last_change":"2024-04-20T10:21:01.617415+0000","last_active":"2024-04-20T10:21:08.540844+0000","last_peered":"2024-04-20T10:21:08.540844+0000","last_clean":"2024-04-20T10:21:08.540844+0000","last_became_active":"2024-04-20T10:21:01.617082+0000","last_became_peered":"2024-04-20T10:21:01.617082+0000","last_unstale":"2024-04-20T10:21:08.540844+0000","last_undegraded":"2024-04-20T10:21:08.540844+0000","last_fullsized":"2024-04-20T10:21:08.540844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:56:45.348182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.039217+0000","last_change":"2024-04-20T10:21:06.915304+0000","last_active":"2024-04-20T10:21:08.039217+0000","last_peered":"2024-04-20T10:21:08.039217+0000","last_clean":"2024-04-20T10:21:08.039217+0000","last_became_active":"2024-04-20T10:21:06.914915+0000","last_became_peered":"2024-04-20T10:21:06.914915+0000","last_unstale":"2024-04-20T10:21:08.039217+0000","last_undegraded":"2024-04-20T10:21:08.039217+0000","last_fullsized":"2024-04-20T10:21:08.039217+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:52:28.298927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540918+0000","last_change":"2024-04-20T10:21:02.646474+0000","last_active":"2024-04-20T10:21:08.540918+0000","last_peered":"2024-04-20T10:21:08.540918+0000","last_clean":"2024-04-20T10:21:08.540918+0000","last_became_active":"2024-04-20T10:21:02.646219+0000","last_became_peered":"2024-04-20T10:21:02.646219+0000","last_unstale":"2024-04-20T10:21:08.540918+0000","last_undegraded":"2024-04-20T10:21:08.540918+0000","last_fullsized":"2024-04-20T10:21:08.540918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:06:59.919488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541015+0000","last_change":"2024-04-20T10:21:01.609406+0000","last_active":"2024-04-20T10:21:08.541015+0000","last_peered":"2024-04-20T10:21:08.541015+0000","last_clean":"2024-04-20T10:21:08.541015+0000","last_became_active":"2024-04-20T10:21:01.609064+0000","last_became_peered":"2024-04-20T10:21:01.609064+0000","last_unstale":"2024-04-20T10:21:08.541015+0000","last_undegraded":"2024-04-20T10:21:08.541015+0000","last_fullsized":"2024-04-20T10:21:08.541015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:28:01.100842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.039237+0000","last_change":"2024-04-20T10:21:01.615891+0000","last_active":"2024-04-20T10:21:08.039237+0000","last_peered":"2024-04-20T10:21:08.039237+0000","last_clean":"2024-04-20T10:21:08.039237+0000","last_became_active":"2024-04-20T10:21:01.615584+0000","last_became_peered":"2024-04-20T10:21:01.615584+0000","last_unstale":"2024-04-20T10:21:08.039237+0000","last_undegraded":"2024-04-20T10:21:08.039237+0000","last_fullsized":"2024-04-20T10:21:08.039237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:59:37.356548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.544232+0000","last_change":"2024-04-20T10:21:02.643605+0000","last_active":"2024-04-20T10:21:08.544232+0000","last_peered":"2024-04-20T10:21:08.544232+0000","last_clean":"2024-04-20T10:21:08.544232+0000","last_became_active":"2024-04-20T10:21:02.643327+0000","last_became_peered":"2024-04-20T10:21:02.643327+0000","last_unstale":"2024-04-20T10:21:08.544232+0000","last_undegraded":"2024-04-20T10:21:08.544232+0000","last_fullsized":"2024-04-20T10:21:08.544232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:01:55.363202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.882164+0000","last_change":"2024-04-20T10:21:06.892955+0000","last_active":"2024-04-20T10:21:07.882164+0000","last_peered":"2024-04-20T10:21:07.882164+0000","last_clean":"2024-04-20T10:21:07.882164+0000","last_became_active":"2024-04-20T10:21:06.892534+0000","last_became_peered":"2024-04-20T10:21:06.892534+0000","last_unstale":"2024-04-20T10:21:07.882164+0000","last_undegraded":"2024-04-20T10:21:07.882164+0000","last_fullsized":"2024-04-20T10:21:07.882164+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:17:28.840720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541245+0000","last_change":"2024-04-20T10:21:01.618877+0000","last_active":"2024-04-20T10:21:08.541245+0000","last_peered":"2024-04-20T10:21:08.541245+0000","last_clean":"2024-04-20T10:21:08.541245+0000","last_became_active":"2024-04-20T10:21:01.618549+0000","last_became_peered":"2024-04-20T10:21:01.618549+0000","last_unstale":"2024-04-20T10:21:08.541245+0000","last_undegraded":"2024-04-20T10:21:08.541245+0000","last_fullsized":"2024-04-20T10:21:08.541245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:36:15.966290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.886244+0000","last_change":"2024-04-20T10:21:02.645042+0000","last_active":"2024-04-20T10:21:07.886244+0000","last_peered":"2024-04-20T10:21:07.886244+0000","last_clean":"2024-04-20T10:21:07.886244+0000","last_became_active":"2024-04-20T10:21:02.644630+0000","last_became_peered":"2024-04-20T10:21:02.644630+0000","last_unstale":"2024-04-20T10:21:07.886244+0000","last_undegraded":"2024-04-20T10:21:07.886244+0000","last_fullsized":"2024-04-20T10:21:07.886244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:59:38.370072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542570+0000","last_change":"2024-04-20T10:21:06.888033+0000","last_active":"2024-04-20T10:21:08.542570+0000","last_peered":"2024-04-20T10:21:08.542570+0000","last_clean":"2024-04-20T10:21:08.542570+0000","last_became_active":"2024-04-20T10:21:06.887672+0000","last_became_peered":"2024-04-20T10:21:06.887672+0000","last_unstale":"2024-04-20T10:21:08.542570+0000","last_undegraded":"2024-04-20T10:21:08.542570+0000","last_fullsized":"2024-04-20T10:21:08.542570+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:13:54.194111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.038167+0000","last_change":"2024-04-20T10:21:02.644672+0000","last_active":"2024-04-20T10:21:08.038167+0000","last_peered":"2024-04-20T10:21:08.038167+0000","last_clean":"2024-04-20T10:21:08.038167+0000","last_became_active":"2024-04-20T10:21:02.644327+0000","last_became_peered":"2024-04-20T10:21:02.644327+0000","last_unstale":"2024-04-20T10:21:08.038167+0000","last_undegraded":"2024-04-20T10:21:08.038167+0000","last_fullsized":"2024-04-20T10:21:08.038167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:28:54.408773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.881607+0000","last_change":"2024-04-20T10:21:02.624910+0000","last_active":"2024-04-20T10:21:07.881607+0000","last_peered":"2024-04-20T10:21:07.881607+0000","last_clean":"2024-04-20T10:21:07.881607+0000","last_became_active":"2024-04-20T10:21:02.624555+0000","last_became_peered":"2024-04-20T10:21:02.624555+0000","last_unstale":"2024-04-20T10:21:07.881607+0000","last_undegraded":"2024-04-20T10:21:07.881607+0000","last_fullsized":"2024-04-20T10:21:07.881607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:07:51.297896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541176+0000","last_change":"2024-04-20T10:21:02.636958+0000","last_active":"2024-04-20T10:21:08.541176+0000","last_peered":"2024-04-20T10:21:08.541176+0000","last_clean":"2024-04-20T10:21:08.541176+0000","last_became_active":"2024-04-20T10:21:02.636594+0000","last_became_peered":"2024-04-20T10:21:02.636594+0000","last_unstale":"2024-04-20T10:21:08.541176+0000","last_undegraded":"2024-04-20T10:21:08.541176+0000","last_fullsized":"2024-04-20T10:21:08.541176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:12:07.469367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.543932+0000","last_change":"2024-04-20T10:21:02.649761+0000","last_active":"2024-04-20T10:21:08.543932+0000","last_peered":"2024-04-20T10:21:08.543932+0000","last_clean":"2024-04-20T10:21:08.543932+0000","last_became_active":"2024-04-20T10:21:02.649471+0000","last_became_peered":"2024-04-20T10:21:02.649471+0000","last_unstale":"2024-04-20T10:21:08.543932+0000","last_undegraded":"2024-04-20T10:21:08.543932+0000","last_fullsized":"2024-04-20T10:21:08.543932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:14:16.934710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.543183+0000","last_change":"2024-04-20T10:21:02.644223+0000","last_active":"2024-04-20T10:21:08.543183+0000","last_peered":"2024-04-20T10:21:08.543183+0000","last_clean":"2024-04-20T10:21:08.543183+0000","last_became_active":"2024-04-20T10:21:02.643893+0000","last_became_peered":"2024-04-20T10:21:02.643893+0000","last_unstale":"2024-04-20T10:21:08.543183+0000","last_undegraded":"2024-04-20T10:21:08.543183+0000","last_fullsized":"2024-04-20T10:21:08.543183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:09:41.966188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441369+0000","last_change":"2024-04-20T10:21:02.636144+0000","last_active":"2024-04-20T10:21:08.441369+0000","last_peered":"2024-04-20T10:21:08.441369+0000","last_clean":"2024-04-20T10:21:08.441369+0000","last_became_active":"2024-04-20T10:21:02.635834+0000","last_became_peered":"2024-04-20T10:21:02.635834+0000","last_unstale":"2024-04-20T10:21:08.441369+0000","last_undegraded":"2024-04-20T10:21:08.441369+0000","last_fullsized":"2024-04-20T10:21:08.441369+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:10:15.206686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.039750+0000","last_change":"2024-04-20T10:21:02.643146+0000","last_active":"2024-04-20T10:21:08.039750+0000","last_peered":"2024-04-20T10:21:08.039750+0000","last_clean":"2024-04-20T10:21:08.039750+0000","last_became_active":"2024-04-20T10:21:02.639521+0000","last_became_peered":"2024-04-20T10:21:02.639521+0000","last_unstale":"2024-04-20T10:21:08.039750+0000","last_undegraded":"2024-04-20T10:21:08.039750+0000","last_fullsized":"2024-04-20T10:21:08.039750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:01:55.363202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542473+0000","last_change":"2024-04-20T10:21:02.631553+0000","last_active":"2024-04-20T10:21:08.542473+0000","last_peered":"2024-04-20T10:21:08.542473+0000","last_clean":"2024-04-20T10:21:08.542473+0000","last_became_active":"2024-04-20T10:21:02.626210+0000","last_became_peered":"2024-04-20T10:21:02.626210+0000","last_unstale":"2024-04-20T10:21:08.542473+0000","last_undegraded":"2024-04-20T10:21:08.542473+0000","last_fullsized":"2024-04-20T10:21:08.542473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:02:26.716730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540787+0000","last_change":"2024-04-20T10:21:02.625457+0000","last_active":"2024-04-20T10:21:08.540787+0000","last_peered":"2024-04-20T10:21:08.540787+0000","last_clean":"2024-04-20T10:21:08.540787+0000","last_became_active":"2024-04-20T10:21:02.624987+0000","last_became_peered":"2024-04-20T10:21:02.624987+0000","last_unstale":"2024-04-20T10:21:08.540787+0000","last_undegraded":"2024-04-20T10:21:08.540787+0000","last_fullsized":"2024-04-20T10:21:08.540787+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:13:44.633309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540862+0000","last_change":"2024-04-20T10:21:02.628208+0000","last_active":"2024-04-20T10:21:08.540862+0000","last_peered":"2024-04-20T10:21:08.540862+0000","last_clean":"2024-04-20T10:21:08.540862+0000","last_became_active":"2024-04-20T10:21:02.625255+0000","last_became_peered":"2024-04-20T10:21:02.625255+0000","last_unstale":"2024-04-20T10:21:08.540862+0000","last_undegraded":"2024-04-20T10:21:08.540862+0000","last_fullsized":"2024-04-20T10:21:08.540862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:06:35.349863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541152+0000","last_change":"2024-04-20T10:21:02.629528+0000","last_active":"2024-04-20T10:21:08.541152+0000","last_peered":"2024-04-20T10:21:08.541152+0000","last_clean":"2024-04-20T10:21:08.541152+0000","last_became_active":"2024-04-20T10:21:02.626300+0000","last_became_peered":"2024-04-20T10:21:02.626300+0000","last_unstale":"2024-04-20T10:21:08.541152+0000","last_undegraded":"2024-04-20T10:21:08.541152+0000","last_fullsized":"2024-04-20T10:21:08.541152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:17:15.177678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.543489+0000","last_change":"2024-04-20T10:21:02.657102+0000","last_active":"2024-04-20T10:21:08.543489+0000","last_peered":"2024-04-20T10:21:08.543489+0000","last_clean":"2024-04-20T10:21:08.543489+0000","last_became_active":"2024-04-20T10:21:02.656810+0000","last_became_peered":"2024-04-20T10:21:02.656810+0000","last_unstale":"2024-04-20T10:21:08.543489+0000","last_undegraded":"2024-04-20T10:21:08.543489+0000","last_fullsized":"2024-04-20T10:21:08.543489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:04:04.238410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.039198+0000","last_change":"2024-04-20T10:21:02.639499+0000","last_active":"2024-04-20T10:21:08.039198+0000","last_peered":"2024-04-20T10:21:08.039198+0000","last_clean":"2024-04-20T10:21:08.039198+0000","last_became_active":"2024-04-20T10:21:02.638713+0000","last_became_peered":"2024-04-20T10:21:02.638713+0000","last_unstale":"2024-04-20T10:21:08.039198+0000","last_undegraded":"2024-04-20T10:21:08.039198+0000","last_fullsized":"2024-04-20T10:21:08.039198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:31:53.442042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.391493+0000","last_change":"2024-04-20T10:21:02.639481+0000","last_active":"2024-04-20T10:21:08.391493+0000","last_peered":"2024-04-20T10:21:08.391493+0000","last_clean":"2024-04-20T10:21:08.391493+0000","last_became_active":"2024-04-20T10:21:02.639198+0000","last_became_peered":"2024-04-20T10:21:02.639198+0000","last_unstale":"2024-04-20T10:21:08.391493+0000","last_undegraded":"2024-04-20T10:21:08.391493+0000","last_fullsized":"2024-04-20T10:21:08.391493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:35:54.235610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.883015+0000","last_change":"2024-04-20T10:21:02.640221+0000","last_active":"2024-04-20T10:21:07.883015+0000","last_peered":"2024-04-20T10:21:07.883015+0000","last_clean":"2024-04-20T10:21:07.883015+0000","last_became_active":"2024-04-20T10:21:02.639833+0000","last_became_peered":"2024-04-20T10:21:02.639833+0000","last_unstale":"2024-04-20T10:21:07.883015+0000","last_undegraded":"2024-04-20T10:21:07.883015+0000","last_fullsized":"2024-04-20T10:21:07.883015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:38:11.700275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542925+0000","last_change":"2024-04-20T10:21:02.640233+0000","last_active":"2024-04-20T10:21:08.542925+0000","last_peered":"2024-04-20T10:21:08.542925+0000","last_clean":"2024-04-20T10:21:08.542925+0000","last_became_active":"2024-04-20T10:21:02.639913+0000","last_became_peered":"2024-04-20T10:21:02.639913+0000","last_unstale":"2024-04-20T10:21:08.542925+0000","last_undegraded":"2024-04-20T10:21:08.542925+0000","last_fullsized":"2024-04-20T10:21:08.542925+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:45:33.753385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540514+0000","last_change":"2024-04-20T10:21:02.643714+0000","last_active":"2024-04-20T10:21:08.540514+0000","last_peered":"2024-04-20T10:21:08.540514+0000","last_clean":"2024-04-20T10:21:08.540514+0000","last_became_active":"2024-04-20T10:21:02.643403+0000","last_became_peered":"2024-04-20T10:21:02.643403+0000","last_unstale":"2024-04-20T10:21:08.540514+0000","last_undegraded":"2024-04-20T10:21:08.540514+0000","last_fullsized":"2024-04-20T10:21:08.540514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:27:44.403086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.440671+0000","last_change":"2024-04-20T10:21:02.643476+0000","last_active":"2024-04-20T10:21:08.440671+0000","last_peered":"2024-04-20T10:21:08.440671+0000","last_clean":"2024-04-20T10:21:08.440671+0000","last_became_active":"2024-04-20T10:21:02.643180+0000","last_became_peered":"2024-04-20T10:21:02.643180+0000","last_unstale":"2024-04-20T10:21:08.440671+0000","last_undegraded":"2024-04-20T10:21:08.440671+0000","last_fullsized":"2024-04-20T10:21:08.440671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:31:53.442042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.543269+0000","last_change":"2024-04-20T10:21:02.646081+0000","last_active":"2024-04-20T10:21:08.543269+0000","last_peered":"2024-04-20T10:21:08.543269+0000","last_clean":"2024-04-20T10:21:08.543269+0000","last_became_active":"2024-04-20T10:21:02.645798+0000","last_became_peered":"2024-04-20T10:21:02.645798+0000","last_unstale":"2024-04-20T10:21:08.543269+0000","last_undegraded":"2024-04-20T10:21:08.543269+0000","last_fullsized":"2024-04-20T10:21:08.543269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:45:52.178877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540989+0000","last_change":"2024-04-20T10:21:02.639147+0000","last_active":"2024-04-20T10:21:08.540989+0000","last_peered":"2024-04-20T10:21:08.540989+0000","last_clean":"2024-04-20T10:21:08.540989+0000","last_became_active":"2024-04-20T10:21:02.638861+0000","last_became_peered":"2024-04-20T10:21:02.638861+0000","last_unstale":"2024-04-20T10:21:08.540989+0000","last_undegraded":"2024-04-20T10:21:08.540989+0000","last_fullsized":"2024-04-20T10:21:08.540989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:22:36.605724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.887047+0000","last_change":"2024-04-20T10:21:02.638429+0000","last_active":"2024-04-20T10:21:07.887047+0000","last_peered":"2024-04-20T10:21:07.887047+0000","last_clean":"2024-04-20T10:21:07.887047+0000","last_became_active":"2024-04-20T10:21:02.638131+0000","last_became_peered":"2024-04-20T10:21:02.638131+0000","last_unstale":"2024-04-20T10:21:07.887047+0000","last_undegraded":"2024-04-20T10:21:07.887047+0000","last_fullsized":"2024-04-20T10:21:07.887047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:50:28.730495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.039192+0000","last_change":"2024-04-20T10:21:02.646725+0000","last_active":"2024-04-20T10:21:08.039192+0000","last_peered":"2024-04-20T10:21:08.039192+0000","last_clean":"2024-04-20T10:21:08.039192+0000","last_became_active":"2024-04-20T10:21:02.646439+0000","last_became_peered":"2024-04-20T10:21:02.646439+0000","last_unstale":"2024-04-20T10:21:08.039192+0000","last_undegraded":"2024-04-20T10:21:08.039192+0000","last_fullsized":"2024-04-20T10:21:08.039192+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:06:35.349863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.039464+0000","last_change":"2024-04-20T10:21:02.641401+0000","last_active":"2024-04-20T10:21:08.039464+0000","last_peered":"2024-04-20T10:21:08.039464+0000","last_clean":"2024-04-20T10:21:08.039464+0000","last_became_active":"2024-04-20T10:21:02.640962+0000","last_became_peered":"2024-04-20T10:21:02.640962+0000","last_unstale":"2024-04-20T10:21:08.039464+0000","last_undegraded":"2024-04-20T10:21:08.039464+0000","last_fullsized":"2024-04-20T10:21:08.039464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:22:36.605724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.391828+0000","last_change":"2024-04-20T10:21:02.654762+0000","last_active":"2024-04-20T10:21:08.391828+0000","last_peered":"2024-04-20T10:21:08.391828+0000","last_clean":"2024-04-20T10:21:08.391828+0000","last_became_active":"2024-04-20T10:21:02.654497+0000","last_became_peered":"2024-04-20T10:21:02.654497+0000","last_unstale":"2024-04-20T10:21:08.391828+0000","last_undegraded":"2024-04-20T10:21:08.391828+0000","last_fullsized":"2024-04-20T10:21:08.391828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:46:41.173563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540653+0000","last_change":"2024-04-20T10:21:02.630507+0000","last_active":"2024-04-20T10:21:08.540653+0000","last_peered":"2024-04-20T10:21:08.540653+0000","last_clean":"2024-04-20T10:21:08.540653+0000","last_became_active":"2024-04-20T10:21:02.626482+0000","last_became_peered":"2024-04-20T10:21:02.626482+0000","last_unstale":"2024-04-20T10:21:08.540653+0000","last_undegraded":"2024-04-20T10:21:08.540653+0000","last_fullsized":"2024-04-20T10:21:08.540653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:28:54.408773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.390187+0000","last_change":"2024-04-20T10:21:02.649796+0000","last_active":"2024-04-20T10:21:08.390187+0000","last_peered":"2024-04-20T10:21:08.390187+0000","last_clean":"2024-04-20T10:21:08.390187+0000","last_became_active":"2024-04-20T10:21:02.649541+0000","last_became_peered":"2024-04-20T10:21:02.649541+0000","last_unstale":"2024-04-20T10:21:08.390187+0000","last_undegraded":"2024-04-20T10:21:08.390187+0000","last_fullsized":"2024-04-20T10:21:08.390187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:02:26.716730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540951+0000","last_change":"2024-04-20T10:21:02.653552+0000","last_active":"2024-04-20T10:21:08.540951+0000","last_peered":"2024-04-20T10:21:08.540951+0000","last_clean":"2024-04-20T10:21:08.540951+0000","last_became_active":"2024-04-20T10:21:02.653288+0000","last_became_peered":"2024-04-20T10:21:02.653288+0000","last_unstale":"2024-04-20T10:21:08.540951+0000","last_undegraded":"2024-04-20T10:21:08.540951+0000","last_fullsized":"2024-04-20T10:21:08.540951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:49:26.986908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.390549+0000","last_change":"2024-04-20T10:21:02.633119+0000","last_active":"2024-04-20T10:21:08.390549+0000","last_peered":"2024-04-20T10:21:08.390549+0000","last_clean":"2024-04-20T10:21:08.390549+0000","last_became_active":"2024-04-20T10:21:02.632816+0000","last_became_peered":"2024-04-20T10:21:02.632816+0000","last_unstale":"2024-04-20T10:21:08.390549+0000","last_undegraded":"2024-04-20T10:21:08.390549+0000","last_fullsized":"2024-04-20T10:21:08.390549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:16:04.625809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541721+0000","last_change":"2024-04-20T10:21:02.638226+0000","last_active":"2024-04-20T10:21:08.541721+0000","last_peered":"2024-04-20T10:21:08.541721+0000","last_clean":"2024-04-20T10:21:08.541721+0000","last_became_active":"2024-04-20T10:21:02.637640+0000","last_became_peered":"2024-04-20T10:21:02.637640+0000","last_unstale":"2024-04-20T10:21:08.541721+0000","last_undegraded":"2024-04-20T10:21:08.541721+0000","last_fullsized":"2024-04-20T10:21:08.541721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:44:55.208482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.544254+0000","last_change":"2024-04-20T10:21:02.641981+0000","last_active":"2024-04-20T10:21:08.544254+0000","last_peered":"2024-04-20T10:21:08.544254+0000","last_clean":"2024-04-20T10:21:08.544254+0000","last_became_active":"2024-04-20T10:21:02.640073+0000","last_became_peered":"2024-04-20T10:21:02.640073+0000","last_unstale":"2024-04-20T10:21:08.544254+0000","last_undegraded":"2024-04-20T10:21:08.544254+0000","last_fullsized":"2024-04-20T10:21:08.544254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:28:54.408773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441547+0000","last_change":"2024-04-20T10:21:02.646636+0000","last_active":"2024-04-20T10:21:08.441547+0000","last_peered":"2024-04-20T10:21:08.441547+0000","last_clean":"2024-04-20T10:21:08.441547+0000","last_became_active":"2024-04-20T10:21:02.646359+0000","last_became_peered":"2024-04-20T10:21:02.646359+0000","last_unstale":"2024-04-20T10:21:08.441547+0000","last_undegraded":"2024-04-20T10:21:08.441547+0000","last_fullsized":"2024-04-20T10:21:08.441547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:22:36.605724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.391583+0000","last_change":"2024-04-20T10:21:02.652784+0000","last_active":"2024-04-20T10:21:08.391583+0000","last_peered":"2024-04-20T10:21:08.391583+0000","last_clean":"2024-04-20T10:21:08.391583+0000","last_became_active":"2024-04-20T10:21:02.652454+0000","last_became_peered":"2024-04-20T10:21:02.652454+0000","last_unstale":"2024-04-20T10:21:08.391583+0000","last_undegraded":"2024-04-20T10:21:08.391583+0000","last_fullsized":"2024-04-20T10:21:08.391583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:38:01.376108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542155,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":694509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542156,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":107401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542155,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":103649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542155,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":103685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542155,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":107401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542155,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":103559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542157,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":103719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542157,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":868352,"data_stored":694565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T10:21:37.220 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T10:21:37.457 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.457+0000 7f43c9e00640 1 Processor -- start 2024-04-20T10:21:37.457 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.457+0000 7f43c9e00640 1 -- start start 2024-04-20T10:21:37.459 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.459+0000 7f43c9e00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43c415b7e0 0x7f43c415bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:37.459 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.459+0000 7f43c9e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43c406ccc0 con 0x7f43c415b400 2024-04-20T10:21:37.459 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.459+0000 7f43c9e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43c406cfa0 con 0x7f43c415c190 2024-04-20T10:21:37.459 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.459+0000 7f43c9e00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f43c406d280 con 0x7f43c415b7e0 2024-04-20T10:21:37.459 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.459+0000 7f43c8a00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f43c415b400 0x7f43c4072d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:47606/0 (socket says 172.21.15.121:47606) 2024-04-20T10:21:37.459 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.459+0000 7f43c8a00640 1 -- 172.21.15.121:0/527094022 learned_addr learned my addr 172.21.15.121:0/527094022 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:37.459 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.459+0000 7f43c3e00640 1 --2- 172.21.15.121:0/527094022 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43c415b7e0 0x7f43c415bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:37.459 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.459+0000 7f43c3400640 1 -- 172.21.15.121:0/527094022 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2794488872 0 0) 0x7f43c406ccc0 con 0x7f43c415b400 2024-04-20T10:21:37.459 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.459+0000 7f43c3400640 1 -- 172.21.15.121:0/527094022 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43ac003520 con 0x7f43c415b400 2024-04-20T10:21:37.462 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.460+0000 7f43c3400640 1 -- 172.21.15.121:0/527094022 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3961052538 0 0) 0x7f43ac003520 con 0x7f43c415b400 2024-04-20T10:21:37.462 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.460+0000 7f43c3400640 1 -- 172.21.15.121:0/527094022 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43c415b7e0 msgr2=0x7f43c415bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:37.462 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.460+0000 7f43c3400640 1 --2- 172.21.15.121:0/527094022 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43c415b7e0 0x7f43c415bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:37.462 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.460+0000 7f43c3400640 1 -- 172.21.15.121:0/527094022 >> v1:172.21.15.123:6789/0 conn(0x7f43c415c190 legacy=0x7f43c406ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:37.462 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.460+0000 7f43c3400640 1 -- 172.21.15.121:0/527094022 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c406e440 con 0x7f43c415b400 2024-04-20T10:21:37.462 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.460+0000 7f43c3400640 1 -- 172.21.15.121:0/527094022 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f43b8001670 con 0x7f43c415b400 2024-04-20T10:21:37.463 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.460+0000 7f43c3400640 1 -- 172.21.15.121:0/527094022 >> v1:172.21.15.121:6789/0 conn(0x7f43c415b400 legacy=0x7f43c4072d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:37.463 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.460+0000 7f43c3400640 1 --2- 172.21.15.121:0/527094022 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43ac0039e0 0x7f43ac003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:37.463 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.460+0000 7f43c3400640 1 --2- 172.21.15.121:0/527094022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43ac004800 0x7f43ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:37.463 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.460+0000 7f43c3400640 1 --2- 172.21.15.121:0/527094022 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43c415b7e0 0x7f43ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:37.463 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.460+0000 7f43c3400640 1 -- 172.21.15.121:0/527094022 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f43c406d280 con 0x7f43ac0039e0 2024-04-20T10:21:37.463 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.460+0000 7f43c3400640 1 -- 172.21.15.121:0/527094022 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f43ac00bad0 con 0x7f43ac004800 2024-04-20T10:21:37.463 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.460+0000 7f43c3400640 1 -- 172.21.15.121:0/527094022 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f43ac00dee0 con 0x7f43c415b7e0 2024-04-20T10:21:37.463 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.460+0000 7f43c8a00640 1 --2- 172.21.15.121:0/527094022 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43ac0039e0 0x7f43ac003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:37.463 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.461+0000 7f43c3e00640 1 --2- 172.21.15.121:0/527094022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43ac004800 0x7f43ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:37.463 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.461+0000 7f43c8a00640 1 -- 172.21.15.121:0/527094022 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43c415b7e0 msgr2=0x7f43ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:37.463 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.461+0000 7f43c8a00640 1 --2- 172.21.15.121:0/527094022 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43c415b7e0 0x7f43ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:37.463 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.461+0000 7f43c8a00640 1 -- 172.21.15.121:0/527094022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43ac004800 msgr2=0x7f43ac004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:37.463 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.461+0000 7f43c8a00640 1 --2- 172.21.15.121:0/527094022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43ac004800 0x7f43ac004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:37.464 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.461+0000 7f43c8a00640 1 -- 172.21.15.121:0/527094022 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43ac003520 con 0x7f43ac0039e0 2024-04-20T10:21:37.464 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.461+0000 7f43c3e00640 1 --2- 172.21.15.121:0/527094022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43ac004800 0x7f43ac004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:37.464 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.462+0000 7f43c8a00640 1 --2- 172.21.15.121:0/527094022 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43ac0039e0 0x7f43ac003e00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f43b800c0e0 tx=0x7f43ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccb35c6274227e6f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:37.464 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.462+0000 7f43c3400640 1 -- 172.21.15.121:0/527094022 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b8007c40 con 0x7f43ac0039e0 2024-04-20T10:21:37.464 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.462+0000 7f43c3400640 1 -- 172.21.15.121:0/527094022 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43b8030070 con 0x7f43ac0039e0 2024-04-20T10:21:37.464 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.462+0000 7f43c3400640 1 -- 172.21.15.121:0/527094022 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b8005d30 con 0x7f43ac0039e0 2024-04-20T10:21:37.464 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.462+0000 7f43c9e00640 1 -- 172.21.15.121:0/527094022 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43ac0039e0 msgr2=0x7f43ac003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:37.464 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.462+0000 7f43c9e00640 1 --2- 172.21.15.121:0/527094022 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43ac0039e0 0x7f43ac003e00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f43b800c0e0 tx=0x7f43ac005830 comp rx=0 tx=0).stop 2024-04-20T10:21:37.464 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.462+0000 7f43c9400640 1 -- 172.21.15.121:0/527094022 reap_dead start 2024-04-20T10:21:37.464 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.463+0000 7f43c9e00640 1 -- 172.21.15.121:0/527094022 shutdown_connections 2024-04-20T10:21:37.464 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.463+0000 7f43c9e00640 1 -- 172.21.15.121:0/527094022 >> 172.21.15.121:0/527094022 conn(0x7f43c40d0040 msgr2=0x7f43c40d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:37.464 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.463+0000 7f43c9e00640 1 -- 172.21.15.121:0/527094022 shutdown_connections 2024-04-20T10:21:37.465 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.463+0000 7f43c9e00640 1 -- 172.21.15.121:0/527094022 wait complete. 2024-04-20T10:21:37.466 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.464+0000 7f43c9e00640 1 Processor -- start 2024-04-20T10:21:37.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.464+0000 7f43c9e00640 1 -- start start 2024-04-20T10:21:37.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.465+0000 7f43c9e00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43ac0039e0 0x7f43c40ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:37.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.465+0000 7f43c9e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43c4153680 0x7f43c409d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:37.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.465+0000 7f43c9e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43c409ac60 0x7f43c40ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:37.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.465+0000 7f43c9e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f43c415c730 con 0x7f43c409ac60 2024-04-20T10:21:37.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.465+0000 7f43c9e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f43c415c5b0 con 0x7f43c4153680 2024-04-20T10:21:37.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.465+0000 7f43c9e00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f43c415bc50 con 0x7f43ac0039e0 2024-04-20T10:21:37.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.466+0000 7f43c9400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43c409ac60 0x7f43c40ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:37.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.466+0000 7f43c9400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43c409ac60 0x7f43c40ca2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58592/0 (socket says 172.21.15.121:58592) 2024-04-20T10:21:37.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.466+0000 7f43c9400640 1 -- 172.21.15.121:0/1641392011 learned_addr learned my addr 172.21.15.121:0/1641392011 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:37.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.466+0000 7f43c8a00640 1 --2- 172.21.15.121:0/1641392011 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43ac0039e0 0x7f43c40ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:37.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.466+0000 7f43c3e00640 1 --2- 172.21.15.121:0/1641392011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43c4153680 0x7f43c409d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:37.468 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.466+0000 7f43c9400640 1 -- 172.21.15.121:0/1641392011 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43ac0039e0 msgr2=0x7f43c40ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:37.468 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.466+0000 7f43c9400640 1 --2- 172.21.15.121:0/1641392011 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43ac0039e0 0x7f43c40ccf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:37.468 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.466+0000 7f43c9400640 1 -- 172.21.15.121:0/1641392011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43c4153680 msgr2=0x7f43c409d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:37.468 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.466+0000 7f43c9400640 1 --2- 172.21.15.121:0/1641392011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43c4153680 0x7f43c409d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:37.468 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.466+0000 7f43c9400640 1 -- 172.21.15.121:0/1641392011 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c409b8e0 con 0x7f43c409ac60 2024-04-20T10:21:37.468 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.466+0000 7f43c9400640 1 --2- 172.21.15.121:0/1641392011 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43c409ac60 0x7f43c40ca2c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f43c4072280 tx=0x7f43bc00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:37.468 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.467+0000 7f43c0c00640 1 -- 172.21.15.121:0/1641392011 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc005d40 con 0x7f43c409ac60 2024-04-20T10:21:37.468 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.467+0000 7f43c0c00640 1 -- 172.21.15.121:0/1641392011 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43bc00a610 con 0x7f43c409ac60 2024-04-20T10:21:37.468 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.467+0000 7f43c0c00640 1 -- 172.21.15.121:0/1641392011 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43bc00aa00 con 0x7f43c409ac60 2024-04-20T10:21:37.468 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.467+0000 7f43c9e00640 1 -- 172.21.15.121:0/1641392011 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43c409df50 con 0x7f43c409ac60 2024-04-20T10:21:37.468 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.467+0000 7f43c9e00640 1 -- 172.21.15.121:0/1641392011 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43c40c9ac0 con 0x7f43c409ac60 2024-04-20T10:21:37.469 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.468+0000 7f43c9e00640 1 -- 172.21.15.121:0/1641392011 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43c4075b60 con 0x7f43c409ac60 2024-04-20T10:21:37.469 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.469+0000 7f43c0c00640 1 -- 172.21.15.121:0/1641392011 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f43bc011070 con 0x7f43c409ac60 2024-04-20T10:21:37.469 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.470+0000 7f43c0c00640 1 --2- 172.21.15.121:0/1641392011 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f43a406c8f0 0x7f43a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:37.469 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.470+0000 7f43c0c00640 1 -- 172.21.15.121:0/1641392011 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f43bc08b310 con 0x7f43c409ac60 2024-04-20T10:21:37.469 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.470+0000 7f43c8a00640 1 --2- 172.21.15.121:0/1641392011 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f43a406c8f0 0x7f43a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:37.470 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.471+0000 7f43c8a00640 1 --2- 172.21.15.121:0/1641392011 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f43a406c8f0 0x7f43a406edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f43b8034000 tx=0x7f43b8009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:37.473 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.474+0000 7f43c0c00640 1 -- 172.21.15.121:0/1641392011 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f43bc019060 con 0x7f43c409ac60 2024-04-20T10:21:37.654 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.654+0000 7f43c9e00640 1 -- 172.21.15.121:0/1641392011 --> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f43c4153a60 con 0x7f43a406c8f0 2024-04-20T10:21:37.687 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.687+0000 7f43c0c00640 1 -- 172.21.15.121:0/1641392011 <== mgr.4103 v2:172.21.15.121:6832/1408233272 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332537 (secure 0 0 0) 0x7f43b8030040 con 0x7f43a406c8f0 2024-04-20T10:21:37.690 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:21:37.693 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-04-20T10:21:37.695 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.695+0000 7f43c9e00640 1 -- 172.21.15.121:0/1641392011 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f43a406c8f0 msgr2=0x7f43a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:37.695 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.695+0000 7f43c9e00640 1 --2- 172.21.15.121:0/1641392011 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f43a406c8f0 0x7f43a406edb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f43b8034000 tx=0x7f43b8009030 comp rx=0 tx=0).stop 2024-04-20T10:21:37.695 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.695+0000 7f43c9e00640 1 -- 172.21.15.121:0/1641392011 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43c409ac60 msgr2=0x7f43c40ca2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:37.695 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.695+0000 7f43c9e00640 1 --2- 172.21.15.121:0/1641392011 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43c409ac60 0x7f43c40ca2c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f43c4072280 tx=0x7f43bc00cb90 comp rx=0 tx=0).stop 2024-04-20T10:21:37.695 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.695+0000 7f43c9e00640 1 -- 172.21.15.121:0/1641392011 shutdown_connections 2024-04-20T10:21:37.696 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.695+0000 7f43c9e00640 1 --2- 172.21.15.121:0/1641392011 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f43c409ac60 0x7f43c40ca2c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:37.696 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.695+0000 7f43c9e00640 1 --2- 172.21.15.121:0/1641392011 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f43a406c8f0 0x7f43a406edb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:37.696 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.695+0000 7f43c9e00640 1 --2- 172.21.15.121:0/1641392011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43c4153680 0x7f43c409d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:37.696 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.695+0000 7f43c9e00640 1 --2- 172.21.15.121:0/1641392011 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f43ac0039e0 0x7f43c40ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:37.696 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.695+0000 7f43c9e00640 1 -- 172.21.15.121:0/1641392011 >> 172.21.15.121:0/1641392011 conn(0x7f43c40d0040 msgr2=0x7f43c40b14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:37.696 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.696+0000 7f43c9e00640 1 -- 172.21.15.121:0/1641392011 shutdown_connections 2024-04-20T10:21:37.696 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.696+0000 7f43c9e00640 1 -- 172.21.15.121:0/1641392011 wait complete. 2024-04-20T10:21:37.713 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2024-04-20T10:21:37.121076+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219436,"kb_used_data":3272,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755284,"statfs":{"total":773094113280,"available":772869410816,"internally_reserved":0,"allocated":3350528,"data_stored":2018488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005504"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.544361+0000","last_change":"2024-04-20T10:21:01.620269+0000","last_active":"2024-04-20T10:21:08.544361+0000","last_peered":"2024-04-20T10:21:08.544361+0000","last_clean":"2024-04-20T10:21:08.544361+0000","last_became_active":"2024-04-20T10:21:01.619751+0000","last_became_peered":"2024-04-20T10:21:01.619751+0000","last_unstale":"2024-04-20T10:21:08.544361+0000","last_undegraded":"2024-04-20T10:21:08.544361+0000","last_fullsized":"2024-04-20T10:21:08.544361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:28:01.100842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.544942+0000","last_change":"2024-04-20T10:21:02.642147+0000","last_active":"2024-04-20T10:21:08.544942+0000","last_peered":"2024-04-20T10:21:08.544942+0000","last_clean":"2024-04-20T10:21:08.544942+0000","last_became_active":"2024-04-20T10:21:02.641636+0000","last_became_peered":"2024-04-20T10:21:02.641636+0000","last_unstale":"2024-04-20T10:21:08.544942+0000","last_undegraded":"2024-04-20T10:21:08.544942+0000","last_fullsized":"2024-04-20T10:21:08.544942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:53:57.418208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540582+0000","last_change":"2024-04-20T10:21:06.909318+0000","last_active":"2024-04-20T10:21:08.540582+0000","last_peered":"2024-04-20T10:21:08.540582+0000","last_clean":"2024-04-20T10:21:08.540582+0000","last_became_active":"2024-04-20T10:21:06.907900+0000","last_became_peered":"2024-04-20T10:21:06.907900+0000","last_unstale":"2024-04-20T10:21:08.540582+0000","last_undegraded":"2024-04-20T10:21:08.540582+0000","last_fullsized":"2024-04-20T10:21:08.540582+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:35:58.510637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.545148+0000","last_change":"2024-04-20T10:21:01.609344+0000","last_active":"2024-04-20T10:21:08.545148+0000","last_peered":"2024-04-20T10:21:08.545148+0000","last_clean":"2024-04-20T10:21:08.545148+0000","last_became_active":"2024-04-20T10:21:01.608863+0000","last_became_peered":"2024-04-20T10:21:01.608863+0000","last_unstale":"2024-04-20T10:21:08.545148+0000","last_undegraded":"2024-04-20T10:21:08.545148+0000","last_fullsized":"2024-04-20T10:21:08.545148+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:32:02.981747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.543880+0000","last_change":"2024-04-20T10:21:02.649607+0000","last_active":"2024-04-20T10:21:08.543880+0000","last_peered":"2024-04-20T10:21:08.543880+0000","last_clean":"2024-04-20T10:21:08.543880+0000","last_became_active":"2024-04-20T10:21:02.649283+0000","last_became_peered":"2024-04-20T10:21:02.649283+0000","last_unstale":"2024-04-20T10:21:08.543880+0000","last_undegraded":"2024-04-20T10:21:08.543880+0000","last_fullsized":"2024-04-20T10:21:08.543880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:06:35.349863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.885288+0000","last_change":"2024-04-20T10:21:06.910300+0000","last_active":"2024-04-20T10:21:07.885288+0000","last_peered":"2024-04-20T10:21:07.885288+0000","last_clean":"2024-04-20T10:21:07.885288+0000","last_became_active":"2024-04-20T10:21:06.909341+0000","last_became_peered":"2024-04-20T10:21:06.909341+0000","last_unstale":"2024-04-20T10:21:07.885288+0000","last_undegraded":"2024-04-20T10:21:07.885288+0000","last_fullsized":"2024-04-20T10:21:07.885288+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:06:39.624891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.391510+0000","last_change":"2024-04-20T10:21:01.623714+0000","last_active":"2024-04-20T10:21:08.391510+0000","last_peered":"2024-04-20T10:21:08.391510+0000","last_clean":"2024-04-20T10:21:08.391510+0000","last_became_active":"2024-04-20T10:21:01.623427+0000","last_became_peered":"2024-04-20T10:21:01.623427+0000","last_unstale":"2024-04-20T10:21:08.391510+0000","last_undegraded":"2024-04-20T10:21:08.391510+0000","last_fullsized":"2024-04-20T10:21:08.391510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:25:59.939855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541788+0000","last_change":"2024-04-20T10:21:02.636522+0000","last_active":"2024-04-20T10:21:08.541788+0000","last_peered":"2024-04-20T10:21:08.541788+0000","last_clean":"2024-04-20T10:21:08.541788+0000","last_became_active":"2024-04-20T10:21:02.633235+0000","last_became_peered":"2024-04-20T10:21:02.633235+0000","last_unstale":"2024-04-20T10:21:08.541788+0000","last_undegraded":"2024-04-20T10:21:08.541788+0000","last_fullsized":"2024-04-20T10:21:08.541788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:55:00.905038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.886695+0000","last_change":"2024-04-20T10:21:06.911285+0000","last_active":"2024-04-20T10:21:07.886695+0000","last_peered":"2024-04-20T10:21:07.886695+0000","last_clean":"2024-04-20T10:21:07.886695+0000","last_became_active":"2024-04-20T10:21:06.908548+0000","last_became_peered":"2024-04-20T10:21:06.908548+0000","last_unstale":"2024-04-20T10:21:07.886695+0000","last_undegraded":"2024-04-20T10:21:07.886695+0000","last_fullsized":"2024-04-20T10:21:07.886695+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:09:46.241216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.391163+0000","last_change":"2024-04-20T10:21:01.626685+0000","last_active":"2024-04-20T10:21:08.391163+0000","last_peered":"2024-04-20T10:21:08.391163+0000","last_clean":"2024-04-20T10:21:08.391163+0000","last_became_active":"2024-04-20T10:21:01.626354+0000","last_became_peered":"2024-04-20T10:21:01.626354+0000","last_unstale":"2024-04-20T10:21:08.391163+0000","last_undegraded":"2024-04-20T10:21:08.391163+0000","last_fullsized":"2024-04-20T10:21:08.391163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:07:50.284372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542097+0000","last_change":"2024-04-20T10:21:02.636800+0000","last_active":"2024-04-20T10:21:08.542097+0000","last_peered":"2024-04-20T10:21:08.542097+0000","last_clean":"2024-04-20T10:21:08.542097+0000","last_became_active":"2024-04-20T10:21:02.636523+0000","last_became_peered":"2024-04-20T10:21:02.636523+0000","last_unstale":"2024-04-20T10:21:08.542097+0000","last_undegraded":"2024-04-20T10:21:08.542097+0000","last_fullsized":"2024-04-20T10:21:08.542097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:31:07.363373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.885879+0000","last_change":"2024-04-20T10:21:06.893126+0000","last_active":"2024-04-20T10:21:07.885879+0000","last_peered":"2024-04-20T10:21:07.885879+0000","last_clean":"2024-04-20T10:21:07.885879+0000","last_became_active":"2024-04-20T10:21:06.892722+0000","last_became_peered":"2024-04-20T10:21:06.892722+0000","last_unstale":"2024-04-20T10:21:07.885879+0000","last_undegraded":"2024-04-20T10:21:07.885879+0000","last_fullsized":"2024-04-20T10:21:07.885879+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:10:19.481713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.038387+0000","last_change":"2024-04-20T10:21:02.633548+0000","last_active":"2024-04-20T10:21:08.038387+0000","last_peered":"2024-04-20T10:21:08.038387+0000","last_clean":"2024-04-20T10:21:08.038387+0000","last_became_active":"2024-04-20T10:21:02.631142+0000","last_became_peered":"2024-04-20T10:21:02.631142+0000","last_unstale":"2024-04-20T10:21:08.038387+0000","last_undegraded":"2024-04-20T10:21:08.038387+0000","last_fullsized":"2024-04-20T10:21:08.038387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:14:16.934710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.390972+0000","last_change":"2024-04-20T10:21:01.622399+0000","last_active":"2024-04-20T10:21:08.390972+0000","last_peered":"2024-04-20T10:21:08.390972+0000","last_clean":"2024-04-20T10:21:08.390972+0000","last_became_active":"2024-04-20T10:21:01.622072+0000","last_became_peered":"2024-04-20T10:21:01.622072+0000","last_unstale":"2024-04-20T10:21:08.390972+0000","last_undegraded":"2024-04-20T10:21:08.390972+0000","last_fullsized":"2024-04-20T10:21:08.390972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:35:02.394948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.887355+0000","last_change":"2024-04-20T10:21:06.897024+0000","last_active":"2024-04-20T10:21:07.887355+0000","last_peered":"2024-04-20T10:21:07.887355+0000","last_clean":"2024-04-20T10:21:07.887355+0000","last_became_active":"2024-04-20T10:21:06.896315+0000","last_became_peered":"2024-04-20T10:21:06.896315+0000","last_unstale":"2024-04-20T10:21:07.887355+0000","last_undegraded":"2024-04-20T10:21:07.887355+0000","last_fullsized":"2024-04-20T10:21:07.887355+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:21:29.812831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.544646+0000","last_change":"2024-04-20T10:21:02.646217+0000","last_active":"2024-04-20T10:21:08.544646+0000","last_peered":"2024-04-20T10:21:08.544646+0000","last_clean":"2024-04-20T10:21:08.544646+0000","last_became_active":"2024-04-20T10:21:02.645926+0000","last_became_peered":"2024-04-20T10:21:02.645926+0000","last_unstale":"2024-04-20T10:21:08.544646+0000","last_undegraded":"2024-04-20T10:21:08.544646+0000","last_fullsized":"2024-04-20T10:21:08.544646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:10:15.206686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441134+0000","last_change":"2024-04-20T10:21:01.617948+0000","last_active":"2024-04-20T10:21:08.441134+0000","last_peered":"2024-04-20T10:21:08.441134+0000","last_clean":"2024-04-20T10:21:08.441134+0000","last_became_active":"2024-04-20T10:21:01.617628+0000","last_became_peered":"2024-04-20T10:21:01.617628+0000","last_unstale":"2024-04-20T10:21:08.441134+0000","last_undegraded":"2024-04-20T10:21:08.441134+0000","last_fullsized":"2024-04-20T10:21:08.441134+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:55:41.208744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.886801+0000","last_change":"2024-04-20T10:21:06.900164+0000","last_active":"2024-04-20T10:21:07.886801+0000","last_peered":"2024-04-20T10:21:07.886801+0000","last_clean":"2024-04-20T10:21:07.886801+0000","last_became_active":"2024-04-20T10:21:06.899783+0000","last_became_peered":"2024-04-20T10:21:06.899783+0000","last_unstale":"2024-04-20T10:21:07.886801+0000","last_undegraded":"2024-04-20T10:21:07.886801+0000","last_fullsized":"2024-04-20T10:21:07.886801+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:54:01.693236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541211+0000","last_change":"2024-04-20T10:21:02.629982+0000","last_active":"2024-04-20T10:21:08.541211+0000","last_peered":"2024-04-20T10:21:08.541211+0000","last_clean":"2024-04-20T10:21:08.541211+0000","last_became_active":"2024-04-20T10:21:02.629729+0000","last_became_peered":"2024-04-20T10:21:02.629729+0000","last_unstale":"2024-04-20T10:21:08.541211+0000","last_undegraded":"2024-04-20T10:21:08.541211+0000","last_fullsized":"2024-04-20T10:21:08.541211+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:31:53.442042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.440779+0000","last_change":"2024-04-20T10:21:01.604313+0000","last_active":"2024-04-20T10:21:08.440779+0000","last_peered":"2024-04-20T10:21:08.440779+0000","last_clean":"2024-04-20T10:21:08.440779+0000","last_became_active":"2024-04-20T10:21:01.603629+0000","last_became_peered":"2024-04-20T10:21:01.603629+0000","last_unstale":"2024-04-20T10:21:08.440779+0000","last_undegraded":"2024-04-20T10:21:08.440779+0000","last_fullsized":"2024-04-20T10:21:08.440779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:01:19.566351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441622+0000","last_change":"2024-04-20T10:21:06.903477+0000","last_active":"2024-04-20T10:21:08.441622+0000","last_peered":"2024-04-20T10:21:08.441622+0000","last_clean":"2024-04-20T10:21:08.441622+0000","last_became_active":"2024-04-20T10:21:06.903010+0000","last_became_peered":"2024-04-20T10:21:06.903010+0000","last_unstale":"2024-04-20T10:21:08.441622+0000","last_undegraded":"2024-04-20T10:21:08.441622+0000","last_fullsized":"2024-04-20T10:21:08.441622+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:46:45.448591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.885935+0000","last_change":"2024-04-20T10:21:02.614476+0000","last_active":"2024-04-20T10:21:07.885935+0000","last_peered":"2024-04-20T10:21:07.885935+0000","last_clean":"2024-04-20T10:21:07.885935+0000","last_became_active":"2024-04-20T10:21:02.614140+0000","last_became_peered":"2024-04-20T10:21:02.614140+0000","last_unstale":"2024-04-20T10:21:07.885935+0000","last_undegraded":"2024-04-20T10:21:07.885935+0000","last_fullsized":"2024-04-20T10:21:07.885935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:01:20.579875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.389864+0000","last_change":"2024-04-20T10:21:01.619958+0000","last_active":"2024-04-20T10:21:08.389864+0000","last_peered":"2024-04-20T10:21:08.389864+0000","last_clean":"2024-04-20T10:21:08.389864+0000","last_became_active":"2024-04-20T10:21:01.619688+0000","last_became_peered":"2024-04-20T10:21:01.619688+0000","last_unstale":"2024-04-20T10:21:08.389864+0000","last_undegraded":"2024-04-20T10:21:08.389864+0000","last_fullsized":"2024-04-20T10:21:08.389864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:45:51.165353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.887140+0000","last_change":"2024-04-20T10:21:06.908789+0000","last_active":"2024-04-20T10:21:07.887140+0000","last_peered":"2024-04-20T10:21:07.887140+0000","last_clean":"2024-04-20T10:21:07.887140+0000","last_became_active":"2024-04-20T10:21:06.908405+0000","last_became_peered":"2024-04-20T10:21:06.908405+0000","last_unstale":"2024-04-20T10:21:07.887140+0000","last_undegraded":"2024-04-20T10:21:07.887140+0000","last_fullsized":"2024-04-20T10:21:07.887140+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:02:30.991758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.544406+0000","last_change":"2024-04-20T10:21:02.649177+0000","last_active":"2024-04-20T10:21:08.544406+0000","last_peered":"2024-04-20T10:21:08.544406+0000","last_clean":"2024-04-20T10:21:08.544406+0000","last_became_active":"2024-04-20T10:21:02.648861+0000","last_became_peered":"2024-04-20T10:21:02.648861+0000","last_unstale":"2024-04-20T10:21:08.544406+0000","last_undegraded":"2024-04-20T10:21:08.544406+0000","last_fullsized":"2024-04-20T10:21:08.544406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:31:53.442042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.039602+0000","last_change":"2024-04-20T10:21:01.621228+0000","last_active":"2024-04-20T10:21:08.039602+0000","last_peered":"2024-04-20T10:21:08.039602+0000","last_clean":"2024-04-20T10:21:08.039602+0000","last_became_active":"2024-04-20T10:21:01.620525+0000","last_became_peered":"2024-04-20T10:21:01.620525+0000","last_unstale":"2024-04-20T10:21:08.039602+0000","last_undegraded":"2024-04-20T10:21:08.039602+0000","last_fullsized":"2024-04-20T10:21:08.039602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:50:27.716971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.885713+0000","last_change":"2024-04-20T10:21:06.903693+0000","last_active":"2024-04-20T10:21:07.885713+0000","last_peered":"2024-04-20T10:21:07.885713+0000","last_clean":"2024-04-20T10:21:07.885713+0000","last_became_active":"2024-04-20T10:21:06.903087+0000","last_became_peered":"2024-04-20T10:21:06.903087+0000","last_unstale":"2024-04-20T10:21:07.885713+0000","last_undegraded":"2024-04-20T10:21:07.885713+0000","last_fullsized":"2024-04-20T10:21:07.885713+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:22:40.880752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542691+0000","last_change":"2024-04-20T10:21:02.632360+0000","last_active":"2024-04-20T10:21:08.542691+0000","last_peered":"2024-04-20T10:21:08.542691+0000","last_clean":"2024-04-20T10:21:08.542691+0000","last_became_active":"2024-04-20T10:21:02.632062+0000","last_became_peered":"2024-04-20T10:21:02.632062+0000","last_unstale":"2024-04-20T10:21:08.542691+0000","last_undegraded":"2024-04-20T10:21:08.542691+0000","last_fullsized":"2024-04-20T10:21:08.542691+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:06:11.427561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441791+0000","last_change":"2024-04-20T10:21:01.622049+0000","last_active":"2024-04-20T10:21:08.441791+0000","last_peered":"2024-04-20T10:21:08.441791+0000","last_clean":"2024-04-20T10:21:08.441791+0000","last_became_active":"2024-04-20T10:21:01.621779+0000","last_became_peered":"2024-04-20T10:21:01.621779+0000","last_unstale":"2024-04-20T10:21:08.441791+0000","last_undegraded":"2024-04-20T10:21:08.441791+0000","last_fullsized":"2024-04-20T10:21:08.441791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:17:23.552169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540642+0000","last_change":"2024-04-20T10:21:06.908686+0000","last_active":"2024-04-20T10:21:08.540642+0000","last_peered":"2024-04-20T10:21:08.540642+0000","last_clean":"2024-04-20T10:21:08.540642+0000","last_became_active":"2024-04-20T10:21:06.907525+0000","last_became_peered":"2024-04-20T10:21:06.907525+0000","last_unstale":"2024-04-20T10:21:08.540642+0000","last_undegraded":"2024-04-20T10:21:08.540642+0000","last_fullsized":"2024-04-20T10:21:08.540642+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:38:05.651135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441452+0000","last_change":"2024-04-20T10:21:02.633193+0000","last_active":"2024-04-20T10:21:08.441452+0000","last_peered":"2024-04-20T10:21:08.441452+0000","last_clean":"2024-04-20T10:21:08.441452+0000","last_became_active":"2024-04-20T10:21:02.632896+0000","last_became_peered":"2024-04-20T10:21:02.632896+0000","last_unstale":"2024-04-20T10:21:08.441452+0000","last_undegraded":"2024-04-20T10:21:08.441452+0000","last_fullsized":"2024-04-20T10:21:08.441452+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:06:35.349863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.391636+0000","last_change":"2024-04-20T10:21:01.624712+0000","last_active":"2024-04-20T10:21:08.391636+0000","last_peered":"2024-04-20T10:21:08.391636+0000","last_clean":"2024-04-20T10:21:08.391636+0000","last_became_active":"2024-04-20T10:21:01.624381+0000","last_became_peered":"2024-04-20T10:21:01.624381+0000","last_unstale":"2024-04-20T10:21:08.391636+0000","last_undegraded":"2024-04-20T10:21:08.391636+0000","last_fullsized":"2024-04-20T10:21:08.391636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:17:23.552169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.543354+0000","last_change":"2024-04-20T10:21:06.903399+0000","last_active":"2024-04-20T10:21:08.543354+0000","last_peered":"2024-04-20T10:21:08.543354+0000","last_clean":"2024-04-20T10:21:08.543354+0000","last_became_active":"2024-04-20T10:21:06.902529+0000","last_became_peered":"2024-04-20T10:21:06.902529+0000","last_unstale":"2024-04-20T10:21:08.543354+0000","last_undegraded":"2024-04-20T10:21:08.543354+0000","last_fullsized":"2024-04-20T10:21:08.543354+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:52:28.298927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542324+0000","last_change":"2024-04-20T10:21:02.637139+0000","last_active":"2024-04-20T10:21:08.542324+0000","last_peered":"2024-04-20T10:21:08.542324+0000","last_clean":"2024-04-20T10:21:08.542324+0000","last_became_active":"2024-04-20T10:21:02.636849+0000","last_became_peered":"2024-04-20T10:21:02.636849+0000","last_unstale":"2024-04-20T10:21:08.542324+0000","last_undegraded":"2024-04-20T10:21:08.542324+0000","last_fullsized":"2024-04-20T10:21:08.542324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:44:52.759993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441617+0000","last_change":"2024-04-20T10:21:01.606832+0000","last_active":"2024-04-20T10:21:08.441617+0000","last_peered":"2024-04-20T10:21:08.441617+0000","last_clean":"2024-04-20T10:21:08.441617+0000","last_became_active":"2024-04-20T10:21:01.606562+0000","last_became_peered":"2024-04-20T10:21:01.606562+0000","last_unstale":"2024-04-20T10:21:08.441617+0000","last_undegraded":"2024-04-20T10:21:08.441617+0000","last_fullsized":"2024-04-20T10:21:08.441617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:50:27.716971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.440686+0000","last_change":"2024-04-20T10:21:06.894216+0000","last_active":"2024-04-20T10:21:08.440686+0000","last_peered":"2024-04-20T10:21:08.440686+0000","last_clean":"2024-04-20T10:21:08.440686+0000","last_became_active":"2024-04-20T10:21:06.893762+0000","last_became_peered":"2024-04-20T10:21:06.893762+0000","last_unstale":"2024-04-20T10:21:08.440686+0000","last_undegraded":"2024-04-20T10:21:08.440686+0000","last_fullsized":"2024-04-20T10:21:08.440686+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:31:30.034892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.887359+0000","last_change":"2024-04-20T10:21:02.648574+0000","last_active":"2024-04-20T10:21:07.887359+0000","last_peered":"2024-04-20T10:21:07.887359+0000","last_clean":"2024-04-20T10:21:07.887359+0000","last_became_active":"2024-04-20T10:21:02.648275+0000","last_became_peered":"2024-04-20T10:21:02.648275+0000","last_unstale":"2024-04-20T10:21:07.887359+0000","last_undegraded":"2024-04-20T10:21:07.887359+0000","last_fullsized":"2024-04-20T10:21:07.887359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:55:42.222268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541444+0000","last_change":"2024-04-20T10:21:01.615460+0000","last_active":"2024-04-20T10:21:08.541444+0000","last_peered":"2024-04-20T10:21:08.541444+0000","last_clean":"2024-04-20T10:21:08.541444+0000","last_became_active":"2024-04-20T10:21:01.614405+0000","last_became_peered":"2024-04-20T10:21:01.614405+0000","last_unstale":"2024-04-20T10:21:08.541444+0000","last_undegraded":"2024-04-20T10:21:08.541444+0000","last_fullsized":"2024-04-20T10:21:08.541444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:51:46.790176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441597+0000","last_change":"2024-04-20T10:21:06.899961+0000","last_active":"2024-04-20T10:21:08.441597+0000","last_peered":"2024-04-20T10:21:08.441597+0000","last_clean":"2024-04-20T10:21:08.441597+0000","last_became_active":"2024-04-20T10:21:06.899574+0000","last_became_peered":"2024-04-20T10:21:06.899574+0000","last_unstale":"2024-04-20T10:21:08.441597+0000","last_undegraded":"2024-04-20T10:21:08.441597+0000","last_fullsized":"2024-04-20T10:21:08.441597+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:59:28.847178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.440949+0000","last_change":"2024-04-20T10:21:02.627320+0000","last_active":"2024-04-20T10:21:08.440949+0000","last_peered":"2024-04-20T10:21:08.440949+0000","last_clean":"2024-04-20T10:21:08.440949+0000","last_became_active":"2024-04-20T10:21:02.627008+0000","last_became_peered":"2024-04-20T10:21:02.627008+0000","last_unstale":"2024-04-20T10:21:08.440949+0000","last_undegraded":"2024-04-20T10:21:08.440949+0000","last_fullsized":"2024-04-20T10:21:08.440949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:01:55.363202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.881645+0000","last_change":"2024-04-20T10:21:01.611725+0000","last_active":"2024-04-20T10:21:07.881645+0000","last_peered":"2024-04-20T10:21:07.881645+0000","last_clean":"2024-04-20T10:21:07.881645+0000","last_became_active":"2024-04-20T10:21:01.609982+0000","last_became_peered":"2024-04-20T10:21:01.609982+0000","last_unstale":"2024-04-20T10:21:07.881645+0000","last_undegraded":"2024-04-20T10:21:07.881645+0000","last_fullsized":"2024-04-20T10:21:07.881645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:32:02.981747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441859+0000","last_change":"2024-04-20T10:21:06.897904+0000","last_active":"2024-04-20T10:21:08.441859+0000","last_peered":"2024-04-20T10:21:08.441859+0000","last_clean":"2024-04-20T10:21:08.441859+0000","last_became_active":"2024-04-20T10:21:06.896628+0000","last_became_peered":"2024-04-20T10:21:06.896628+0000","last_unstale":"2024-04-20T10:21:08.441859+0000","last_undegraded":"2024-04-20T10:21:08.441859+0000","last_fullsized":"2024-04-20T10:21:08.441859+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:35:58.510637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542234+0000","last_change":"2024-04-20T10:21:02.651679+0000","last_active":"2024-04-20T10:21:08.542234+0000","last_peered":"2024-04-20T10:21:08.542234+0000","last_clean":"2024-04-20T10:21:08.542234+0000","last_became_active":"2024-04-20T10:21:02.651336+0000","last_became_peered":"2024-04-20T10:21:02.651336+0000","last_unstale":"2024-04-20T10:21:08.542234+0000","last_undegraded":"2024-04-20T10:21:08.542234+0000","last_fullsized":"2024-04-20T10:21:08.542234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:17:24.565693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.389935+0000","last_change":"2024-04-20T10:21:01.620091+0000","last_active":"2024-04-20T10:21:08.389935+0000","last_peered":"2024-04-20T10:21:08.389935+0000","last_clean":"2024-04-20T10:21:08.389935+0000","last_became_active":"2024-04-20T10:21:01.619747+0000","last_became_peered":"2024-04-20T10:21:01.619747+0000","last_unstale":"2024-04-20T10:21:08.389935+0000","last_undegraded":"2024-04-20T10:21:08.389935+0000","last_fullsized":"2024-04-20T10:21:08.389935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:38:10.686751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.544184+0000","last_change":"2024-04-20T10:21:06.902502+0000","last_active":"2024-04-20T10:21:08.544184+0000","last_peered":"2024-04-20T10:21:08.544184+0000","last_clean":"2024-04-20T10:21:08.544184+0000","last_became_active":"2024-04-20T10:21:06.899513+0000","last_became_peered":"2024-04-20T10:21:06.899513+0000","last_unstale":"2024-04-20T10:21:08.544184+0000","last_undegraded":"2024-04-20T10:21:08.544184+0000","last_fullsized":"2024-04-20T10:21:08.544184+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:46:45.448591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541620+0000","last_change":"2024-04-20T10:21:02.628521+0000","last_active":"2024-04-20T10:21:08.541620+0000","last_peered":"2024-04-20T10:21:08.541620+0000","last_clean":"2024-04-20T10:21:08.541620+0000","last_became_active":"2024-04-20T10:21:02.626414+0000","last_became_peered":"2024-04-20T10:21:02.626414+0000","last_unstale":"2024-04-20T10:21:08.541620+0000","last_undegraded":"2024-04-20T10:21:08.541620+0000","last_fullsized":"2024-04-20T10:21:08.541620+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:01:55.363202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541447+0000","last_change":"2024-04-20T10:21:01.610399+0000","last_active":"2024-04-20T10:21:08.541447+0000","last_peered":"2024-04-20T10:21:08.541447+0000","last_clean":"2024-04-20T10:21:08.541447+0000","last_became_active":"2024-04-20T10:21:01.610116+0000","last_became_peered":"2024-04-20T10:21:01.610116+0000","last_unstale":"2024-04-20T10:21:08.541447+0000","last_undegraded":"2024-04-20T10:21:08.541447+0000","last_fullsized":"2024-04-20T10:21:08.541447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:01:18.578396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.390168+0000","last_change":"2024-04-20T10:20:52.916129+0000","last_active":"2024-04-20T10:21:08.390168+0000","last_peered":"2024-04-20T10:21:08.390168+0000","last_clean":"2024-04-20T10:21:08.390168+0000","last_became_active":"2024-04-20T10:20:52.913922+0000","last_became_peered":"2024-04-20T10:20:52.913922+0000","last_unstale":"2024-04-20T10:21:08.390168+0000","last_undegraded":"2024-04-20T10:21:08.390168+0000","last_fullsized":"2024-04-20T10:21:08.390168+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:20:51.267588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:20:51.267588+0000","last_clean_scrub_stamp":"2024-04-20T10:20:51.267588+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:30:44.465836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.039062+0000","last_change":"2024-04-20T10:21:06.913314+0000","last_active":"2024-04-20T10:21:08.039062+0000","last_peered":"2024-04-20T10:21:08.039062+0000","last_clean":"2024-04-20T10:21:08.039062+0000","last_became_active":"2024-04-20T10:21:06.912896+0000","last_became_peered":"2024-04-20T10:21:06.912896+0000","last_unstale":"2024-04-20T10:21:08.039062+0000","last_undegraded":"2024-04-20T10:21:08.039062+0000","last_fullsized":"2024-04-20T10:21:08.039062+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:16:08.900837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.391138+0000","last_change":"2024-04-20T10:21:02.643414+0000","last_active":"2024-04-20T10:21:08.391138+0000","last_peered":"2024-04-20T10:21:08.391138+0000","last_clean":"2024-04-20T10:21:08.391138+0000","last_became_active":"2024-04-20T10:21:02.641460+0000","last_became_peered":"2024-04-20T10:21:02.641460+0000","last_unstale":"2024-04-20T10:21:08.391138+0000","last_undegraded":"2024-04-20T10:21:08.391138+0000","last_fullsized":"2024-04-20T10:21:08.391138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:24:51.806454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.882750+0000","last_change":"2024-04-20T10:21:01.612256+0000","last_active":"2024-04-20T10:21:07.882750+0000","last_peered":"2024-04-20T10:21:07.882750+0000","last_clean":"2024-04-20T10:21:07.882750+0000","last_became_active":"2024-04-20T10:21:01.611856+0000","last_became_peered":"2024-04-20T10:21:01.611856+0000","last_unstale":"2024-04-20T10:21:07.882750+0000","last_undegraded":"2024-04-20T10:21:07.882750+0000","last_fullsized":"2024-04-20T10:21:07.882750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:59:37.356548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.038723+0000","last_change":"2024-04-20T10:21:06.913797+0000","last_active":"2024-04-20T10:21:08.038723+0000","last_peered":"2024-04-20T10:21:08.038723+0000","last_clean":"2024-04-20T10:21:08.038723+0000","last_became_active":"2024-04-20T10:21:06.912928+0000","last_became_peered":"2024-04-20T10:21:06.912928+0000","last_unstale":"2024-04-20T10:21:08.038723+0000","last_undegraded":"2024-04-20T10:21:08.038723+0000","last_fullsized":"2024-04-20T10:21:08.038723+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:21:29.812831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542069+0000","last_change":"2024-04-20T10:21:02.630629+0000","last_active":"2024-04-20T10:21:08.542069+0000","last_peered":"2024-04-20T10:21:08.542069+0000","last_clean":"2024-04-20T10:21:08.542069+0000","last_became_active":"2024-04-20T10:21:02.623112+0000","last_became_peered":"2024-04-20T10:21:02.623112+0000","last_unstale":"2024-04-20T10:21:08.542069+0000","last_undegraded":"2024-04-20T10:21:08.542069+0000","last_fullsized":"2024-04-20T10:21:08.542069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:37:12.899052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441939+0000","last_change":"2024-04-20T10:21:01.598214+0000","last_active":"2024-04-20T10:21:08.441939+0000","last_peered":"2024-04-20T10:21:08.441939+0000","last_clean":"2024-04-20T10:21:08.441939+0000","last_became_active":"2024-04-20T10:21:01.597260+0000","last_became_peered":"2024-04-20T10:21:01.597260+0000","last_unstale":"2024-04-20T10:21:08.441939+0000","last_undegraded":"2024-04-20T10:21:08.441939+0000","last_fullsized":"2024-04-20T10:21:08.441939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:25:59.939855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.543790+0000","last_change":"2024-04-20T10:21:06.894202+0000","last_active":"2024-04-20T10:21:08.543790+0000","last_peered":"2024-04-20T10:21:08.543790+0000","last_clean":"2024-04-20T10:21:08.543790+0000","last_became_active":"2024-04-20T10:21:06.893620+0000","last_became_peered":"2024-04-20T10:21:06.893620+0000","last_unstale":"2024-04-20T10:21:08.543790+0000","last_undegraded":"2024-04-20T10:21:08.543790+0000","last_fullsized":"2024-04-20T10:21:08.543790+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:31:30.034892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.391863+0000","last_change":"2024-04-20T10:21:02.655710+0000","last_active":"2024-04-20T10:21:08.391863+0000","last_peered":"2024-04-20T10:21:08.391863+0000","last_clean":"2024-04-20T10:21:08.391863+0000","last_became_active":"2024-04-20T10:21:02.655531+0000","last_became_peered":"2024-04-20T10:21:02.655531+0000","last_unstale":"2024-04-20T10:21:08.391863+0000","last_undegraded":"2024-04-20T10:21:08.391863+0000","last_fullsized":"2024-04-20T10:21:08.391863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:27:00.187303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.883115+0000","last_change":"2024-04-20T10:21:01.612100+0000","last_active":"2024-04-20T10:21:07.883115+0000","last_peered":"2024-04-20T10:21:07.883115+0000","last_clean":"2024-04-20T10:21:07.883115+0000","last_became_active":"2024-04-20T10:21:01.610465+0000","last_became_peered":"2024-04-20T10:21:01.610465+0000","last_unstale":"2024-04-20T10:21:07.883115+0000","last_undegraded":"2024-04-20T10:21:07.883115+0000","last_fullsized":"2024-04-20T10:21:07.883115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:01:19.566351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540603+0000","last_change":"2024-04-20T10:21:06.908159+0000","last_active":"2024-04-20T10:21:08.540603+0000","last_peered":"2024-04-20T10:21:08.540603+0000","last_clean":"2024-04-20T10:21:08.540603+0000","last_became_active":"2024-04-20T10:21:06.907687+0000","last_became_peered":"2024-04-20T10:21:06.907687+0000","last_unstale":"2024-04-20T10:21:08.540603+0000","last_undegraded":"2024-04-20T10:21:08.540603+0000","last_fullsized":"2024-04-20T10:21:08.540603+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:17:43.493429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.391503+0000","last_change":"2024-04-20T10:21:02.652617+0000","last_active":"2024-04-20T10:21:08.391503+0000","last_peered":"2024-04-20T10:21:08.391503+0000","last_clean":"2024-04-20T10:21:08.391503+0000","last_became_active":"2024-04-20T10:21:02.652321+0000","last_became_peered":"2024-04-20T10:21:02.652321+0000","last_unstale":"2024-04-20T10:21:08.391503+0000","last_undegraded":"2024-04-20T10:21:08.391503+0000","last_fullsized":"2024-04-20T10:21:08.391503+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:59:24.572150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.887281+0000","last_change":"2024-04-20T10:21:01.608355+0000","last_active":"2024-04-20T10:21:07.887281+0000","last_peered":"2024-04-20T10:21:07.887281+0000","last_clean":"2024-04-20T10:21:07.887281+0000","last_became_active":"2024-04-20T10:21:01.607981+0000","last_became_peered":"2024-04-20T10:21:01.607981+0000","last_unstale":"2024-04-20T10:21:07.887281+0000","last_undegraded":"2024-04-20T10:21:07.887281+0000","last_fullsized":"2024-04-20T10:21:07.887281+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:10:57.859781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.885574+0000","last_change":"2024-04-20T10:21:06.900202+0000","last_active":"2024-04-20T10:21:07.885574+0000","last_peered":"2024-04-20T10:21:07.885574+0000","last_clean":"2024-04-20T10:21:07.885574+0000","last_became_active":"2024-04-20T10:21:06.899795+0000","last_became_peered":"2024-04-20T10:21:06.899795+0000","last_unstale":"2024-04-20T10:21:07.885574+0000","last_undegraded":"2024-04-20T10:21:07.885574+0000","last_fullsized":"2024-04-20T10:21:07.885574+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:01:59.638230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.390393+0000","last_change":"2024-04-20T10:21:02.646213+0000","last_active":"2024-04-20T10:21:08.390393+0000","last_peered":"2024-04-20T10:21:08.390393+0000","last_clean":"2024-04-20T10:21:08.390393+0000","last_became_active":"2024-04-20T10:21:02.645881+0000","last_became_peered":"2024-04-20T10:21:02.645881+0000","last_unstale":"2024-04-20T10:21:08.390393+0000","last_undegraded":"2024-04-20T10:21:08.390393+0000","last_fullsized":"2024-04-20T10:21:08.390393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:21:25.537803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.881093+0000","last_change":"2024-04-20T10:21:01.601030+0000","last_active":"2024-04-20T10:21:07.881093+0000","last_peered":"2024-04-20T10:21:07.881093+0000","last_clean":"2024-04-20T10:21:07.881093+0000","last_became_active":"2024-04-20T10:21:01.600702+0000","last_became_peered":"2024-04-20T10:21:01.600702+0000","last_unstale":"2024-04-20T10:21:07.881093+0000","last_undegraded":"2024-04-20T10:21:07.881093+0000","last_fullsized":"2024-04-20T10:21:07.881093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:36:15.966290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.543987+0000","last_change":"2024-04-20T10:21:06.904141+0000","last_active":"2024-04-20T10:21:08.543987+0000","last_peered":"2024-04-20T10:21:08.543987+0000","last_clean":"2024-04-20T10:21:08.543987+0000","last_became_active":"2024-04-20T10:21:06.903212+0000","last_became_peered":"2024-04-20T10:21:06.903212+0000","last_unstale":"2024-04-20T10:21:08.543987+0000","last_undegraded":"2024-04-20T10:21:08.543987+0000","last_fullsized":"2024-04-20T10:21:08.543987+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:59:28.847178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.038508+0000","last_change":"2024-04-20T10:21:02.644636+0000","last_active":"2024-04-20T10:21:08.038508+0000","last_peered":"2024-04-20T10:21:08.038508+0000","last_clean":"2024-04-20T10:21:08.038508+0000","last_became_active":"2024-04-20T10:21:02.644320+0000","last_became_peered":"2024-04-20T10:21:02.644320+0000","last_unstale":"2024-04-20T10:21:08.038508+0000","last_undegraded":"2024-04-20T10:21:08.038508+0000","last_fullsized":"2024-04-20T10:21:08.038508+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:06:59.919488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.038609+0000","last_change":"2024-04-20T10:21:01.616010+0000","last_active":"2024-04-20T10:21:08.038609+0000","last_peered":"2024-04-20T10:21:08.038609+0000","last_clean":"2024-04-20T10:21:08.038609+0000","last_became_active":"2024-04-20T10:21:01.615701+0000","last_became_peered":"2024-04-20T10:21:01.615701+0000","last_unstale":"2024-04-20T10:21:08.038609+0000","last_undegraded":"2024-04-20T10:21:08.038609+0000","last_fullsized":"2024-04-20T10:21:08.038609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:32:02.981747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542308+0000","last_change":"2024-04-20T10:21:06.909393+0000","last_active":"2024-04-20T10:21:08.542308+0000","last_peered":"2024-04-20T10:21:08.542308+0000","last_clean":"2024-04-20T10:21:08.542308+0000","last_became_active":"2024-04-20T10:21:06.907616+0000","last_became_peered":"2024-04-20T10:21:06.907616+0000","last_unstale":"2024-04-20T10:21:08.542308+0000","last_undegraded":"2024-04-20T10:21:08.542308+0000","last_fullsized":"2024-04-20T10:21:08.542308+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:07:32.815495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.390789+0000","last_change":"2024-04-20T10:21:02.654823+0000","last_active":"2024-04-20T10:21:08.390789+0000","last_peered":"2024-04-20T10:21:08.390789+0000","last_clean":"2024-04-20T10:21:08.390789+0000","last_became_active":"2024-04-20T10:21:02.654556+0000","last_became_peered":"2024-04-20T10:21:02.654556+0000","last_unstale":"2024-04-20T10:21:08.390789+0000","last_undegraded":"2024-04-20T10:21:08.390789+0000","last_fullsized":"2024-04-20T10:21:08.390789+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:52:24.023900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541418+0000","last_change":"2024-04-20T10:21:01.618588+0000","last_active":"2024-04-20T10:21:08.541418+0000","last_peered":"2024-04-20T10:21:08.541418+0000","last_clean":"2024-04-20T10:21:08.541418+0000","last_became_active":"2024-04-20T10:21:01.618294+0000","last_became_peered":"2024-04-20T10:21:01.618294+0000","last_unstale":"2024-04-20T10:21:08.541418+0000","last_undegraded":"2024-04-20T10:21:08.541418+0000","last_fullsized":"2024-04-20T10:21:08.541418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:34:27.492302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542602+0000","last_change":"2024-04-20T10:21:06.897913+0000","last_active":"2024-04-20T10:21:08.542602+0000","last_peered":"2024-04-20T10:21:08.542602+0000","last_clean":"2024-04-20T10:21:08.542602+0000","last_became_active":"2024-04-20T10:21:06.897536+0000","last_became_peered":"2024-04-20T10:21:06.897536+0000","last_unstale":"2024-04-20T10:21:08.542602+0000","last_undegraded":"2024-04-20T10:21:08.542602+0000","last_fullsized":"2024-04-20T10:21:08.542602+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:27:49.300931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.543513+0000","last_change":"2024-04-20T10:21:02.644302+0000","last_active":"2024-04-20T10:21:08.543513+0000","last_peered":"2024-04-20T10:21:08.543513+0000","last_clean":"2024-04-20T10:21:08.543513+0000","last_became_active":"2024-04-20T10:21:02.643236+0000","last_became_peered":"2024-04-20T10:21:02.643236+0000","last_unstale":"2024-04-20T10:21:08.543513+0000","last_undegraded":"2024-04-20T10:21:08.543513+0000","last_fullsized":"2024-04-20T10:21:08.543513+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:06:59.919488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.544270+0000","last_change":"2024-04-20T10:21:01.604692+0000","last_active":"2024-04-20T10:21:08.544270+0000","last_peered":"2024-04-20T10:21:08.544270+0000","last_clean":"2024-04-20T10:21:08.544270+0000","last_became_active":"2024-04-20T10:21:01.604345+0000","last_became_peered":"2024-04-20T10:21:01.604345+0000","last_unstale":"2024-04-20T10:21:08.544270+0000","last_undegraded":"2024-04-20T10:21:08.544270+0000","last_fullsized":"2024-04-20T10:21:08.544270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:29:47.733236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542165+0000","last_change":"2024-04-20T10:21:06.901480+0000","last_active":"2024-04-20T10:21:08.542165+0000","last_peered":"2024-04-20T10:21:08.542165+0000","last_clean":"2024-04-20T10:21:08.542165+0000","last_became_active":"2024-04-20T10:21:06.900820+0000","last_became_peered":"2024-04-20T10:21:06.900820+0000","last_unstale":"2024-04-20T10:21:08.542165+0000","last_undegraded":"2024-04-20T10:21:08.542165+0000","last_fullsized":"2024-04-20T10:21:08.542165+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:15:30.919284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.391163+0000","last_change":"2024-04-20T10:21:02.642723+0000","last_active":"2024-04-20T10:21:08.391163+0000","last_peered":"2024-04-20T10:21:08.391163+0000","last_clean":"2024-04-20T10:21:08.391163+0000","last_became_active":"2024-04-20T10:21:02.641828+0000","last_became_peered":"2024-04-20T10:21:02.641828+0000","last_unstale":"2024-04-20T10:21:08.391163+0000","last_undegraded":"2024-04-20T10:21:08.391163+0000","last_fullsized":"2024-04-20T10:21:08.391163+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:31:25.759865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:21.609415+0000","last_change":"2024-04-20T10:21:01.619387+0000","last_active":"2024-04-20T10:21:21.609415+0000","last_peered":"2024-04-20T10:21:21.609415+0000","last_clean":"2024-04-20T10:21:21.609415+0000","last_became_active":"2024-04-20T10:21:01.619119+0000","last_became_peered":"2024-04-20T10:21:01.619119+0000","last_unstale":"2024-04-20T10:21:21.609415+0000","last_undegraded":"2024-04-20T10:21:21.609415+0000","last_fullsized":"2024-04-20T10:21:21.609415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:55:41.208744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441019+0000","last_change":"2024-04-20T10:21:06.911043+0000","last_active":"2024-04-20T10:21:08.441019+0000","last_peered":"2024-04-20T10:21:08.441019+0000","last_clean":"2024-04-20T10:21:08.441019+0000","last_became_active":"2024-04-20T10:21:06.909896+0000","last_became_peered":"2024-04-20T10:21:06.909896+0000","last_unstale":"2024-04-20T10:21:08.441019+0000","last_undegraded":"2024-04-20T10:21:08.441019+0000","last_fullsized":"2024-04-20T10:21:08.441019+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:24:56.081482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.887153+0000","last_change":"2024-04-20T10:21:02.624575+0000","last_active":"2024-04-20T10:21:07.887153+0000","last_peered":"2024-04-20T10:21:07.887153+0000","last_clean":"2024-04-20T10:21:07.887153+0000","last_became_active":"2024-04-20T10:21:02.624230+0000","last_became_peered":"2024-04-20T10:21:02.624230+0000","last_unstale":"2024-04-20T10:21:07.887153+0000","last_undegraded":"2024-04-20T10:21:07.887153+0000","last_fullsized":"2024-04-20T10:21:07.887153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:07:51.297896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441862+0000","last_change":"2024-04-20T10:21:01.619231+0000","last_active":"2024-04-20T10:21:08.441862+0000","last_peered":"2024-04-20T10:21:08.441862+0000","last_clean":"2024-04-20T10:21:08.441862+0000","last_became_active":"2024-04-20T10:21:01.618872+0000","last_became_peered":"2024-04-20T10:21:01.618872+0000","last_unstale":"2024-04-20T10:21:08.441862+0000","last_undegraded":"2024-04-20T10:21:08.441862+0000","last_fullsized":"2024-04-20T10:21:08.441862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:38:10.686751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.886989+0000","last_change":"2024-04-20T10:21:06.908460+0000","last_active":"2024-04-20T10:21:07.886989+0000","last_peered":"2024-04-20T10:21:07.886989+0000","last_clean":"2024-04-20T10:21:07.886989+0000","last_became_active":"2024-04-20T10:21:06.907722+0000","last_became_peered":"2024-04-20T10:21:06.907722+0000","last_unstale":"2024-04-20T10:21:07.886989+0000","last_undegraded":"2024-04-20T10:21:07.886989+0000","last_fullsized":"2024-04-20T10:21:07.886989+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:06:15.702589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.544085+0000","last_change":"2024-04-20T10:21:02.646092+0000","last_active":"2024-04-20T10:21:08.544085+0000","last_peered":"2024-04-20T10:21:08.544085+0000","last_clean":"2024-04-20T10:21:08.544085+0000","last_became_active":"2024-04-20T10:21:02.645788+0000","last_became_peered":"2024-04-20T10:21:02.645788+0000","last_unstale":"2024-04-20T10:21:08.544085+0000","last_undegraded":"2024-04-20T10:21:08.544085+0000","last_fullsized":"2024-04-20T10:21:08.544085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:22:36.605724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542309+0000","last_change":"2024-04-20T10:21:01.623128+0000","last_active":"2024-04-20T10:21:08.542309+0000","last_peered":"2024-04-20T10:21:08.542309+0000","last_clean":"2024-04-20T10:21:08.542309+0000","last_became_active":"2024-04-20T10:21:01.622867+0000","last_became_peered":"2024-04-20T10:21:01.622867+0000","last_unstale":"2024-04-20T10:21:08.542309+0000","last_undegraded":"2024-04-20T10:21:08.542309+0000","last_fullsized":"2024-04-20T10:21:08.542309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:50:23.957089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541079+0000","last_change":"2024-04-20T10:21:06.915147+0000","last_active":"2024-04-20T10:21:08.541079+0000","last_peered":"2024-04-20T10:21:08.541079+0000","last_clean":"2024-04-20T10:21:08.541079+0000","last_became_active":"2024-04-20T10:21:06.913832+0000","last_became_peered":"2024-04-20T10:21:06.913832+0000","last_unstale":"2024-04-20T10:21:08.541079+0000","last_undegraded":"2024-04-20T10:21:08.541079+0000","last_fullsized":"2024-04-20T10:21:08.541079+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:10:39.166791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542260+0000","last_change":"2024-04-20T10:21:02.638354+0000","last_active":"2024-04-20T10:21:08.542260+0000","last_peered":"2024-04-20T10:21:08.542260+0000","last_clean":"2024-04-20T10:21:08.542260+0000","last_became_active":"2024-04-20T10:21:02.638065+0000","last_became_peered":"2024-04-20T10:21:02.638065+0000","last_unstale":"2024-04-20T10:21:08.542260+0000","last_undegraded":"2024-04-20T10:21:08.542260+0000","last_fullsized":"2024-04-20T10:21:08.542260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:12:12.144216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.039425+0000","last_change":"2024-04-20T10:21:01.621766+0000","last_active":"2024-04-20T10:21:08.039425+0000","last_peered":"2024-04-20T10:21:08.039425+0000","last_clean":"2024-04-20T10:21:08.039425+0000","last_became_active":"2024-04-20T10:21:01.621546+0000","last_became_peered":"2024-04-20T10:21:01.621546+0000","last_unstale":"2024-04-20T10:21:08.039425+0000","last_undegraded":"2024-04-20T10:21:08.039425+0000","last_fullsized":"2024-04-20T10:21:08.039425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:01:19.566351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540739+0000","last_change":"2024-04-20T10:21:06.910220+0000","last_active":"2024-04-20T10:21:08.540739+0000","last_peered":"2024-04-20T10:21:08.540739+0000","last_clean":"2024-04-20T10:21:08.540739+0000","last_became_active":"2024-04-20T10:21:06.909876+0000","last_became_peered":"2024-04-20T10:21:06.909876+0000","last_unstale":"2024-04-20T10:21:08.540739+0000","last_undegraded":"2024-04-20T10:21:08.540739+0000","last_fullsized":"2024-04-20T10:21:08.540739+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:27:04.462330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.543797+0000","last_change":"2024-04-20T10:21:06.903823+0000","last_active":"2024-04-20T10:21:08.543797+0000","last_peered":"2024-04-20T10:21:08.543797+0000","last_clean":"2024-04-20T10:21:08.543797+0000","last_became_active":"2024-04-20T10:21:06.903100+0000","last_became_peered":"2024-04-20T10:21:06.903100+0000","last_unstale":"2024-04-20T10:21:08.543797+0000","last_undegraded":"2024-04-20T10:21:08.543797+0000","last_fullsized":"2024-04-20T10:21:08.543797+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:24:56.081482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.440526+0000","last_change":"2024-04-20T10:21:02.636194+0000","last_active":"2024-04-20T10:21:08.440526+0000","last_peered":"2024-04-20T10:21:08.440526+0000","last_clean":"2024-04-20T10:21:08.440526+0000","last_became_active":"2024-04-20T10:21:02.635907+0000","last_became_peered":"2024-04-20T10:21:02.635907+0000","last_unstale":"2024-04-20T10:21:08.440526+0000","last_undegraded":"2024-04-20T10:21:08.440526+0000","last_fullsized":"2024-04-20T10:21:08.440526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:06:59.919488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540844+0000","last_change":"2024-04-20T10:21:01.617415+0000","last_active":"2024-04-20T10:21:08.540844+0000","last_peered":"2024-04-20T10:21:08.540844+0000","last_clean":"2024-04-20T10:21:08.540844+0000","last_became_active":"2024-04-20T10:21:01.617082+0000","last_became_peered":"2024-04-20T10:21:01.617082+0000","last_unstale":"2024-04-20T10:21:08.540844+0000","last_undegraded":"2024-04-20T10:21:08.540844+0000","last_fullsized":"2024-04-20T10:21:08.540844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:56:45.348182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.039217+0000","last_change":"2024-04-20T10:21:06.915304+0000","last_active":"2024-04-20T10:21:08.039217+0000","last_peered":"2024-04-20T10:21:08.039217+0000","last_clean":"2024-04-20T10:21:08.039217+0000","last_became_active":"2024-04-20T10:21:06.914915+0000","last_became_peered":"2024-04-20T10:21:06.914915+0000","last_unstale":"2024-04-20T10:21:08.039217+0000","last_undegraded":"2024-04-20T10:21:08.039217+0000","last_fullsized":"2024-04-20T10:21:08.039217+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:52:28.298927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540918+0000","last_change":"2024-04-20T10:21:02.646474+0000","last_active":"2024-04-20T10:21:08.540918+0000","last_peered":"2024-04-20T10:21:08.540918+0000","last_clean":"2024-04-20T10:21:08.540918+0000","last_became_active":"2024-04-20T10:21:02.646219+0000","last_became_peered":"2024-04-20T10:21:02.646219+0000","last_unstale":"2024-04-20T10:21:08.540918+0000","last_undegraded":"2024-04-20T10:21:08.540918+0000","last_fullsized":"2024-04-20T10:21:08.540918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:06:59.919488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541015+0000","last_change":"2024-04-20T10:21:01.609406+0000","last_active":"2024-04-20T10:21:08.541015+0000","last_peered":"2024-04-20T10:21:08.541015+0000","last_clean":"2024-04-20T10:21:08.541015+0000","last_became_active":"2024-04-20T10:21:01.609064+0000","last_became_peered":"2024-04-20T10:21:01.609064+0000","last_unstale":"2024-04-20T10:21:08.541015+0000","last_undegraded":"2024-04-20T10:21:08.541015+0000","last_fullsized":"2024-04-20T10:21:08.541015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:28:01.100842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.039237+0000","last_change":"2024-04-20T10:21:01.615891+0000","last_active":"2024-04-20T10:21:08.039237+0000","last_peered":"2024-04-20T10:21:08.039237+0000","last_clean":"2024-04-20T10:21:08.039237+0000","last_became_active":"2024-04-20T10:21:01.615584+0000","last_became_peered":"2024-04-20T10:21:01.615584+0000","last_unstale":"2024-04-20T10:21:08.039237+0000","last_undegraded":"2024-04-20T10:21:08.039237+0000","last_fullsized":"2024-04-20T10:21:08.039237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:59:37.356548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.544232+0000","last_change":"2024-04-20T10:21:02.643605+0000","last_active":"2024-04-20T10:21:08.544232+0000","last_peered":"2024-04-20T10:21:08.544232+0000","last_clean":"2024-04-20T10:21:08.544232+0000","last_became_active":"2024-04-20T10:21:02.643327+0000","last_became_peered":"2024-04-20T10:21:02.643327+0000","last_unstale":"2024-04-20T10:21:08.544232+0000","last_undegraded":"2024-04-20T10:21:08.544232+0000","last_fullsized":"2024-04-20T10:21:08.544232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:01:55.363202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.882164+0000","last_change":"2024-04-20T10:21:06.892955+0000","last_active":"2024-04-20T10:21:07.882164+0000","last_peered":"2024-04-20T10:21:07.882164+0000","last_clean":"2024-04-20T10:21:07.882164+0000","last_became_active":"2024-04-20T10:21:06.892534+0000","last_became_peered":"2024-04-20T10:21:06.892534+0000","last_unstale":"2024-04-20T10:21:07.882164+0000","last_undegraded":"2024-04-20T10:21:07.882164+0000","last_fullsized":"2024-04-20T10:21:07.882164+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:17:28.840720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541245+0000","last_change":"2024-04-20T10:21:01.618877+0000","last_active":"2024-04-20T10:21:08.541245+0000","last_peered":"2024-04-20T10:21:08.541245+0000","last_clean":"2024-04-20T10:21:08.541245+0000","last_became_active":"2024-04-20T10:21:01.618549+0000","last_became_peered":"2024-04-20T10:21:01.618549+0000","last_unstale":"2024-04-20T10:21:08.541245+0000","last_undegraded":"2024-04-20T10:21:08.541245+0000","last_fullsized":"2024-04-20T10:21:08.541245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:00.552591+0000","last_clean_scrub_stamp":"2024-04-20T10:21:00.552591+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:36:15.966290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.886244+0000","last_change":"2024-04-20T10:21:02.645042+0000","last_active":"2024-04-20T10:21:07.886244+0000","last_peered":"2024-04-20T10:21:07.886244+0000","last_clean":"2024-04-20T10:21:07.886244+0000","last_became_active":"2024-04-20T10:21:02.644630+0000","last_became_peered":"2024-04-20T10:21:02.644630+0000","last_unstale":"2024-04-20T10:21:07.886244+0000","last_undegraded":"2024-04-20T10:21:07.886244+0000","last_fullsized":"2024-04-20T10:21:07.886244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:59:38.370072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542570+0000","last_change":"2024-04-20T10:21:06.888033+0000","last_active":"2024-04-20T10:21:08.542570+0000","last_peered":"2024-04-20T10:21:08.542570+0000","last_clean":"2024-04-20T10:21:08.542570+0000","last_became_active":"2024-04-20T10:21:06.887672+0000","last_became_peered":"2024-04-20T10:21:06.887672+0000","last_unstale":"2024-04-20T10:21:08.542570+0000","last_undegraded":"2024-04-20T10:21:08.542570+0000","last_fullsized":"2024-04-20T10:21:08.542570+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:05.841143+0000","last_clean_scrub_stamp":"2024-04-20T10:21:05.841143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:13:54.194111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.038167+0000","last_change":"2024-04-20T10:21:02.644672+0000","last_active":"2024-04-20T10:21:08.038167+0000","last_peered":"2024-04-20T10:21:08.038167+0000","last_clean":"2024-04-20T10:21:08.038167+0000","last_became_active":"2024-04-20T10:21:02.644327+0000","last_became_peered":"2024-04-20T10:21:02.644327+0000","last_unstale":"2024-04-20T10:21:08.038167+0000","last_undegraded":"2024-04-20T10:21:08.038167+0000","last_fullsized":"2024-04-20T10:21:08.038167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:28:54.408773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.881607+0000","last_change":"2024-04-20T10:21:02.624910+0000","last_active":"2024-04-20T10:21:07.881607+0000","last_peered":"2024-04-20T10:21:07.881607+0000","last_clean":"2024-04-20T10:21:07.881607+0000","last_became_active":"2024-04-20T10:21:02.624555+0000","last_became_peered":"2024-04-20T10:21:02.624555+0000","last_unstale":"2024-04-20T10:21:07.881607+0000","last_undegraded":"2024-04-20T10:21:07.881607+0000","last_fullsized":"2024-04-20T10:21:07.881607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:07:51.297896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541176+0000","last_change":"2024-04-20T10:21:02.636958+0000","last_active":"2024-04-20T10:21:08.541176+0000","last_peered":"2024-04-20T10:21:08.541176+0000","last_clean":"2024-04-20T10:21:08.541176+0000","last_became_active":"2024-04-20T10:21:02.636594+0000","last_became_peered":"2024-04-20T10:21:02.636594+0000","last_unstale":"2024-04-20T10:21:08.541176+0000","last_undegraded":"2024-04-20T10:21:08.541176+0000","last_fullsized":"2024-04-20T10:21:08.541176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:12:07.469367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.543932+0000","last_change":"2024-04-20T10:21:02.649761+0000","last_active":"2024-04-20T10:21:08.543932+0000","last_peered":"2024-04-20T10:21:08.543932+0000","last_clean":"2024-04-20T10:21:08.543932+0000","last_became_active":"2024-04-20T10:21:02.649471+0000","last_became_peered":"2024-04-20T10:21:02.649471+0000","last_unstale":"2024-04-20T10:21:08.543932+0000","last_undegraded":"2024-04-20T10:21:08.543932+0000","last_fullsized":"2024-04-20T10:21:08.543932+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:14:16.934710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.543183+0000","last_change":"2024-04-20T10:21:02.644223+0000","last_active":"2024-04-20T10:21:08.543183+0000","last_peered":"2024-04-20T10:21:08.543183+0000","last_clean":"2024-04-20T10:21:08.543183+0000","last_became_active":"2024-04-20T10:21:02.643893+0000","last_became_peered":"2024-04-20T10:21:02.643893+0000","last_unstale":"2024-04-20T10:21:08.543183+0000","last_undegraded":"2024-04-20T10:21:08.543183+0000","last_fullsized":"2024-04-20T10:21:08.543183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:09:41.966188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441369+0000","last_change":"2024-04-20T10:21:02.636144+0000","last_active":"2024-04-20T10:21:08.441369+0000","last_peered":"2024-04-20T10:21:08.441369+0000","last_clean":"2024-04-20T10:21:08.441369+0000","last_became_active":"2024-04-20T10:21:02.635834+0000","last_became_peered":"2024-04-20T10:21:02.635834+0000","last_unstale":"2024-04-20T10:21:08.441369+0000","last_undegraded":"2024-04-20T10:21:08.441369+0000","last_fullsized":"2024-04-20T10:21:08.441369+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:10:15.206686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.039750+0000","last_change":"2024-04-20T10:21:02.643146+0000","last_active":"2024-04-20T10:21:08.039750+0000","last_peered":"2024-04-20T10:21:08.039750+0000","last_clean":"2024-04-20T10:21:08.039750+0000","last_became_active":"2024-04-20T10:21:02.639521+0000","last_became_peered":"2024-04-20T10:21:02.639521+0000","last_unstale":"2024-04-20T10:21:08.039750+0000","last_undegraded":"2024-04-20T10:21:08.039750+0000","last_fullsized":"2024-04-20T10:21:08.039750+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:01:55.363202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542473+0000","last_change":"2024-04-20T10:21:02.631553+0000","last_active":"2024-04-20T10:21:08.542473+0000","last_peered":"2024-04-20T10:21:08.542473+0000","last_clean":"2024-04-20T10:21:08.542473+0000","last_became_active":"2024-04-20T10:21:02.626210+0000","last_became_peered":"2024-04-20T10:21:02.626210+0000","last_unstale":"2024-04-20T10:21:08.542473+0000","last_undegraded":"2024-04-20T10:21:08.542473+0000","last_fullsized":"2024-04-20T10:21:08.542473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:02:26.716730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540787+0000","last_change":"2024-04-20T10:21:02.625457+0000","last_active":"2024-04-20T10:21:08.540787+0000","last_peered":"2024-04-20T10:21:08.540787+0000","last_clean":"2024-04-20T10:21:08.540787+0000","last_became_active":"2024-04-20T10:21:02.624987+0000","last_became_peered":"2024-04-20T10:21:02.624987+0000","last_unstale":"2024-04-20T10:21:08.540787+0000","last_undegraded":"2024-04-20T10:21:08.540787+0000","last_fullsized":"2024-04-20T10:21:08.540787+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:13:44.633309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540862+0000","last_change":"2024-04-20T10:21:02.628208+0000","last_active":"2024-04-20T10:21:08.540862+0000","last_peered":"2024-04-20T10:21:08.540862+0000","last_clean":"2024-04-20T10:21:08.540862+0000","last_became_active":"2024-04-20T10:21:02.625255+0000","last_became_peered":"2024-04-20T10:21:02.625255+0000","last_unstale":"2024-04-20T10:21:08.540862+0000","last_undegraded":"2024-04-20T10:21:08.540862+0000","last_fullsized":"2024-04-20T10:21:08.540862+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:06:35.349863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541152+0000","last_change":"2024-04-20T10:21:02.629528+0000","last_active":"2024-04-20T10:21:08.541152+0000","last_peered":"2024-04-20T10:21:08.541152+0000","last_clean":"2024-04-20T10:21:08.541152+0000","last_became_active":"2024-04-20T10:21:02.626300+0000","last_became_peered":"2024-04-20T10:21:02.626300+0000","last_unstale":"2024-04-20T10:21:08.541152+0000","last_undegraded":"2024-04-20T10:21:08.541152+0000","last_fullsized":"2024-04-20T10:21:08.541152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:17:15.177678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.543489+0000","last_change":"2024-04-20T10:21:02.657102+0000","last_active":"2024-04-20T10:21:08.543489+0000","last_peered":"2024-04-20T10:21:08.543489+0000","last_clean":"2024-04-20T10:21:08.543489+0000","last_became_active":"2024-04-20T10:21:02.656810+0000","last_became_peered":"2024-04-20T10:21:02.656810+0000","last_unstale":"2024-04-20T10:21:08.543489+0000","last_undegraded":"2024-04-20T10:21:08.543489+0000","last_fullsized":"2024-04-20T10:21:08.543489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:04:04.238410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.039198+0000","last_change":"2024-04-20T10:21:02.639499+0000","last_active":"2024-04-20T10:21:08.039198+0000","last_peered":"2024-04-20T10:21:08.039198+0000","last_clean":"2024-04-20T10:21:08.039198+0000","last_became_active":"2024-04-20T10:21:02.638713+0000","last_became_peered":"2024-04-20T10:21:02.638713+0000","last_unstale":"2024-04-20T10:21:08.039198+0000","last_undegraded":"2024-04-20T10:21:08.039198+0000","last_fullsized":"2024-04-20T10:21:08.039198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:31:53.442042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.391493+0000","last_change":"2024-04-20T10:21:02.639481+0000","last_active":"2024-04-20T10:21:08.391493+0000","last_peered":"2024-04-20T10:21:08.391493+0000","last_clean":"2024-04-20T10:21:08.391493+0000","last_became_active":"2024-04-20T10:21:02.639198+0000","last_became_peered":"2024-04-20T10:21:02.639198+0000","last_unstale":"2024-04-20T10:21:08.391493+0000","last_undegraded":"2024-04-20T10:21:08.391493+0000","last_fullsized":"2024-04-20T10:21:08.391493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:35:54.235610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.883015+0000","last_change":"2024-04-20T10:21:02.640221+0000","last_active":"2024-04-20T10:21:07.883015+0000","last_peered":"2024-04-20T10:21:07.883015+0000","last_clean":"2024-04-20T10:21:07.883015+0000","last_became_active":"2024-04-20T10:21:02.639833+0000","last_became_peered":"2024-04-20T10:21:02.639833+0000","last_unstale":"2024-04-20T10:21:07.883015+0000","last_undegraded":"2024-04-20T10:21:07.883015+0000","last_fullsized":"2024-04-20T10:21:07.883015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:38:11.700275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.542925+0000","last_change":"2024-04-20T10:21:02.640233+0000","last_active":"2024-04-20T10:21:08.542925+0000","last_peered":"2024-04-20T10:21:08.542925+0000","last_clean":"2024-04-20T10:21:08.542925+0000","last_became_active":"2024-04-20T10:21:02.639913+0000","last_became_peered":"2024-04-20T10:21:02.639913+0000","last_unstale":"2024-04-20T10:21:08.542925+0000","last_undegraded":"2024-04-20T10:21:08.542925+0000","last_fullsized":"2024-04-20T10:21:08.542925+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:45:33.753385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540514+0000","last_change":"2024-04-20T10:21:02.643714+0000","last_active":"2024-04-20T10:21:08.540514+0000","last_peered":"2024-04-20T10:21:08.540514+0000","last_clean":"2024-04-20T10:21:08.540514+0000","last_became_active":"2024-04-20T10:21:02.643403+0000","last_became_peered":"2024-04-20T10:21:02.643403+0000","last_unstale":"2024-04-20T10:21:08.540514+0000","last_undegraded":"2024-04-20T10:21:08.540514+0000","last_fullsized":"2024-04-20T10:21:08.540514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:27:44.403086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.440671+0000","last_change":"2024-04-20T10:21:02.643476+0000","last_active":"2024-04-20T10:21:08.440671+0000","last_peered":"2024-04-20T10:21:08.440671+0000","last_clean":"2024-04-20T10:21:08.440671+0000","last_became_active":"2024-04-20T10:21:02.643180+0000","last_became_peered":"2024-04-20T10:21:02.643180+0000","last_unstale":"2024-04-20T10:21:08.440671+0000","last_undegraded":"2024-04-20T10:21:08.440671+0000","last_fullsized":"2024-04-20T10:21:08.440671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:31:53.442042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.543269+0000","last_change":"2024-04-20T10:21:02.646081+0000","last_active":"2024-04-20T10:21:08.543269+0000","last_peered":"2024-04-20T10:21:08.543269+0000","last_clean":"2024-04-20T10:21:08.543269+0000","last_became_active":"2024-04-20T10:21:02.645798+0000","last_became_peered":"2024-04-20T10:21:02.645798+0000","last_unstale":"2024-04-20T10:21:08.543269+0000","last_undegraded":"2024-04-20T10:21:08.543269+0000","last_fullsized":"2024-04-20T10:21:08.543269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:45:52.178877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540989+0000","last_change":"2024-04-20T10:21:02.639147+0000","last_active":"2024-04-20T10:21:08.540989+0000","last_peered":"2024-04-20T10:21:08.540989+0000","last_clean":"2024-04-20T10:21:08.540989+0000","last_became_active":"2024-04-20T10:21:02.638861+0000","last_became_peered":"2024-04-20T10:21:02.638861+0000","last_unstale":"2024-04-20T10:21:08.540989+0000","last_undegraded":"2024-04-20T10:21:08.540989+0000","last_fullsized":"2024-04-20T10:21:08.540989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:22:36.605724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:07.887047+0000","last_change":"2024-04-20T10:21:02.638429+0000","last_active":"2024-04-20T10:21:07.887047+0000","last_peered":"2024-04-20T10:21:07.887047+0000","last_clean":"2024-04-20T10:21:07.887047+0000","last_became_active":"2024-04-20T10:21:02.638131+0000","last_became_peered":"2024-04-20T10:21:02.638131+0000","last_unstale":"2024-04-20T10:21:07.887047+0000","last_undegraded":"2024-04-20T10:21:07.887047+0000","last_fullsized":"2024-04-20T10:21:07.887047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:50:28.730495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.039192+0000","last_change":"2024-04-20T10:21:02.646725+0000","last_active":"2024-04-20T10:21:08.039192+0000","last_peered":"2024-04-20T10:21:08.039192+0000","last_clean":"2024-04-20T10:21:08.039192+0000","last_became_active":"2024-04-20T10:21:02.646439+0000","last_became_peered":"2024-04-20T10:21:02.646439+0000","last_unstale":"2024-04-20T10:21:08.039192+0000","last_undegraded":"2024-04-20T10:21:08.039192+0000","last_fullsized":"2024-04-20T10:21:08.039192+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:06:35.349863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.039464+0000","last_change":"2024-04-20T10:21:02.641401+0000","last_active":"2024-04-20T10:21:08.039464+0000","last_peered":"2024-04-20T10:21:08.039464+0000","last_clean":"2024-04-20T10:21:08.039464+0000","last_became_active":"2024-04-20T10:21:02.640962+0000","last_became_peered":"2024-04-20T10:21:02.640962+0000","last_unstale":"2024-04-20T10:21:08.039464+0000","last_undegraded":"2024-04-20T10:21:08.039464+0000","last_fullsized":"2024-04-20T10:21:08.039464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:22:36.605724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.391828+0000","last_change":"2024-04-20T10:21:02.654762+0000","last_active":"2024-04-20T10:21:08.391828+0000","last_peered":"2024-04-20T10:21:08.391828+0000","last_clean":"2024-04-20T10:21:08.391828+0000","last_became_active":"2024-04-20T10:21:02.654497+0000","last_became_peered":"2024-04-20T10:21:02.654497+0000","last_unstale":"2024-04-20T10:21:08.391828+0000","last_undegraded":"2024-04-20T10:21:08.391828+0000","last_fullsized":"2024-04-20T10:21:08.391828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:46:41.173563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540653+0000","last_change":"2024-04-20T10:21:02.630507+0000","last_active":"2024-04-20T10:21:08.540653+0000","last_peered":"2024-04-20T10:21:08.540653+0000","last_clean":"2024-04-20T10:21:08.540653+0000","last_became_active":"2024-04-20T10:21:02.626482+0000","last_became_peered":"2024-04-20T10:21:02.626482+0000","last_unstale":"2024-04-20T10:21:08.540653+0000","last_undegraded":"2024-04-20T10:21:08.540653+0000","last_fullsized":"2024-04-20T10:21:08.540653+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:28:54.408773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.390187+0000","last_change":"2024-04-20T10:21:02.649796+0000","last_active":"2024-04-20T10:21:08.390187+0000","last_peered":"2024-04-20T10:21:08.390187+0000","last_clean":"2024-04-20T10:21:08.390187+0000","last_became_active":"2024-04-20T10:21:02.649541+0000","last_became_peered":"2024-04-20T10:21:02.649541+0000","last_unstale":"2024-04-20T10:21:08.390187+0000","last_undegraded":"2024-04-20T10:21:08.390187+0000","last_fullsized":"2024-04-20T10:21:08.390187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:02:26.716730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.540951+0000","last_change":"2024-04-20T10:21:02.653552+0000","last_active":"2024-04-20T10:21:08.540951+0000","last_peered":"2024-04-20T10:21:08.540951+0000","last_clean":"2024-04-20T10:21:08.540951+0000","last_became_active":"2024-04-20T10:21:02.653288+0000","last_became_peered":"2024-04-20T10:21:02.653288+0000","last_unstale":"2024-04-20T10:21:08.540951+0000","last_undegraded":"2024-04-20T10:21:08.540951+0000","last_fullsized":"2024-04-20T10:21:08.540951+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:49:26.986908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.390549+0000","last_change":"2024-04-20T10:21:02.633119+0000","last_active":"2024-04-20T10:21:08.390549+0000","last_peered":"2024-04-20T10:21:08.390549+0000","last_clean":"2024-04-20T10:21:08.390549+0000","last_became_active":"2024-04-20T10:21:02.632816+0000","last_became_peered":"2024-04-20T10:21:02.632816+0000","last_unstale":"2024-04-20T10:21:08.390549+0000","last_undegraded":"2024-04-20T10:21:08.390549+0000","last_fullsized":"2024-04-20T10:21:08.390549+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:16:04.625809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.541721+0000","last_change":"2024-04-20T10:21:02.638226+0000","last_active":"2024-04-20T10:21:08.541721+0000","last_peered":"2024-04-20T10:21:08.541721+0000","last_clean":"2024-04-20T10:21:08.541721+0000","last_became_active":"2024-04-20T10:21:02.637640+0000","last_became_peered":"2024-04-20T10:21:02.637640+0000","last_unstale":"2024-04-20T10:21:08.541721+0000","last_undegraded":"2024-04-20T10:21:08.541721+0000","last_fullsized":"2024-04-20T10:21:08.541721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:44:55.208482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.544254+0000","last_change":"2024-04-20T10:21:02.641981+0000","last_active":"2024-04-20T10:21:08.544254+0000","last_peered":"2024-04-20T10:21:08.544254+0000","last_clean":"2024-04-20T10:21:08.544254+0000","last_became_active":"2024-04-20T10:21:02.640073+0000","last_became_peered":"2024-04-20T10:21:02.640073+0000","last_unstale":"2024-04-20T10:21:08.544254+0000","last_undegraded":"2024-04-20T10:21:08.544254+0000","last_fullsized":"2024-04-20T10:21:08.544254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:28:54.408773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.441547+0000","last_change":"2024-04-20T10:21:02.646636+0000","last_active":"2024-04-20T10:21:08.441547+0000","last_peered":"2024-04-20T10:21:08.441547+0000","last_clean":"2024-04-20T10:21:08.441547+0000","last_became_active":"2024-04-20T10:21:02.646359+0000","last_became_peered":"2024-04-20T10:21:02.646359+0000","last_unstale":"2024-04-20T10:21:08.441547+0000","last_undegraded":"2024-04-20T10:21:08.441547+0000","last_fullsized":"2024-04-20T10:21:08.441547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:22:36.605724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-04-20T10:21:08.391583+0000","last_change":"2024-04-20T10:21:02.652784+0000","last_active":"2024-04-20T10:21:08.391583+0000","last_peered":"2024-04-20T10:21:08.391583+0000","last_clean":"2024-04-20T10:21:08.391583+0000","last_became_active":"2024-04-20T10:21:02.652454+0000","last_became_peered":"2024-04-20T10:21:02.652454+0000","last_unstale":"2024-04-20T10:21:08.391583+0000","last_undegraded":"2024-04-20T10:21:08.391583+0000","last_fullsized":"2024-04-20T10:21:08.391583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:21:01.566115+0000","last_clean_scrub_stamp":"2024-04-20T10:21:01.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:38:01.376108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542155,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27860,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343980,"statfs":{"total":96636764160,"available":96608235520,"internally_reserved":0,"allocated":860160,"data_stored":694509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542156,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":107401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542155,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":103649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542155,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":103685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542155,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":274432,"data_stored":107401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542155,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":262144,"data_stored":103559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542157,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":103719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542157,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":868352,"data_stored":694565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T10:21:37.719 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-20T10:21:37.719 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-20T10:21:37.719 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-20T10:21:37.719 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-20T10:21:37.952 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.952+0000 7fb52e600640 1 Processor -- start 2024-04-20T10:21:37.952 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.953+0000 7fb52e600640 1 -- start start 2024-04-20T10:21:37.954 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.954+0000 7fb52e600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb528159e10 0x7fb528071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:37.954 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.954+0000 7fb52e600640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb52806da90 0x7fb528071fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:37.954 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.954+0000 7fb52e600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb52815cc90 con 0x7fb52815a1f0 2024-04-20T10:21:37.954 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.954+0000 7fb52e600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fb528072900 con 0x7fb528159e10 2024-04-20T10:21:37.954 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.954+0000 7fb52e600640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fb52815ca40 con 0x7fb52806da90 2024-04-20T10:21:37.954 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.955+0000 7fb52d200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb528159e10 0x7fb528071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:37.954 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.955+0000 7fb52d200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb528159e10 0x7fb528071a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58608/0 (socket says 172.21.15.121:58608) 2024-04-20T10:21:37.954 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.955+0000 7fb52d200640 1 -- 172.21.15.121:0/2754948083 learned_addr learned my addr 172.21.15.121:0/2754948083 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:37.955 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.955+0000 7fb52dc00640 1 --2- 172.21.15.121:0/2754948083 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb52806da90 0x7fb528071fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:37.955 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.955+0000 7fb52d200640 1 -- 172.21.15.121:0/2754948083 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb52806da90 msgr2=0x7fb528071fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:37.956 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.955+0000 7fb52d200640 1 --2- 172.21.15.121:0/2754948083 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb52806da90 0x7fb528071fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:37.956 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.955+0000 7fb52d200640 1 -- 172.21.15.121:0/2754948083 >> v1:172.21.15.123:6789/0 conn(0x7fb52815a1f0 legacy=0x7fb528160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:37.956 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.955+0000 7fb52d200640 1 -- 172.21.15.121:0/2754948083 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb52815cf70 con 0x7fb528159e10 2024-04-20T10:21:37.956 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.956+0000 7fb52d200640 1 --2- 172.21.15.121:0/2754948083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb528159e10 0x7fb528071a80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb518009e50 tx=0x7fb51802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b19f1d51c579a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:37.957 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.956+0000 7fb527e00640 1 -- 172.21.15.121:0/2754948083 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51802cb00 con 0x7fb528159e10 2024-04-20T10:21:37.957 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.957+0000 7fb527e00640 1 -- 172.21.15.121:0/2754948083 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb51802cca0 con 0x7fb528159e10 2024-04-20T10:21:37.957 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.957+0000 7fb527e00640 1 -- 172.21.15.121:0/2754948083 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb518034610 con 0x7fb528159e10 2024-04-20T10:21:37.957 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.957+0000 7fb52e600640 1 -- 172.21.15.121:0/2754948083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb528159e10 msgr2=0x7fb528071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:37.957 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.957+0000 7fb52e600640 1 --2- 172.21.15.121:0/2754948083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb528159e10 0x7fb528071a80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb518009e50 tx=0x7fb51802bf60 comp rx=0 tx=0).stop 2024-04-20T10:21:37.957 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.957+0000 7fb52e600640 1 -- 172.21.15.121:0/2754948083 shutdown_connections 2024-04-20T10:21:37.957 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.957+0000 7fb52e600640 1 --2- 172.21.15.121:0/2754948083 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb52806da90 0x7fb528071fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:37.957 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.957+0000 7fb52e600640 1 --2- 172.21.15.121:0/2754948083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb528159e10 0x7fb528071a80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:37.957 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.957+0000 7fb52e600640 1 -- 172.21.15.121:0/2754948083 >> 172.21.15.121:0/2754948083 conn(0x7fb5280d0040 msgr2=0x7fb5280d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:37.958 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.957+0000 7fb52e600640 1 -- 172.21.15.121:0/2754948083 shutdown_connections 2024-04-20T10:21:37.958 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.957+0000 7fb52e600640 1 -- 172.21.15.121:0/2754948083 wait complete. 2024-04-20T10:21:37.958 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.959+0000 7fb52e600640 1 Processor -- start 2024-04-20T10:21:37.958 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.959+0000 7fb52e600640 1 -- start start 2024-04-20T10:21:37.961 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.960+0000 7fb52e600640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb52806da90 0x7fb5280b3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:37.961 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.960+0000 7fb52e600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb528159e10 0x7fb52815ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:37.961 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.960+0000 7fb52e600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb52815a1f0 0x7fb52815b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:37.961 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.960+0000 7fb52e600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb52815d810 con 0x7fb528159e10 2024-04-20T10:21:37.961 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.960+0000 7fb52e600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb52815d530 con 0x7fb52815a1f0 2024-04-20T10:21:37.961 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.960+0000 7fb52e600640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fb52806b900 con 0x7fb52806da90 2024-04-20T10:21:37.961 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.960+0000 7fb51fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb528159e10 0x7fb52815ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:37.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.961+0000 7fb51fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb528159e10 0x7fb52815ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58612/0 (socket says 172.21.15.121:58612) 2024-04-20T10:21:37.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.961+0000 7fb51fe00640 1 -- 172.21.15.121:0/4001584216 learned_addr learned my addr 172.21.15.121:0/4001584216 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:37.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.961+0000 7fb52dc00640 1 --2- 172.21.15.121:0/4001584216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb52815a1f0 0x7fb52815b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:37.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.961+0000 7fb52d200640 1 --2- 172.21.15.121:0/4001584216 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb52806da90 0x7fb5280b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:37.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.961+0000 7fb51fe00640 1 -- 172.21.15.121:0/4001584216 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb52806da90 msgr2=0x7fb5280b3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:37.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.961+0000 7fb51fe00640 1 --2- 172.21.15.121:0/4001584216 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb52806da90 0x7fb5280b3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:37.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.961+0000 7fb51fe00640 1 -- 172.21.15.121:0/4001584216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb52815a1f0 msgr2=0x7fb52815b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:37.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.961+0000 7fb51fe00640 1 --2- 172.21.15.121:0/4001584216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb52815a1f0 0x7fb52815b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:37.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.961+0000 7fb51fe00640 1 -- 172.21.15.121:0/4001584216 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5280b3560 con 0x7fb528159e10 2024-04-20T10:21:37.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.961+0000 7fb52dc00640 1 --2- 172.21.15.121:0/4001584216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb52815a1f0 0x7fb52815b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:37.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.961+0000 7fb51fe00640 1 --2- 172.21.15.121:0/4001584216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb528159e10 0x7fb52815ad50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb51000c3c0 tx=0x7fb51000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:37.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.961+0000 7fb525600640 1 -- 172.21.15.121:0/4001584216 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb510017070 con 0x7fb528159e10 2024-04-20T10:21:37.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.962+0000 7fb525600640 1 -- 172.21.15.121:0/4001584216 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5100052f0 con 0x7fb528159e10 2024-04-20T10:21:37.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.962+0000 7fb525600640 1 -- 172.21.15.121:0/4001584216 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb510003850 con 0x7fb528159e10 2024-04-20T10:21:37.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.962+0000 7fb52e600640 1 -- 172.21.15.121:0/4001584216 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5280b3800 con 0x7fb528159e10 2024-04-20T10:21:37.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.962+0000 7fb52e600640 1 -- 172.21.15.121:0/4001584216 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5280b3dc0 con 0x7fb528159e10 2024-04-20T10:21:37.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.963+0000 7fb52e600640 1 -- 172.21.15.121:0/4001584216 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb528075b60 con 0x7fb528159e10 2024-04-20T10:21:37.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.964+0000 7fb525600640 1 -- 172.21.15.121:0/4001584216 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb51001a070 con 0x7fb528159e10 2024-04-20T10:21:37.964 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.965+0000 7fb525600640 1 --2- 172.21.15.121:0/4001584216 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fb50406c920 0x7fb50406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:37.964 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.965+0000 7fb525600640 1 -- 172.21.15.121:0/4001584216 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb51008b0d0 con 0x7fb528159e10 2024-04-20T10:21:37.964 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.965+0000 7fb52d200640 1 --2- 172.21.15.121:0/4001584216 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fb50406c920 0x7fb50406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:37.965 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.966+0000 7fb52d200640 1 --2- 172.21.15.121:0/4001584216 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fb50406c920 0x7fb50406ede0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb518035000 tx=0x7fb518030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:37.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:37.969+0000 7fb525600640 1 -- 172.21.15.121:0/4001584216 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb5100591b0 con 0x7fb528159e10 2024-04-20T10:21:38.212 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.213+0000 7fb52e600640 1 -- 172.21.15.121:0/4001584216 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb528160400 con 0x7fb528159e10 2024-04-20T10:21:38.213 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.214+0000 7fb525600640 1 -- 172.21.15.121:0/4001584216 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fb510058ab0 con 0x7fb528159e10 2024-04-20T10:21:38.213 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:21:38.213 INFO:teuthology.orchestra.run.smithi121.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-20T10:21:38.216 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.216+0000 7fb52e600640 1 -- 172.21.15.121:0/4001584216 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fb50406c920 msgr2=0x7fb50406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:38.216 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.216+0000 7fb52e600640 1 --2- 172.21.15.121:0/4001584216 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fb50406c920 0x7fb50406ede0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb518035000 tx=0x7fb518030000 comp rx=0 tx=0).stop 2024-04-20T10:21:38.216 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.216+0000 7fb52e600640 1 -- 172.21.15.121:0/4001584216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb528159e10 msgr2=0x7fb52815ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:38.216 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.216+0000 7fb52e600640 1 --2- 172.21.15.121:0/4001584216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb528159e10 0x7fb52815ad50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb51000c3c0 tx=0x7fb51000c880 comp rx=0 tx=0).stop 2024-04-20T10:21:38.216 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.216+0000 7fb52e600640 1 -- 172.21.15.121:0/4001584216 shutdown_connections 2024-04-20T10:21:38.217 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.216+0000 7fb52e600640 1 --2- 172.21.15.121:0/4001584216 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fb50406c920 0x7fb50406ede0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:38.217 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.216+0000 7fb52e600640 1 --2- 172.21.15.121:0/4001584216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb52815a1f0 0x7fb52815b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:38.217 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.216+0000 7fb52e600640 1 --2- 172.21.15.121:0/4001584216 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb528159e10 0x7fb52815ad50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:38.217 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.216+0000 7fb52e600640 1 --2- 172.21.15.121:0/4001584216 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb52806da90 0x7fb5280b3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:38.217 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.216+0000 7fb52e600640 1 -- 172.21.15.121:0/4001584216 >> 172.21.15.121:0/4001584216 conn(0x7fb5280d0040 msgr2=0x7fb528071a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:38.217 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.217+0000 7fb52e600640 1 -- 172.21.15.121:0/4001584216 shutdown_connections 2024-04-20T10:21:38.217 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.217+0000 7fb52e600640 1 -- 172.21.15.121:0/4001584216 wait complete. 2024-04-20T10:21:38.234 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-20T10:21:38.235 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:21:38.428 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.428+0000 7f809ea00640 1 Processor -- start 2024-04-20T10:21:38.428 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.428+0000 7f809ea00640 1 -- start start 2024-04-20T10:21:38.429 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.430+0000 7f809ea00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f80980b2f70 0x7f80980b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:38.430 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.430+0000 7f809ea00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8098071a80 con 0x7f80980adf60 2024-04-20T10:21:38.430 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.430+0000 7f809ea00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80980ae3f0 con 0x7f809806b900 2024-04-20T10:21:38.430 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.430+0000 7f809ea00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f8098071cf0 con 0x7f80980b2f70 2024-04-20T10:21:38.430 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.430+0000 7f809d600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f80980b2f70 0x7f80980b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:38.430 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.430+0000 7f809d600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f80980b2f70 0x7f80980b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58628/0 (socket says 172.21.15.121:58628) 2024-04-20T10:21:38.430 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.430+0000 7f809cc00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f80980adf60 0x7f809815f5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:47614/0 (socket says 172.21.15.121:47614) 2024-04-20T10:21:38.430 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.430+0000 7f809d600640 1 -- 172.21.15.121:0/4102664022 learned_addr learned my addr 172.21.15.121:0/4102664022 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:38.431 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.431+0000 7f809d600640 1 -- 172.21.15.121:0/4102664022 >> v1:172.21.15.123:6789/0 conn(0x7f809806b900 legacy=0x7f809806bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:38.431 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.431+0000 7f809d600640 1 -- 172.21.15.121:0/4102664022 >> v1:172.21.15.121:6789/0 conn(0x7f80980adf60 legacy=0x7f809815f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:38.431 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.431+0000 7f8097e00640 1 -- 172.21.15.121:0/4102664022 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3675353527 0 0) 0x7f8098071a80 con 0x7f80980adf60 2024-04-20T10:21:38.431 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.431+0000 7f809d600640 1 -- 172.21.15.121:0/4102664022 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8098071fd0 con 0x7f80980b2f70 2024-04-20T10:21:38.432 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.432+0000 7f809d600640 1 --2- 172.21.15.121:0/4102664022 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80980b2f70 0x7f80980b4320 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f808c009870 tx=0x7f808c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b616797422bdca0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:38.432 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.432+0000 7f8097e00640 1 -- 172.21.15.121:0/4102664022 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808c034070 con 0x7f80980b2f70 2024-04-20T10:21:38.432 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.432+0000 7f8097e00640 1 -- 172.21.15.121:0/4102664022 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f808c030860 con 0x7f80980b2f70 2024-04-20T10:21:38.432 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.432+0000 7f8097e00640 1 -- 172.21.15.121:0/4102664022 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808c030c30 con 0x7f80980b2f70 2024-04-20T10:21:38.433 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.433+0000 7f809ea00640 1 -- 172.21.15.121:0/4102664022 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80980b2f70 msgr2=0x7f80980b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:38.433 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.433+0000 7f809ea00640 1 --2- 172.21.15.121:0/4102664022 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80980b2f70 0x7f80980b4320 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f808c009870 tx=0x7f808c02b9a0 comp rx=0 tx=0).stop 2024-04-20T10:21:38.433 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.433+0000 7f809ea00640 1 -- 172.21.15.121:0/4102664022 shutdown_connections 2024-04-20T10:21:38.433 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.433+0000 7f809ea00640 1 --2- 172.21.15.121:0/4102664022 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80980b2f70 0x7f80980b4320 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:38.433 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.433+0000 7f809ea00640 1 -- 172.21.15.121:0/4102664022 >> 172.21.15.121:0/4102664022 conn(0x7f80980d0040 msgr2=0x7f80980d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:38.433 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.433+0000 7f809ea00640 1 -- 172.21.15.121:0/4102664022 shutdown_connections 2024-04-20T10:21:38.433 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.433+0000 7f809ea00640 1 -- 172.21.15.121:0/4102664022 wait complete. 2024-04-20T10:21:38.434 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.434+0000 7f809ea00640 1 Processor -- start 2024-04-20T10:21:38.434 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.435+0000 7f809ea00640 1 -- start start 2024-04-20T10:21:38.435 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.435+0000 7f809ea00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f809806b900 0x7f80981f7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:38.435 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.436+0000 7f809ea00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80980adf60 0x7f80981e4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:38.435 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.436+0000 7f809ea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80980b2f70 0x7f809815b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:38.435 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.436+0000 7f809ea00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f809815fcc0 con 0x7f80980adf60 2024-04-20T10:21:38.435 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.436+0000 7f809ea00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f80980736c0 con 0x7f80980b2f70 2024-04-20T10:21:38.435 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.436+0000 7f809ea00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f809815fe40 con 0x7f809806b900 2024-04-20T10:21:38.436 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.436+0000 7f809cc00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80980adf60 0x7f80981e4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:38.436 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.436+0000 7f809cc00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80980adf60 0x7f80981e4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58634/0 (socket says 172.21.15.121:58634) 2024-04-20T10:21:38.436 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.436+0000 7f809cc00640 1 -- 172.21.15.121:0/2435283013 learned_addr learned my addr 172.21.15.121:0/2435283013 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:38.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.436+0000 7f809e000640 1 --2- 172.21.15.121:0/2435283013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80980b2f70 0x7f809815b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:38.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.436+0000 7f809d600640 1 --2- 172.21.15.121:0/2435283013 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f809806b900 0x7f80981f7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:38.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.436+0000 7f809cc00640 1 -- 172.21.15.121:0/2435283013 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f809806b900 msgr2=0x7f80981f7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:38.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.436+0000 7f809cc00640 1 --2- 172.21.15.121:0/2435283013 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f809806b900 0x7f80981f7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:38.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.436+0000 7f809cc00640 1 -- 172.21.15.121:0/2435283013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80980b2f70 msgr2=0x7f809815b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:38.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.436+0000 7f809cc00640 1 --2- 172.21.15.121:0/2435283013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80980b2f70 0x7f809815b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:38.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.436+0000 7f809cc00640 1 -- 172.21.15.121:0/2435283013 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8098146e70 con 0x7f80980adf60 2024-04-20T10:21:38.438 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.437+0000 7f809cc00640 1 --2- 172.21.15.121:0/2435283013 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80980adf60 0x7f80981e4ec0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f808800c500 tx=0x7f808800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:38.438 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.437+0000 7f8095600640 1 -- 172.21.15.121:0/2435283013 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8088018070 con 0x7f80980adf60 2024-04-20T10:21:38.438 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.437+0000 7f8095600640 1 -- 172.21.15.121:0/2435283013 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8088007580 con 0x7f80980adf60 2024-04-20T10:21:38.438 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.437+0000 7f809ea00640 1 -- 172.21.15.121:0/2435283013 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8098147150 con 0x7f80980adf60 2024-04-20T10:21:38.438 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.437+0000 7f8095600640 1 -- 172.21.15.121:0/2435283013 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8088004a20 con 0x7f80980adf60 2024-04-20T10:21:38.438 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.437+0000 7f809ea00640 1 -- 172.21.15.121:0/2435283013 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80981479f0 con 0x7f80980adf60 2024-04-20T10:21:38.439 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.438+0000 7f809ea00640 1 -- 172.21.15.121:0/2435283013 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8098147bf0 con 0x7f80980adf60 2024-04-20T10:21:38.439 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.439+0000 7f8095600640 1 -- 172.21.15.121:0/2435283013 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f808801b070 con 0x7f80980adf60 2024-04-20T10:21:38.440 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.440+0000 7f8095600640 1 --2- 172.21.15.121:0/2435283013 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f807c06c920 0x7f807c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:38.440 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.440+0000 7f8095600640 1 -- 172.21.15.121:0/2435283013 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f808808b220 con 0x7f80980adf60 2024-04-20T10:21:38.440 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.441+0000 7f809d600640 1 --2- 172.21.15.121:0/2435283013 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f807c06c920 0x7f807c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:38.440 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.441+0000 7f809d600640 1 --2- 172.21.15.121:0/2435283013 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f807c06c920 0x7f807c06ede0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f808c036000 tx=0x7f808c03b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:38.443 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.444+0000 7f8095600640 1 -- 172.21.15.121:0/2435283013 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8098147bf0 con 0x7f80980adf60 2024-04-20T10:21:38.644 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.644+0000 7f809ea00640 1 -- 172.21.15.121:0/2435283013 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f80980afd10 con 0x7f80980adf60 2024-04-20T10:21:38.646 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.647+0000 7f8095600640 1 -- 172.21.15.121:0/2435283013 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7f8088059270 con 0x7f80980adf60 2024-04-20T10:21:38.646 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:21:38.647 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":25,"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","created":"2024-04-20T10:20:38.076076+0000","modified":"2024-04-20T10:21:07.336542+0000","last_up_change":"2024-04-20T10:20:50.162700+0000","last_in_change":"2024-04-20T10:20:41.510382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:20:51.141365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:21:00.436937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:21:01.026957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:21:04.862693+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"629a7b8c-a40a-4a2a-a3cc-d9f357774310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6809","nonce":1044588835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6811","nonce":1044588835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6815","nonce":1044588835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6813","nonce":1044588835}]},"public_addr":"172.21.15.121:6809/1044588835","cluster_addr":"172.21.15.121:6811/1044588835","heartbeat_back_addr":"172.21.15.121:6815/1044588835","heartbeat_front_addr":"172.21.15.121:6813/1044588835","state":["exists","up"]},{"osd":1,"uuid":"f2570399-aa50-4ff3-bf5d-3d09207290db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6801","nonce":3505199624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6803","nonce":3505199624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6807","nonce":3505199624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6805","nonce":3505199624}]},"public_addr":"172.21.15.121:6801/3505199624","cluster_addr":"172.21.15.121:6803/3505199624","heartbeat_back_addr":"172.21.15.121:6807/3505199624","heartbeat_front_addr":"172.21.15.121:6805/3505199624","state":["exists","up"]},{"osd":2,"uuid":"50893807-dc30-49f4-b41f-739366626658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6817","nonce":3656724207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6819","nonce":3656724207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6827","nonce":3656724207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6822","nonce":3656724207}]},"public_addr":"172.21.15.121:6817/3656724207","cluster_addr":"172.21.15.121:6819/3656724207","heartbeat_back_addr":"172.21.15.121:6827/3656724207","heartbeat_front_addr":"172.21.15.121:6822/3656724207","state":["exists","up"]},{"osd":3,"uuid":"dadb01bc-5491-4ffc-8824-d48a057240b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6823","nonce":2232342832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6826","nonce":2232342832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6831","nonce":2232342832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6829","nonce":2232342832}]},"public_addr":"172.21.15.121:6823/2232342832","cluster_addr":"172.21.15.121:6826/2232342832","heartbeat_back_addr":"172.21.15.121:6831/2232342832","heartbeat_front_addr":"172.21.15.121:6829/2232342832","state":["exists","up"]},{"osd":4,"uuid":"b3f3b4a1-b375-4526-b69e-344ab24c2da1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6801","nonce":99353111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6803","nonce":99353111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6807","nonce":99353111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6805","nonce":99353111}]},"public_addr":"172.21.15.123:6801/99353111","cluster_addr":"172.21.15.123:6803/99353111","heartbeat_back_addr":"172.21.15.123:6807/99353111","heartbeat_front_addr":"172.21.15.123:6805/99353111","state":["exists","up"]},{"osd":5,"uuid":"2ee24a08-1624-4961-9642-d7b510282484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6818","nonce":2504374688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6822","nonce":2504374688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6830","nonce":2504374688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6826","nonce":2504374688}]},"public_addr":"172.21.15.123:6818/2504374688","cluster_addr":"172.21.15.123:6822/2504374688","heartbeat_back_addr":"172.21.15.123:6830/2504374688","heartbeat_front_addr":"172.21.15.123:6826/2504374688","state":["exists","up"]},{"osd":6,"uuid":"186118cc-6128-49de-acca-55260945c1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6819","nonce":4139299599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6823","nonce":4139299599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6829","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6831","nonce":4139299599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6827","nonce":4139299599}]},"public_addr":"172.21.15.123:6819/4139299599","cluster_addr":"172.21.15.123:6823/4139299599","heartbeat_back_addr":"172.21.15.123:6831/4139299599","heartbeat_front_addr":"172.21.15.123:6827/4139299599","state":["exists","up"]},{"osd":7,"uuid":"a00aac6e-4bd9-415a-a93d-ae85210865ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6809","nonce":4043981069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6811","nonce":4043981069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6815","nonce":4043981069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6813","nonce":4043981069}]},"public_addr":"172.21.15.123:6809/4043981069","cluster_addr":"172.21.15.123:6811/4043981069","heartbeat_back_addr":"172.21.15.123:6815/4043981069","heartbeat_front_addr":"172.21.15.123:6813/4043981069","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.904540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.958585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.616367+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.885244+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.103127+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.167248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.884348+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.862356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:21:38.649 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.649+0000 7f809ea00640 1 -- 172.21.15.121:0/2435283013 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f807c06c920 msgr2=0x7f807c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:38.649 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.649+0000 7f809ea00640 1 --2- 172.21.15.121:0/2435283013 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f807c06c920 0x7f807c06ede0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f808c036000 tx=0x7f808c03b040 comp rx=0 tx=0).stop 2024-04-20T10:21:38.649 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.649+0000 7f809ea00640 1 -- 172.21.15.121:0/2435283013 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80980adf60 msgr2=0x7f80981e4ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:38.649 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.649+0000 7f809ea00640 1 --2- 172.21.15.121:0/2435283013 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80980adf60 0x7f80981e4ec0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f808800c500 tx=0x7f808800c9c0 comp rx=0 tx=0).stop 2024-04-20T10:21:38.649 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.649+0000 7f809ea00640 1 -- 172.21.15.121:0/2435283013 shutdown_connections 2024-04-20T10:21:38.649 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.649+0000 7f809ea00640 1 --2- 172.21.15.121:0/2435283013 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f807c06c920 0x7f807c06ede0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:38.649 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.649+0000 7f809ea00640 1 --2- 172.21.15.121:0/2435283013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80980b2f70 0x7f809815b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:38.649 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.649+0000 7f809ea00640 1 --2- 172.21.15.121:0/2435283013 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f80980adf60 0x7f80981e4ec0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:38.649 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.649+0000 7f809ea00640 1 --2- 172.21.15.121:0/2435283013 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f809806b900 0x7f80981f7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:38.650 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.649+0000 7f809ea00640 1 -- 172.21.15.121:0/2435283013 >> 172.21.15.121:0/2435283013 conn(0x7f80980d0040 msgr2=0x7f809806c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:38.650 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.649+0000 7f809ea00640 1 -- 172.21.15.121:0/2435283013 shutdown_connections 2024-04-20T10:21:38.650 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.649+0000 7f809ea00640 1 -- 172.21.15.121:0/2435283013 wait complete. 2024-04-20T10:21:38.666 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:20:51.141365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:21:00.436937+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:21:01.026957+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T10:21:04.862693+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T10:21:38.667 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:21:38.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.863+0000 7ffa0a400640 1 Processor -- start 2024-04-20T10:21:38.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.863+0000 7ffa0a400640 1 -- start start 2024-04-20T10:21:38.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.865+0000 7ffa0a400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ffa040cfde0 0x7ffa0415ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:38.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.865+0000 7ffa0a400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa04158780 0x7ffa04158c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:38.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.865+0000 7ffa0a400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa040ce4a0 con 0x7ffa040cfa00 2024-04-20T10:21:38.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.865+0000 7ffa0a400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ffa040d0680 con 0x7ffa040cfde0 2024-04-20T10:21:38.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.865+0000 7ffa0a400640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ffa04159170 con 0x7ffa04158780 2024-04-20T10:21:38.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.865+0000 7ffa03400640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7ffa040cfa00 0x7ffa040cdcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:47618/0 (socket says 172.21.15.121:47618) 2024-04-20T10:21:38.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.865+0000 7ffa03400640 1 -- 172.21.15.121:0/1558687164 learned_addr learned my addr 172.21.15.121:0/1558687164 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:38.867 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.865+0000 7ffa02a00640 1 --2- 172.21.15.121:0/1558687164 >> v2:172.21.15.123:3300/0 conn(0x7ffa040cfde0 0x7ffa0415ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:38.867 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.865+0000 7ffa03e00640 1 --2- 172.21.15.121:0/1558687164 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa04158780 0x7ffa04158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:38.867 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.866+0000 7ffa02000640 1 -- 172.21.15.121:0/1558687164 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3940926908 0 0) 0x7ffa040ce4a0 con 0x7ffa040cfa00 2024-04-20T10:21:38.867 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.866+0000 7ffa02000640 1 -- 172.21.15.121:0/1558687164 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff9e4003520 con 0x7ffa040cfa00 2024-04-20T10:21:38.867 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.866+0000 7ffa02000640 1 -- 172.21.15.121:0/1558687164 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3251898145 0 0) 0x7ff9e4003520 con 0x7ffa040cfa00 2024-04-20T10:21:38.867 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.866+0000 7ffa02000640 1 -- 172.21.15.121:0/1558687164 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa04158780 msgr2=0x7ffa04158c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:38.867 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.866+0000 7ffa02000640 1 --2- 172.21.15.121:0/1558687164 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa04158780 0x7ffa04158c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:38.867 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.866+0000 7ffa02000640 1 -- 172.21.15.121:0/1558687164 >> v2:172.21.15.123:3300/0 conn(0x7ffa040cfde0 msgr2=0x7ffa0415ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:38.867 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.866+0000 7ffa02000640 1 --2- 172.21.15.121:0/1558687164 >> v2:172.21.15.123:3300/0 conn(0x7ffa040cfde0 0x7ffa0415ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:38.867 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.866+0000 7ffa02000640 1 -- 172.21.15.121:0/1558687164 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa040ce780 con 0x7ffa040cfa00 2024-04-20T10:21:38.868 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.866+0000 7ffa02000640 1 -- 172.21.15.121:0/1558687164 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7ff9f4001670 con 0x7ffa040cfa00 2024-04-20T10:21:38.868 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.866+0000 7ffa02000640 1 -- 172.21.15.121:0/1558687164 >> v1:172.21.15.121:6789/0 conn(0x7ffa040cfa00 legacy=0x7ffa040cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:38.868 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.866+0000 7ffa02000640 1 --2- 172.21.15.121:0/1558687164 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa04158780 0x7ff9e40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:38.868 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.866+0000 7ffa02000640 1 --2- 172.21.15.121:0/1558687164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9e4003f30 0x7ff9e4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:38.868 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.866+0000 7ffa02000640 1 --2- 172.21.15.121:0/1558687164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9e4004d40 0x7ff9e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:38.868 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.866+0000 7ffa02000640 1 -- 172.21.15.121:0/1558687164 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ffa040d0680 con 0x7ff9e4004d40 2024-04-20T10:21:38.868 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.866+0000 7ffa02000640 1 -- 172.21.15.121:0/1558687164 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffa04159170 con 0x7ff9e4003f30 2024-04-20T10:21:38.868 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.866+0000 7ffa02000640 1 -- 172.21.15.121:0/1558687164 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ff9e400ba10 con 0x7ffa04158780 2024-04-20T10:21:38.868 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.866+0000 7ffa03e00640 1 --2- 172.21.15.121:0/1558687164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9e4004d40 0x7ff9e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:38.868 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.867+0000 7ffa02a00640 1 --2- 172.21.15.121:0/1558687164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9e4003f30 0x7ff9e4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:38.868 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.867+0000 7ffa03400640 1 --2- 172.21.15.121:0/1558687164 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa04158780 0x7ff9e40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:38.868 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.867+0000 7ffa03e00640 1 -- 172.21.15.121:0/1558687164 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa04158780 msgr2=0x7ff9e40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:38.868 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.867+0000 7ffa03e00640 1 --2- 172.21.15.121:0/1558687164 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa04158780 0x7ff9e40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:38.868 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.867+0000 7ffa03e00640 1 -- 172.21.15.121:0/1558687164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9e4003f30 msgr2=0x7ff9e4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:38.869 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.867+0000 7ffa03e00640 1 --2- 172.21.15.121:0/1558687164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9e4003f30 0x7ff9e4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:38.869 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.867+0000 7ffa03e00640 1 -- 172.21.15.121:0/1558687164 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e4003520 con 0x7ff9e4004d40 2024-04-20T10:21:38.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.868+0000 7ffa03e00640 1 --2- 172.21.15.121:0/1558687164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9e4004d40 0x7ff9e40051c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc009af0 tx=0x7ff9e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a7efcc0440bfe63 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:38.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.868+0000 7ffa02000640 1 -- 172.21.15.121:0/1558687164 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc039070 con 0x7ff9e4004d40 2024-04-20T10:21:38.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.868+0000 7ffa02000640 1 -- 172.21.15.121:0/1558687164 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9fc02c6b0 con 0x7ff9e4004d40 2024-04-20T10:21:38.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.868+0000 7ffa02000640 1 -- 172.21.15.121:0/1558687164 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc009690 con 0x7ff9e4004d40 2024-04-20T10:21:38.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.869+0000 7ffa0a400640 1 -- 172.21.15.121:0/1558687164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9e4004d40 msgr2=0x7ff9e40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:38.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.869+0000 7ffa0a400640 1 --2- 172.21.15.121:0/1558687164 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9e4004d40 0x7ff9e40051c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc009af0 tx=0x7ff9e4005830 comp rx=0 tx=0).stop 2024-04-20T10:21:38.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.869+0000 7ffa03e00640 1 -- 172.21.15.121:0/1558687164 reap_dead start 2024-04-20T10:21:38.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.869+0000 7ffa0a400640 1 -- 172.21.15.121:0/1558687164 shutdown_connections 2024-04-20T10:21:38.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.869+0000 7ffa0a400640 1 -- 172.21.15.121:0/1558687164 >> 172.21.15.121:0/1558687164 conn(0x7ffa04070450 msgr2=0x7ffa04070860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:38.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.869+0000 7ffa0a400640 1 -- 172.21.15.121:0/1558687164 shutdown_connections 2024-04-20T10:21:38.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.869+0000 7ffa0a400640 1 -- 172.21.15.121:0/1558687164 wait complete. 2024-04-20T10:21:38.871 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.871+0000 7ffa0a400640 1 Processor -- start 2024-04-20T10:21:38.871 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.871+0000 7ffa0a400640 1 -- start start 2024-04-20T10:21:38.872 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.872+0000 7ffa0a400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9e4004d40 0x7ffa04146d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:38.872 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.872+0000 7ffa0a400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa040b1ee0 0x7ffa040b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:38.872 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.872+0000 7ffa0a400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa040b2970 0x7ffa041e9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:38.872 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.872+0000 7ffa0a400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ffa040b4350 con 0x7ff9e4004d40 2024-04-20T10:21:38.872 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.872+0000 7ffa0a400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffa040ced40 con 0x7ffa040b2970 2024-04-20T10:21:38.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.872+0000 7ffa0a400640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ffa04158c30 con 0x7ffa040b1ee0 2024-04-20T10:21:38.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.872+0000 7ffa03400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9e4004d40 0x7ffa04146d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:38.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.872+0000 7ffa03400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9e4004d40 0x7ffa04146d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58644/0 (socket says 172.21.15.121:58644) 2024-04-20T10:21:38.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.872+0000 7ffa03400640 1 -- 172.21.15.121:0/3052439196 learned_addr learned my addr 172.21.15.121:0/3052439196 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:38.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.873+0000 7ffa03400640 1 -- 172.21.15.121:0/3052439196 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa040b1ee0 msgr2=0x7ffa040b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:38.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.873+0000 7ffa03400640 1 --2- 172.21.15.121:0/3052439196 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa040b1ee0 0x7ffa040b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:38.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.873+0000 7ffa03400640 1 -- 172.21.15.121:0/3052439196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa040b2970 msgr2=0x7ffa041e9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:38.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.873+0000 7ffa03400640 1 --2- 172.21.15.121:0/3052439196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa040b2970 0x7ffa041e9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:38.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.873+0000 7ffa03400640 1 -- 172.21.15.121:0/3052439196 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa040b0e40 con 0x7ff9e4004d40 2024-04-20T10:21:38.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.873+0000 7ffa03400640 1 --2- 172.21.15.121:0/3052439196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9e4004d40 0x7ffa04146d90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400e920 tx=0x7ff9f400ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:38.874 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.873+0000 7ff9fb400640 1 -- 172.21.15.121:0/3052439196 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f400c930 con 0x7ff9e4004d40 2024-04-20T10:21:38.874 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.873+0000 7ff9fb400640 1 -- 172.21.15.121:0/3052439196 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9f400caf0 con 0x7ff9e4004d40 2024-04-20T10:21:38.874 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.873+0000 7ffa0a400640 1 -- 172.21.15.121:0/3052439196 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa040b1120 con 0x7ff9e4004d40 2024-04-20T10:21:38.874 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.873+0000 7ff9fb400640 1 -- 172.21.15.121:0/3052439196 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9f400cce0 con 0x7ff9e4004d40 2024-04-20T10:21:38.874 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.873+0000 7ffa0a400640 1 -- 172.21.15.121:0/3052439196 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa040b16e0 con 0x7ff9e4004d40 2024-04-20T10:21:38.874 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.875+0000 7ffa0a400640 1 -- 172.21.15.121:0/3052439196 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa04068980 con 0x7ff9e4004d40 2024-04-20T10:21:38.875 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.875+0000 7ff9fb400640 1 -- 172.21.15.121:0/3052439196 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff9f401a020 con 0x7ff9e4004d40 2024-04-20T10:21:38.877 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.877+0000 7ff9fb400640 1 --2- 172.21.15.121:0/3052439196 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff9e006c8f0 0x7ff9e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:38.877 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.878+0000 7ff9fb400640 1 -- 172.21.15.121:0/3052439196 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff9f408b500 con 0x7ff9e4004d40 2024-04-20T10:21:38.877 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.878+0000 7ffa02a00640 1 --2- 172.21.15.121:0/3052439196 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff9e006c8f0 0x7ff9e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:38.877 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.878+0000 7ffa02a00640 1 --2- 172.21.15.121:0/3052439196 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff9e006c8f0 0x7ff9e006edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ffa041f7d20 tx=0x7ff9ec009750 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:38.882 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:38.883+0000 7ff9fb400640 1 -- 172.21.15.121:0/3052439196 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff9f4021020 con 0x7ff9e4004d40 2024-04-20T10:21:39.088 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.089+0000 7ffa0a400640 1 -- 172.21.15.121:0/3052439196 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ffa04067ce0 con 0x7ff9e4004d40 2024-04-20T10:21:39.089 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.090+0000 7ff9fb400640 1 -- 172.21.15.121:0/3052439196 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7ff9f40595f0 con 0x7ff9e4004d40 2024-04-20T10:21:39.089 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2024-04-20T10:21:39.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.092+0000 7ffa0a400640 1 -- 172.21.15.121:0/3052439196 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff9e006c8f0 msgr2=0x7ff9e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:39.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.092+0000 7ffa0a400640 1 --2- 172.21.15.121:0/3052439196 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff9e006c8f0 0x7ff9e006edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ffa041f7d20 tx=0x7ff9ec009750 comp rx=0 tx=0).stop 2024-04-20T10:21:39.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.092+0000 7ffa0a400640 1 -- 172.21.15.121:0/3052439196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9e4004d40 msgr2=0x7ffa04146d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:39.093 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.092+0000 7ffa0a400640 1 --2- 172.21.15.121:0/3052439196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9e4004d40 0x7ffa04146d90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff9f400e920 tx=0x7ff9f400ede0 comp rx=0 tx=0).stop 2024-04-20T10:21:39.093 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.093+0000 7ffa0a400640 1 -- 172.21.15.121:0/3052439196 shutdown_connections 2024-04-20T10:21:39.093 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.093+0000 7ffa0a400640 1 --2- 172.21.15.121:0/3052439196 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff9e006c8f0 0x7ff9e006edb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:39.093 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.093+0000 7ffa0a400640 1 --2- 172.21.15.121:0/3052439196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa040b2970 0x7ffa041e9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:39.093 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.093+0000 7ffa0a400640 1 --2- 172.21.15.121:0/3052439196 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffa040b1ee0 0x7ffa040b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:39.093 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.093+0000 7ffa0a400640 1 --2- 172.21.15.121:0/3052439196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff9e4004d40 0x7ffa04146d90 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:39.093 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.093+0000 7ffa0a400640 1 -- 172.21.15.121:0/3052439196 >> 172.21.15.121:0/3052439196 conn(0x7ffa04070450 msgr2=0x7ffa040b5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:39.093 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.093+0000 7ffa0a400640 1 -- 172.21.15.121:0/3052439196 shutdown_connections 2024-04-20T10:21:39.093 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.093+0000 7ffa0a400640 1 -- 172.21.15.121:0/3052439196 wait complete. 2024-04-20T10:21:39.110 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:21:39.310 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.310+0000 7f7b92200640 1 Processor -- start 2024-04-20T10:21:39.310 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.310+0000 7f7b92200640 1 -- start start 2024-04-20T10:21:39.311 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.312+0000 7f7b92200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7b8c156c20 0x7f7b8c06b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:39.311 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.312+0000 7f7b92200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7b8c0d0360 0x7f7b8c0d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:39.312 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.312+0000 7f7b92200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b8c141c60 con 0x7f7b8c15ca40 2024-04-20T10:21:39.312 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.312+0000 7f7b92200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f7b8c0adab0 con 0x7f7b8c156c20 2024-04-20T10:21:39.312 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.312+0000 7f7b92200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f7b8c15cf90 con 0x7f7b8c0d0360 2024-04-20T10:21:39.313 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.312+0000 7f7b8b400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7b8c156c20 0x7f7b8c06b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:39.313 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.313+0000 7f7b8b400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7b8c156c20 0x7f7b8c06b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58650/0 (socket says 172.21.15.121:58650) 2024-04-20T10:21:39.313 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.313+0000 7f7b8b400640 1 -- 172.21.15.121:0/2215692726 learned_addr learned my addr 172.21.15.121:0/2215692726 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:39.313 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.313+0000 7f7b8be00640 1 --2- 172.21.15.121:0/2215692726 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7b8c0d0360 0x7f7b8c0d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:39.313 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.313+0000 7f7b8b400640 1 -- 172.21.15.121:0/2215692726 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7b8c0d0360 msgr2=0x7f7b8c0d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:39.313 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.313+0000 7f7b8b400640 1 --2- 172.21.15.121:0/2215692726 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7b8c0d0360 0x7f7b8c0d07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:39.313 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.313+0000 7f7b8b400640 1 -- 172.21.15.121:0/2215692726 >> v1:172.21.15.121:6789/0 conn(0x7f7b8c15ca40 legacy=0x7f7b8c0ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:39.313 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.313+0000 7f7b8aa00640 1 -- 172.21.15.121:0/2215692726 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2982765906 0 0) 0x7f7b8c141c60 con 0x7f7b8c15ca40 2024-04-20T10:21:39.313 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.313+0000 7f7b8b400640 1 -- 172.21.15.121:0/2215692726 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b8c15d270 con 0x7f7b8c156c20 2024-04-20T10:21:39.314 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.315+0000 7f7b8b400640 1 --2- 172.21.15.121:0/2215692726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b8c156c20 0x7f7b8c06b900 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c009870 tx=0x7f7b7c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=889c1bb05f4daa31 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:39.315 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.315+0000 7f7b8aa00640 1 -- 172.21.15.121:0/2215692726 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c034070 con 0x7f7b8c156c20 2024-04-20T10:21:39.315 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.315+0000 7f7b8aa00640 1 -- 172.21.15.121:0/2215692726 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b7c030840 con 0x7f7b8c156c20 2024-04-20T10:21:39.315 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.315+0000 7f7b8aa00640 1 -- 172.21.15.121:0/2215692726 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c030c10 con 0x7f7b8c156c20 2024-04-20T10:21:39.316 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.316+0000 7f7b92200640 1 -- 172.21.15.121:0/2215692726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b8c156c20 msgr2=0x7f7b8c06b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:39.316 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.316+0000 7f7b92200640 1 --2- 172.21.15.121:0/2215692726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b8c156c20 0x7f7b8c06b900 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c009870 tx=0x7f7b7c02b980 comp rx=0 tx=0).stop 2024-04-20T10:21:39.316 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.316+0000 7f7b92200640 1 -- 172.21.15.121:0/2215692726 shutdown_connections 2024-04-20T10:21:39.316 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.316+0000 7f7b92200640 1 --2- 172.21.15.121:0/2215692726 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7b8c0d0360 0x7f7b8c0d07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:39.316 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.316+0000 7f7b92200640 1 --2- 172.21.15.121:0/2215692726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b8c156c20 0x7f7b8c06b900 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:39.316 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.316+0000 7f7b92200640 1 -- 172.21.15.121:0/2215692726 >> 172.21.15.121:0/2215692726 conn(0x7f7b8c0b3350 msgr2=0x7f7b8c0b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:39.316 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.316+0000 7f7b92200640 1 -- 172.21.15.121:0/2215692726 shutdown_connections 2024-04-20T10:21:39.316 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.316+0000 7f7b92200640 1 -- 172.21.15.121:0/2215692726 wait complete. 2024-04-20T10:21:39.317 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.318+0000 7f7b92200640 1 Processor -- start 2024-04-20T10:21:39.317 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.318+0000 7f7b92200640 1 -- start start 2024-04-20T10:21:39.319 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.319+0000 7f7b92200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7b8c0d0360 0x7f7b8c099a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:39.319 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.319+0000 7f7b92200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b8c156c20 0x7f7b8c06f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:39.319 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.319+0000 7f7b92200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b8c15ca40 0x7f7b8c1e5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:39.319 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.319+0000 7f7b92200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8c06be40 con 0x7f7b8c156c20 2024-04-20T10:21:39.319 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.319+0000 7f7b92200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7b8c15d830 con 0x7f7b8c15ca40 2024-04-20T10:21:39.319 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.319+0000 7f7b92200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f7b8c06bfc0 con 0x7f7b8c0d0360 2024-04-20T10:21:39.319 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.319+0000 7f7b83e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b8c156c20 0x7f7b8c06f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:39.319 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.319+0000 7f7b83e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b8c156c20 0x7f7b8c06f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58660/0 (socket says 172.21.15.121:58660) 2024-04-20T10:21:39.319 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.319+0000 7f7b83e00640 1 -- 172.21.15.121:0/1772069619 learned_addr learned my addr 172.21.15.121:0/1772069619 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:39.320 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.319+0000 7f7b83e00640 1 -- 172.21.15.121:0/1772069619 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7b8c0d0360 msgr2=0x7f7b8c099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:39.320 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.320+0000 7f7b83e00640 1 --2- 172.21.15.121:0/1772069619 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7b8c0d0360 0x7f7b8c099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:39.320 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.319+0000 7f7b8be00640 1 --2- 172.21.15.121:0/1772069619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b8c15ca40 0x7f7b8c1e5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:39.320 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.320+0000 7f7b83e00640 1 -- 172.21.15.121:0/1772069619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b8c15ca40 msgr2=0x7f7b8c1e5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:39.320 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.320+0000 7f7b83e00640 1 --2- 172.21.15.121:0/1772069619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b8c15ca40 0x7f7b8c1e5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:39.320 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.320+0000 7f7b83e00640 1 -- 172.21.15.121:0/1772069619 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b8c06e660 con 0x7f7b8c156c20 2024-04-20T10:21:39.320 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.320+0000 7f7b8be00640 1 --2- 172.21.15.121:0/1772069619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b8c15ca40 0x7f7b8c1e5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:39.320 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.320+0000 7f7b83e00640 1 --2- 172.21.15.121:0/1772069619 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b8c156c20 0x7f7b8c06f6c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7b7400c410 tx=0x7f7b7400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:39.321 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.320+0000 7f7b83400640 1 -- 172.21.15.121:0/1772069619 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b74017070 con 0x7f7b8c156c20 2024-04-20T10:21:39.321 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.320+0000 7f7b83400640 1 -- 172.21.15.121:0/1772069619 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b740035f0 con 0x7f7b8c156c20 2024-04-20T10:21:39.321 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.320+0000 7f7b83400640 1 -- 172.21.15.121:0/1772069619 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b740039e0 con 0x7f7b8c156c20 2024-04-20T10:21:39.321 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.320+0000 7f7b92200640 1 -- 172.21.15.121:0/1772069619 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b8c06e900 con 0x7f7b8c156c20 2024-04-20T10:21:39.321 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.320+0000 7f7b92200640 1 -- 172.21.15.121:0/1772069619 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b8c06eec0 con 0x7f7b8c156c20 2024-04-20T10:21:39.321 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.321+0000 7f7b92200640 1 -- 172.21.15.121:0/1772069619 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b8c068980 con 0x7f7b8c156c20 2024-04-20T10:21:39.321 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.322+0000 7f7b83400640 1 -- 172.21.15.121:0/1772069619 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7b7401a070 con 0x7f7b8c156c20 2024-04-20T10:21:39.323 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.324+0000 7f7b83400640 1 --2- 172.21.15.121:0/1772069619 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7b6806c8f0 0x7f7b6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:39.323 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.324+0000 7f7b83400640 1 -- 172.21.15.121:0/1772069619 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7b7408b3c0 con 0x7f7b8c156c20 2024-04-20T10:21:39.323 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.324+0000 7f7b8b400640 1 --2- 172.21.15.121:0/1772069619 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7b6806c8f0 0x7f7b6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:39.324 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.325+0000 7f7b8b400640 1 --2- 172.21.15.121:0/1772069619 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7b6806c8f0 0x7f7b6806edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c036000 tx=0x7f7b7c03b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:39.327 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.328+0000 7f7b83400640 1 -- 172.21.15.121:0/1772069619 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7b74059410 con 0x7f7b8c156c20 2024-04-20T10:21:39.530 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.530+0000 7f7b92200640 1 -- 172.21.15.121:0/1772069619 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7b8c0cfd90 con 0x7f7b8c156c20 2024-04-20T10:21:39.531 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.531+0000 7f7b83400640 1 -- 172.21.15.121:0/1772069619 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f7b74058d10 con 0x7f7b8c156c20 2024-04-20T10:21:39.531 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 32 2024-04-20T10:21:39.534 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.534+0000 7f7b92200640 1 -- 172.21.15.121:0/1772069619 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7b6806c8f0 msgr2=0x7f7b6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:39.534 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.534+0000 7f7b92200640 1 --2- 172.21.15.121:0/1772069619 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7b6806c8f0 0x7f7b6806edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c036000 tx=0x7f7b7c03b040 comp rx=0 tx=0).stop 2024-04-20T10:21:39.534 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.534+0000 7f7b92200640 1 -- 172.21.15.121:0/1772069619 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b8c156c20 msgr2=0x7f7b8c06f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:39.534 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.534+0000 7f7b92200640 1 --2- 172.21.15.121:0/1772069619 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b8c156c20 0x7f7b8c06f6c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7b7400c410 tx=0x7f7b7400c8d0 comp rx=0 tx=0).stop 2024-04-20T10:21:39.534 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.534+0000 7f7b92200640 1 -- 172.21.15.121:0/1772069619 shutdown_connections 2024-04-20T10:21:39.534 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.534+0000 7f7b92200640 1 --2- 172.21.15.121:0/1772069619 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7b6806c8f0 0x7f7b6806edb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:39.534 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.534+0000 7f7b92200640 1 --2- 172.21.15.121:0/1772069619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b8c15ca40 0x7f7b8c1e5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:39.534 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.534+0000 7f7b92200640 1 --2- 172.21.15.121:0/1772069619 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7b8c156c20 0x7f7b8c06f6c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:39.534 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.534+0000 7f7b92200640 1 --2- 172.21.15.121:0/1772069619 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7b8c0d0360 0x7f7b8c099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:39.534 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.534+0000 7f7b92200640 1 -- 172.21.15.121:0/1772069619 >> 172.21.15.121:0/1772069619 conn(0x7f7b8c0b3350 msgr2=0x7f7b8c15ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:39.535 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.535+0000 7f7b92200640 1 -- 172.21.15.121:0/1772069619 shutdown_connections 2024-04-20T10:21:39.535 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.535+0000 7f7b92200640 1 -- 172.21.15.121:0/1772069619 wait complete. 2024-04-20T10:21:39.551 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:21:39.756 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.756+0000 7fe468a00640 1 Processor -- start 2024-04-20T10:21:39.756 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.756+0000 7fe468a00640 1 -- start start 2024-04-20T10:21:39.757 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.758+0000 7fe468a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe46415abb0 0x7fe4640725b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:39.758 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.758+0000 7fe468a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe46415f550 con 0x7fe4640af800 2024-04-20T10:21:39.758 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.758+0000 7fe468a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe46406e840 con 0x7fe464071b50 2024-04-20T10:21:39.758 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.758+0000 7fe468a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fe46406eb00 con 0x7fe46415abb0 2024-04-20T10:21:39.758 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.758+0000 7fe462800640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fe4640af800 0x7fe46415ee40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:47642/0 (socket says 172.21.15.121:47642) 2024-04-20T10:21:39.758 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.758+0000 7fe462800640 1 -- 172.21.15.121:0/3293264522 learned_addr learned my addr 172.21.15.121:0/3293264522 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:39.761 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.758+0000 7fe461e00640 1 --2- 172.21.15.121:0/3293264522 >> v2:172.21.15.123:3300/0 conn(0x7fe46415abb0 0x7fe4640725b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:39.761 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.758+0000 7fe460a00640 1 -- 172.21.15.121:0/3293264522 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2139605218 0 0) 0x7fe46415f550 con 0x7fe4640af800 2024-04-20T10:21:39.761 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.759+0000 7fe460a00640 1 -- 172.21.15.121:0/3293264522 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe448003520 con 0x7fe4640af800 2024-04-20T10:21:39.761 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.759+0000 7fe460a00640 1 -- 172.21.15.121:0/3293264522 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3996973380 0 0) 0x7fe448003520 con 0x7fe4640af800 2024-04-20T10:21:39.761 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.759+0000 7fe460a00640 1 -- 172.21.15.121:0/3293264522 >> v2:172.21.15.123:3300/0 conn(0x7fe46415abb0 msgr2=0x7fe4640725b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:39.761 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.759+0000 7fe460a00640 1 --2- 172.21.15.121:0/3293264522 >> v2:172.21.15.123:3300/0 conn(0x7fe46415abb0 0x7fe4640725b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:39.761 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.759+0000 7fe460a00640 1 -- 172.21.15.121:0/3293264522 >> v1:172.21.15.123:6789/0 conn(0x7fe464071b50 legacy=0x7fe46415e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:39.761 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.759+0000 7fe460a00640 1 -- 172.21.15.121:0/3293264522 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe46415b350 con 0x7fe4640af800 2024-04-20T10:21:39.761 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.759+0000 7fe460a00640 1 -- 172.21.15.121:0/3293264522 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7fe45c0019f0 con 0x7fe4640af800 2024-04-20T10:21:39.761 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.759+0000 7fe460a00640 1 -- 172.21.15.121:0/3293264522 >> v1:172.21.15.121:6789/0 conn(0x7fe4640af800 legacy=0x7fe46415ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:39.762 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.759+0000 7fe460a00640 1 --2- 172.21.15.121:0/3293264522 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe448003a00 0x7fe448003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:39.762 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.759+0000 7fe460a00640 1 --2- 172.21.15.121:0/3293264522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe448004800 0x7fe448004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:39.762 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.759+0000 7fe460a00640 1 --2- 172.21.15.121:0/3293264522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe448005210 0x7fe44800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:39.762 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.759+0000 7fe460a00640 1 -- 172.21.15.121:0/3293264522 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe44800c130 con 0x7fe448005210 2024-04-20T10:21:39.762 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.759+0000 7fe460a00640 1 -- 172.21.15.121:0/3293264522 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe44800c2e0 con 0x7fe448004800 2024-04-20T10:21:39.762 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.759+0000 7fe460a00640 1 -- 172.21.15.121:0/3293264522 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe44800c4e0 con 0x7fe448003a00 2024-04-20T10:21:39.762 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.759+0000 7fe462800640 1 --2- 172.21.15.121:0/3293264522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe448005210 0x7fe44800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:39.762 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.760+0000 7fe461e00640 1 --2- 172.21.15.121:0/3293264522 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe448003a00 0x7fe448003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:39.762 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.760+0000 7fe461400640 1 --2- 172.21.15.121:0/3293264522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe448004800 0x7fe448004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:39.762 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.760+0000 7fe462800640 1 -- 172.21.15.121:0/3293264522 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe448003a00 msgr2=0x7fe448003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:39.762 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.760+0000 7fe462800640 1 --2- 172.21.15.121:0/3293264522 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe448003a00 0x7fe448003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:39.762 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.760+0000 7fe462800640 1 -- 172.21.15.121:0/3293264522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe448004800 msgr2=0x7fe448004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:39.762 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.760+0000 7fe462800640 1 --2- 172.21.15.121:0/3293264522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe448004800 0x7fe448004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:39.762 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.760+0000 7fe462800640 1 -- 172.21.15.121:0/3293264522 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe448003520 con 0x7fe448005210 2024-04-20T10:21:39.763 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.761+0000 7fe462800640 1 -- 172.21.15.121:0/3293264522 reap_dead start 2024-04-20T10:21:39.763 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.761+0000 7fe462800640 1 --2- 172.21.15.121:0/3293264522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe448005210 0x7fe44800b9c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe45c00a760 tx=0x7fe448005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8753804558c8b94c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:39.763 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.761+0000 7fe460a00640 1 -- 172.21.15.121:0/3293264522 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45c00a3b0 con 0x7fe448005210 2024-04-20T10:21:39.763 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.761+0000 7fe460a00640 1 -- 172.21.15.121:0/3293264522 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe45c034630 con 0x7fe448005210 2024-04-20T10:21:39.763 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.761+0000 7fe460a00640 1 -- 172.21.15.121:0/3293264522 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45c034a20 con 0x7fe448005210 2024-04-20T10:21:39.763 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.762+0000 7fe468a00640 1 -- 172.21.15.121:0/3293264522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe448005210 msgr2=0x7fe44800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:39.764 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.762+0000 7fe468a00640 1 --2- 172.21.15.121:0/3293264522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe448005210 0x7fe44800b9c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe45c00a760 tx=0x7fe448005830 comp rx=0 tx=0).stop 2024-04-20T10:21:39.764 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.762+0000 7fe468a00640 1 -- 172.21.15.121:0/3293264522 shutdown_connections 2024-04-20T10:21:39.764 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.762+0000 7fe468a00640 1 --2- 172.21.15.121:0/3293264522 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe448005210 0x7fe44800b9c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:39.764 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.762+0000 7fe468a00640 1 -- 172.21.15.121:0/3293264522 >> 172.21.15.121:0/3293264522 conn(0x7fe4640b3350 msgr2=0x7fe4640b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:39.764 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.762+0000 7fe468a00640 1 -- 172.21.15.121:0/3293264522 shutdown_connections 2024-04-20T10:21:39.764 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.762+0000 7fe468a00640 1 -- 172.21.15.121:0/3293264522 wait complete. 2024-04-20T10:21:39.764 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.765+0000 7fe468a00640 1 Processor -- start 2024-04-20T10:21:39.765 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.765+0000 7fe468a00640 1 -- start start 2024-04-20T10:21:39.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.766+0000 7fe468a00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe448005210 0x7fe4641f4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:39.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.766+0000 7fe468a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe46414e470 0x7fe464158310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:39.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.766+0000 7fe468a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe46414e8f0 0x7fe46409d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:39.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.766+0000 7fe468a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe46415b910 con 0x7fe46414e8f0 2024-04-20T10:21:39.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.766+0000 7fe468a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe46415bf60 con 0x7fe46414e470 2024-04-20T10:21:39.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.766+0000 7fe468a00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe46415bbf0 con 0x7fe448005210 2024-04-20T10:21:39.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.767+0000 7fe462800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe46414e8f0 0x7fe46409d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:39.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.767+0000 7fe462800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe46414e8f0 0x7fe46409d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58672/0 (socket says 172.21.15.121:58672) 2024-04-20T10:21:39.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.767+0000 7fe462800640 1 -- 172.21.15.121:0/374606007 learned_addr learned my addr 172.21.15.121:0/374606007 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:39.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.767+0000 7fe461400640 1 --2- 172.21.15.121:0/374606007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe46414e470 0x7fe464158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:39.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.767+0000 7fe461e00640 1 --2- 172.21.15.121:0/374606007 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe448005210 0x7fe4641f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:39.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.767+0000 7fe462800640 1 -- 172.21.15.121:0/374606007 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe448005210 msgr2=0x7fe4641f4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:39.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.767+0000 7fe462800640 1 --2- 172.21.15.121:0/374606007 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe448005210 0x7fe4641f4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:39.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.767+0000 7fe462800640 1 -- 172.21.15.121:0/374606007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe46414e470 msgr2=0x7fe464158310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:39.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.767+0000 7fe462800640 1 --2- 172.21.15.121:0/374606007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe46414e470 0x7fe464158310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:39.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.767+0000 7fe462800640 1 -- 172.21.15.121:0/374606007 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe46409a540 con 0x7fe46414e8f0 2024-04-20T10:21:39.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.768+0000 7fe462800640 1 --2- 172.21.15.121:0/374606007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe46414e8f0 0x7fe46409d6a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe45c00a880 tx=0x7fe46415b520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:39.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.768+0000 7fe45a000640 1 -- 172.21.15.121:0/374606007 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45c045070 con 0x7fe46414e8f0 2024-04-20T10:21:39.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.768+0000 7fe45a000640 1 -- 172.21.15.121:0/374606007 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe45c03a040 con 0x7fe46414e8f0 2024-04-20T10:21:39.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.768+0000 7fe468a00640 1 -- 172.21.15.121:0/374606007 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe46409a820 con 0x7fe46414e8f0 2024-04-20T10:21:39.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.768+0000 7fe45a000640 1 -- 172.21.15.121:0/374606007 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45c034a20 con 0x7fe46414e8f0 2024-04-20T10:21:39.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.768+0000 7fe468a00640 1 -- 172.21.15.121:0/374606007 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe46409ade0 con 0x7fe46414e8f0 2024-04-20T10:21:39.770 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.769+0000 7fe468a00640 1 -- 172.21.15.121:0/374606007 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe464068980 con 0x7fe46414e8f0 2024-04-20T10:21:39.770 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.770+0000 7fe45a000640 1 -- 172.21.15.121:0/374606007 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe45c048070 con 0x7fe46414e8f0 2024-04-20T10:21:39.770 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.771+0000 7fe45a000640 1 --2- 172.21.15.121:0/374606007 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe44406c8f0 0x7fe44406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:39.770 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.771+0000 7fe45a000640 1 -- 172.21.15.121:0/374606007 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe45c0ad9f0 con 0x7fe46414e8f0 2024-04-20T10:21:39.771 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.772+0000 7fe461e00640 1 --2- 172.21.15.121:0/374606007 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe44406c8f0 0x7fe44406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:39.772 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.772+0000 7fe461e00640 1 --2- 172.21.15.121:0/374606007 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe44406c8f0 0x7fe44406edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe45400aa10 tx=0x7fe454008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:39.775 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.776+0000 7fe45a000640 1 -- 172.21.15.121:0/374606007 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe45c07b3f0 con 0x7fe46414e8f0 2024-04-20T10:21:39.972 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.972+0000 7fe468a00640 1 -- 172.21.15.121:0/374606007 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe46406c160 con 0x7fe46414e8f0 2024-04-20T10:21:39.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.973+0000 7fe45a000640 1 -- 172.21.15.121:0/374606007 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7fe45c07f6f0 con 0x7fe46414e8f0 2024-04-20T10:21:39.973 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2024-04-20T10:21:39.975 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.975+0000 7fe468a00640 1 -- 172.21.15.121:0/374606007 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe44406c8f0 msgr2=0x7fe44406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:39.975 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.975+0000 7fe468a00640 1 --2- 172.21.15.121:0/374606007 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe44406c8f0 0x7fe44406edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe45400aa10 tx=0x7fe454008040 comp rx=0 tx=0).stop 2024-04-20T10:21:39.975 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.975+0000 7fe468a00640 1 -- 172.21.15.121:0/374606007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe46414e8f0 msgr2=0x7fe46409d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:39.975 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.975+0000 7fe468a00640 1 --2- 172.21.15.121:0/374606007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe46414e8f0 0x7fe46409d6a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe45c00a880 tx=0x7fe46415b520 comp rx=0 tx=0).stop 2024-04-20T10:21:39.975 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.975+0000 7fe468a00640 1 -- 172.21.15.121:0/374606007 shutdown_connections 2024-04-20T10:21:39.975 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.975+0000 7fe468a00640 1 --2- 172.21.15.121:0/374606007 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe46414e8f0 0x7fe46409d6a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:39.975 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.975+0000 7fe468a00640 1 --2- 172.21.15.121:0/374606007 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe44406c8f0 0x7fe44406edb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:39.975 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.975+0000 7fe468a00640 1 --2- 172.21.15.121:0/374606007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe46414e470 0x7fe464158310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:39.976 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.975+0000 7fe468a00640 1 --2- 172.21.15.121:0/374606007 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe448005210 0x7fe4641f4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:39.976 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.976+0000 7fe468a00640 1 -- 172.21.15.121:0/374606007 >> 172.21.15.121:0/374606007 conn(0x7fe4640b3350 msgr2=0x7fe4640ad810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:39.976 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.976+0000 7fe468a00640 1 -- 172.21.15.121:0/374606007 shutdown_connections 2024-04-20T10:21:39.976 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:39.976+0000 7fe468a00640 1 -- 172.21.15.121:0/374606007 wait complete. 2024-04-20T10:21:39.993 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T10:21:40.196 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.196+0000 7f0bc6c00640 1 Processor -- start 2024-04-20T10:21:40.196 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.197+0000 7f0bc6c00640 1 -- start start 2024-04-20T10:21:40.198 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.198+0000 7f0bc6c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0bc0072cc0 0x7f0bc00ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:40.198 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.198+0000 7f0bc6c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0bc0071b50 0x7f0bc00adde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:40.199 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.198+0000 7f0bc6c00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bc006e840 con 0x7f0bc015abb0 2024-04-20T10:21:40.199 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.198+0000 7f0bc6c00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f0bc006eb00 con 0x7f0bc0072cc0 2024-04-20T10:21:40.199 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.198+0000 7f0bc6c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f0bc015bf60 con 0x7f0bc0071b50 2024-04-20T10:21:40.199 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.198+0000 7f0bbfe00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f0bc015abb0 0x7f0bc00725b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:47648/0 (socket says 172.21.15.121:47648) 2024-04-20T10:21:40.199 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.198+0000 7f0bbfe00640 1 -- 172.21.15.121:0/3736860715 learned_addr learned my addr 172.21.15.121:0/3736860715 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:40.199 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.198+0000 7f0bb7400640 1 --2- 172.21.15.121:0/3736860715 >> v2:172.21.15.121:3300/0 conn(0x7f0bc0072cc0 0x7f0bc00ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:40.199 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.199+0000 7f0bc4c00640 1 --2- 172.21.15.121:0/3736860715 >> v2:172.21.15.123:3300/0 conn(0x7f0bc0071b50 0x7f0bc00adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:40.199 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.199+0000 7f0bbf400640 1 -- 172.21.15.121:0/3736860715 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3727412414 0 0) 0x7f0bc006e840 con 0x7f0bc015abb0 2024-04-20T10:21:40.199 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.199+0000 7f0bbf400640 1 -- 172.21.15.121:0/3736860715 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ba4005730 con 0x7f0bc015abb0 2024-04-20T10:21:40.199 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.199+0000 7f0bb7400640 1 -- 172.21.15.121:0/3736860715 >> v2:172.21.15.123:3300/0 conn(0x7f0bc0071b50 msgr2=0x7f0bc00adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:40.199 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.199+0000 7f0bb7400640 1 --2- 172.21.15.121:0/3736860715 >> v2:172.21.15.123:3300/0 conn(0x7f0bc0071b50 0x7f0bc00adde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:40.199 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.199+0000 7f0bb7400640 1 -- 172.21.15.121:0/3736860715 >> v1:172.21.15.121:6789/0 conn(0x7f0bc015abb0 legacy=0x7f0bc00725b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:40.199 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.199+0000 7f0bb7400640 1 -- 172.21.15.121:0/3736860715 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc015b350 con 0x7f0bc0072cc0 2024-04-20T10:21:40.199 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.199+0000 7f0bc4c00640 1 --2- 172.21.15.121:0/3736860715 >> v2:172.21.15.123:3300/0 conn(0x7f0bc0071b50 0x7f0bc00adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:21:40.200 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.200+0000 7f0bb7400640 1 --2- 172.21.15.121:0/3736860715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bc0072cc0 0x7f0bc00ad8a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0073fd0 tx=0x7f0ba802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922aaac395b9c87c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:40.200 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.200+0000 7f0bbf400640 1 -- 172.21.15.121:0/3736860715 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba8035020 con 0x7f0bc0072cc0 2024-04-20T10:21:40.200 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.200+0000 7f0bbf400640 1 -- 172.21.15.121:0/3736860715 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ba8031830 con 0x7f0bc0072cc0 2024-04-20T10:21:40.200 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.201+0000 7f0bbf400640 1 -- 172.21.15.121:0/3736860715 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba8031c00 con 0x7f0bc0072cc0 2024-04-20T10:21:40.201 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.201+0000 7f0bc6c00640 1 -- 172.21.15.121:0/3736860715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bc0072cc0 msgr2=0x7f0bc00ad8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:40.201 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.201+0000 7f0bc6c00640 1 --2- 172.21.15.121:0/3736860715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bc0072cc0 0x7f0bc00ad8a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0073fd0 tx=0x7f0ba802b8d0 comp rx=0 tx=0).stop 2024-04-20T10:21:40.201 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.201+0000 7f0bc6c00640 1 -- 172.21.15.121:0/3736860715 shutdown_connections 2024-04-20T10:21:40.201 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.201+0000 7f0bc6c00640 1 --2- 172.21.15.121:0/3736860715 >> v2:172.21.15.123:3300/0 conn(0x7f0bc0071b50 0x7f0bc00adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:40.201 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.201+0000 7f0bc6c00640 1 --2- 172.21.15.121:0/3736860715 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bc0072cc0 0x7f0bc00ad8a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:40.201 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.201+0000 7f0bc6c00640 1 -- 172.21.15.121:0/3736860715 >> 172.21.15.121:0/3736860715 conn(0x7f0bc00b3350 msgr2=0x7f0bc00b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:40.201 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.201+0000 7f0bc6c00640 1 -- 172.21.15.121:0/3736860715 shutdown_connections 2024-04-20T10:21:40.201 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.201+0000 7f0bc6c00640 1 -- 172.21.15.121:0/3736860715 wait complete. 2024-04-20T10:21:40.202 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.203+0000 7f0bc6c00640 1 Processor -- start 2024-04-20T10:21:40.203 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.203+0000 7f0bc6c00640 1 -- start start 2024-04-20T10:21:40.204 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.204+0000 7f0bc6c00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0bc0071b50 0x7f0bc0157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:40.204 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.204+0000 7f0bc6c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bc0072cc0 0x7f0bc015d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:40.204 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.204+0000 7f0bc6c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0bc015abb0 0x7f0bc015d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:40.204 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.204+0000 7f0bc6c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc015bbf0 con 0x7f0bc0072cc0 2024-04-20T10:21:40.204 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.204+0000 7f0bc6c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc015b910 con 0x7f0bc015abb0 2024-04-20T10:21:40.205 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.204+0000 7f0bc6c00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f0bc015af90 con 0x7f0bc0071b50 2024-04-20T10:21:40.205 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.204+0000 7f0bb7400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bc0072cc0 0x7f0bc015d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:40.205 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.205+0000 7f0bb7400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bc0072cc0 0x7f0bc015d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58692/0 (socket says 172.21.15.121:58692) 2024-04-20T10:21:40.205 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.205+0000 7f0bb7400640 1 -- 172.21.15.121:0/2623059810 learned_addr learned my addr 172.21.15.121:0/2623059810 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:40.205 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.205+0000 7f0bbfe00640 1 --2- 172.21.15.121:0/2623059810 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0bc0071b50 0x7f0bc0157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:40.205 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.205+0000 7f0bc4c00640 1 --2- 172.21.15.121:0/2623059810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0bc015abb0 0x7f0bc015d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:40.205 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.205+0000 7f0bb7400640 1 -- 172.21.15.121:0/2623059810 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0bc0071b50 msgr2=0x7f0bc0157500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:40.205 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.205+0000 7f0bb7400640 1 --2- 172.21.15.121:0/2623059810 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0bc0071b50 0x7f0bc0157500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:40.205 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.205+0000 7f0bb7400640 1 -- 172.21.15.121:0/2623059810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0bc015abb0 msgr2=0x7f0bc015d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:40.205 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.205+0000 7f0bb7400640 1 --2- 172.21.15.121:0/2623059810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0bc015abb0 0x7f0bc015d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:40.205 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.205+0000 7f0bb7400640 1 -- 172.21.15.121:0/2623059810 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc01f10f0 con 0x7f0bc0072cc0 2024-04-20T10:21:40.206 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.205+0000 7f0bb7400640 1 --2- 172.21.15.121:0/2623059810 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bc0072cc0 0x7f0bc015d2c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8009890 tx=0x7f0ba80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:40.206 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.206+0000 7f0bbcc00640 1 -- 172.21.15.121:0/2623059810 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba8041070 con 0x7f0bc0072cc0 2024-04-20T10:21:40.206 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.206+0000 7f0bbcc00640 1 -- 172.21.15.121:0/2623059810 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ba803b920 con 0x7f0bc0072cc0 2024-04-20T10:21:40.206 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.206+0000 7f0bbcc00640 1 -- 172.21.15.121:0/2623059810 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba8035040 con 0x7f0bc0072cc0 2024-04-20T10:21:40.206 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.206+0000 7f0bc6c00640 1 -- 172.21.15.121:0/2623059810 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bc009ba50 con 0x7f0bc0072cc0 2024-04-20T10:21:40.206 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.206+0000 7f0bc6c00640 1 -- 172.21.15.121:0/2623059810 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bc0157a40 con 0x7f0bc0072cc0 2024-04-20T10:21:40.207 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.207+0000 7f0bc6c00640 1 -- 172.21.15.121:0/2623059810 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bc0068980 con 0x7f0bc0072cc0 2024-04-20T10:21:40.207 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.208+0000 7f0bbcc00640 1 -- 172.21.15.121:0/2623059810 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0ba8038020 con 0x7f0bc0072cc0 2024-04-20T10:21:40.208 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.209+0000 7f0bbcc00640 1 --2- 172.21.15.121:0/2623059810 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f0b9c070d50 0x7f0b9c073210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:40.208 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.209+0000 7f0bbcc00640 1 -- 172.21.15.121:0/2623059810 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0ba80ad8b0 con 0x7f0bc0072cc0 2024-04-20T10:21:40.209 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.209+0000 7f0bbfe00640 1 --2- 172.21.15.121:0/2623059810 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f0b9c070d50 0x7f0b9c073210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:40.209 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.210+0000 7f0bbfe00640 1 --2- 172.21.15.121:0/2623059810 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f0b9c070d50 0x7f0b9c073210 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0005880 tx=0x7f0bb0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:40.213 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.214+0000 7f0bbcc00640 1 -- 172.21.15.121:0/2623059810 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0ba802f2e0 con 0x7f0bc0072cc0 2024-04-20T10:21:40.396 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.396+0000 7f0bc6c00640 1 -- 172.21.15.121:0/2623059810 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f0bc00b1a50 con 0x7f0bc0072cc0 2024-04-20T10:21:40.397 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.398+0000 7f0bbcc00640 1 -- 172.21.15.121:0/2623059810 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f0ba8031800 con 0x7f0bc0072cc0 2024-04-20T10:21:40.397 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 32 2024-04-20T10:21:40.400 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.400+0000 7f0bc6c00640 1 -- 172.21.15.121:0/2623059810 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f0b9c070d50 msgr2=0x7f0b9c073210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:40.400 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.400+0000 7f0bc6c00640 1 --2- 172.21.15.121:0/2623059810 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f0b9c070d50 0x7f0b9c073210 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0005880 tx=0x7f0bb0009040 comp rx=0 tx=0).stop 2024-04-20T10:21:40.400 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.400+0000 7f0bc6c00640 1 -- 172.21.15.121:0/2623059810 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bc0072cc0 msgr2=0x7f0bc015d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:40.400 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.400+0000 7f0bc6c00640 1 --2- 172.21.15.121:0/2623059810 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bc0072cc0 0x7f0bc015d2c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8009890 tx=0x7f0ba80046c0 comp rx=0 tx=0).stop 2024-04-20T10:21:40.400 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.400+0000 7f0bc6c00640 1 -- 172.21.15.121:0/2623059810 shutdown_connections 2024-04-20T10:21:40.400 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.400+0000 7f0bc6c00640 1 --2- 172.21.15.121:0/2623059810 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f0b9c070d50 0x7f0b9c073210 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:40.400 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.400+0000 7f0bc6c00640 1 --2- 172.21.15.121:0/2623059810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0bc015abb0 0x7f0bc015d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:40.401 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.400+0000 7f0bc6c00640 1 --2- 172.21.15.121:0/2623059810 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0bc0072cc0 0x7f0bc015d2c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:40.401 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.400+0000 7f0bc6c00640 1 --2- 172.21.15.121:0/2623059810 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0bc0071b50 0x7f0bc0157500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:40.401 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.400+0000 7f0bc6c00640 1 -- 172.21.15.121:0/2623059810 >> 172.21.15.121:0/2623059810 conn(0x7f0bc00b3350 msgr2=0x7f0bc00ad8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:40.401 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.401+0000 7f0bc6c00640 1 -- 172.21.15.121:0/2623059810 shutdown_connections 2024-04-20T10:21:40.401 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.401+0000 7f0bc6c00640 1 -- 172.21.15.121:0/2623059810 wait complete. 2024-04-20T10:21:40.417 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:21:40.602 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.603+0000 7f2bbca00640 1 Processor -- start 2024-04-20T10:21:40.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.603+0000 7f2bbca00640 1 -- start start 2024-04-20T10:21:40.605 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.604+0000 7f2bbca00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2bb815b400 0x7f2bb8072d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:40.605 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.605+0000 7f2bbca00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2bb815fa40 0x7f2bb806ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:40.605 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.605+0000 7f2bbca00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bb815c7f0 con 0x7f2bb815eec0 2024-04-20T10:21:40.605 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.605+0000 7f2bbca00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f2bb806d280 con 0x7f2bb815b400 2024-04-20T10:21:40.605 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.605+0000 7f2bbca00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f2bb806ccc0 con 0x7f2bb815fa40 2024-04-20T10:21:40.605 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.605+0000 7f2bb7400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2bb815b400 0x7f2bb8072d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:40.605 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.605+0000 7f2bb7400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2bb815b400 0x7f2bb8072d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58694/0 (socket says 172.21.15.121:58694) 2024-04-20T10:21:40.605 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.605+0000 7f2bb7400640 1 -- 172.21.15.121:0/3508064649 learned_addr learned my addr 172.21.15.121:0/3508064649 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:40.605 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.605+0000 7f2bb7400640 1 -- 172.21.15.121:0/3508064649 >> v2:172.21.15.123:3300/0 conn(0x7f2bb815fa40 msgr2=0x7f2bb806ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:40.605 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.605+0000 7f2bb7400640 1 --2- 172.21.15.121:0/3508064649 >> v2:172.21.15.123:3300/0 conn(0x7f2bb815fa40 0x7f2bb806ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:40.605 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.605+0000 7f2bb7400640 1 -- 172.21.15.121:0/3508064649 >> v1:172.21.15.123:6789/0 conn(0x7f2bb815eec0 legacy=0x7f2bb815f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:40.605 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.605+0000 7f2bb7400640 1 -- 172.21.15.121:0/3508064649 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bb806cfa0 con 0x7f2bb815b400 2024-04-20T10:21:40.606 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.606+0000 7f2bb7400640 1 --2- 172.21.15.121:0/3508064649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bb815b400 0x7f2bb8072d00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8072250 tx=0x7f2ba802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c94c201826c776 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:40.606 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.607+0000 7f2bb6a00640 1 -- 172.21.15.121:0/3508064649 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba8035020 con 0x7f2bb815b400 2024-04-20T10:21:40.606 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.607+0000 7f2bb6a00640 1 -- 172.21.15.121:0/3508064649 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ba8031870 con 0x7f2bb815b400 2024-04-20T10:21:40.606 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.607+0000 7f2bb6a00640 1 -- 172.21.15.121:0/3508064649 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba8031c40 con 0x7f2bb815b400 2024-04-20T10:21:40.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.607+0000 7f2bbca00640 1 -- 172.21.15.121:0/3508064649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bb815b400 msgr2=0x7f2bb8072d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:40.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.607+0000 7f2bbca00640 1 --2- 172.21.15.121:0/3508064649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bb815b400 0x7f2bb8072d00 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8072250 tx=0x7f2ba802b910 comp rx=0 tx=0).stop 2024-04-20T10:21:40.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.608+0000 7f2bbca00640 1 -- 172.21.15.121:0/3508064649 shutdown_connections 2024-04-20T10:21:40.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.608+0000 7f2bbca00640 1 --2- 172.21.15.121:0/3508064649 >> v2:172.21.15.123:3300/0 conn(0x7f2bb815fa40 0x7f2bb806ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:40.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.608+0000 7f2bbca00640 1 --2- 172.21.15.121:0/3508064649 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bb815b400 0x7f2bb8072d00 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:40.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.608+0000 7f2bbca00640 1 -- 172.21.15.121:0/3508064649 >> 172.21.15.121:0/3508064649 conn(0x7f2bb80d0040 msgr2=0x7f2bb80d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:40.608 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.608+0000 7f2bbca00640 1 -- 172.21.15.121:0/3508064649 shutdown_connections 2024-04-20T10:21:40.608 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.608+0000 7f2bbca00640 1 -- 172.21.15.121:0/3508064649 wait complete. 2024-04-20T10:21:40.609 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.610+0000 7f2bbca00640 1 Processor -- start 2024-04-20T10:21:40.609 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.610+0000 7f2bbca00640 1 -- start start 2024-04-20T10:21:40.611 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.611+0000 7f2bbca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bb815b400 0x7f2bb806b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:40.611 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.611+0000 7f2bbca00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bb815eec0 0x7f2bb806be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:40.611 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.611+0000 7f2bbca00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2bb815fa40 0x7f2bb8157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:40.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.611+0000 7f2bbca00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb8073240 con 0x7f2bb815eec0 2024-04-20T10:21:40.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.611+0000 7f2bbca00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb806e720 con 0x7f2bb815b400 2024-04-20T10:21:40.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.611+0000 7f2bbca00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f2bb806f180 con 0x7f2bb815fa40 2024-04-20T10:21:40.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.611+0000 7f2baea00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bb815eec0 0x7f2bb806be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:40.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.612+0000 7f2baea00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bb815eec0 0x7f2bb806be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58702/0 (socket says 172.21.15.121:58702) 2024-04-20T10:21:40.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.612+0000 7f2baea00640 1 -- 172.21.15.121:0/2360973361 learned_addr learned my addr 172.21.15.121:0/2360973361 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:40.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.612+0000 7f2bb7400640 1 --2- 172.21.15.121:0/2360973361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bb815b400 0x7f2bb806b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:40.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.612+0000 7f2bb7e00640 1 --2- 172.21.15.121:0/2360973361 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2bb815fa40 0x7f2bb8157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:40.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.612+0000 7f2baea00640 1 -- 172.21.15.121:0/2360973361 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2bb815fa40 msgr2=0x7f2bb8157590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:40.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.612+0000 7f2baea00640 1 --2- 172.21.15.121:0/2360973361 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2bb815fa40 0x7f2bb8157590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:40.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.612+0000 7f2baea00640 1 -- 172.21.15.121:0/2360973361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bb815b400 msgr2=0x7f2bb806b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:40.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.612+0000 7f2baea00640 1 --2- 172.21.15.121:0/2360973361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bb815b400 0x7f2bb806b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:40.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.612+0000 7f2baea00640 1 -- 172.21.15.121:0/2360973361 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bb809da90 con 0x7f2bb815eec0 2024-04-20T10:21:40.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.612+0000 7f2baea00640 1 --2- 172.21.15.121:0/2360973361 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bb815eec0 0x7f2bb806be40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2ba000c440 tx=0x7f2ba000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:40.613 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.612+0000 7f2bafe00640 1 -- 172.21.15.121:0/2360973361 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba0017070 con 0x7f2bb815eec0 2024-04-20T10:21:40.613 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.613+0000 7f2bafe00640 1 -- 172.21.15.121:0/2360973361 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ba00035f0 con 0x7f2bb815eec0 2024-04-20T10:21:40.613 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.613+0000 7f2bafe00640 1 -- 172.21.15.121:0/2360973361 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba00039e0 con 0x7f2bb815eec0 2024-04-20T10:21:40.613 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.613+0000 7f2bbca00640 1 -- 172.21.15.121:0/2360973361 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bb8158170 con 0x7f2bb815eec0 2024-04-20T10:21:40.613 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.613+0000 7f2bbca00640 1 -- 172.21.15.121:0/2360973361 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bb8158730 con 0x7f2bb815eec0 2024-04-20T10:21:40.614 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.614+0000 7f2bbca00640 1 -- 172.21.15.121:0/2360973361 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bb8075b60 con 0x7f2bb815eec0 2024-04-20T10:21:40.614 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.615+0000 7f2bafe00640 1 -- 172.21.15.121:0/2360973361 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2ba001a070 con 0x7f2bb815eec0 2024-04-20T10:21:40.615 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.616+0000 7f2bafe00640 1 --2- 172.21.15.121:0/2360973361 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f2b9406c8f0 0x7f2b9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:40.615 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.616+0000 7f2bafe00640 1 -- 172.21.15.121:0/2360973361 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2ba008b2e0 con 0x7f2bb815eec0 2024-04-20T10:21:40.616 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.616+0000 7f2bb7400640 1 --2- 172.21.15.121:0/2360973361 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f2b9406c8f0 0x7f2b9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:40.616 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.617+0000 7f2bb7400640 1 --2- 172.21.15.121:0/2360973361 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f2b9406c8f0 0x7f2b9406edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8036000 tx=0x7f2ba803c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:40.619 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.620+0000 7f2bafe00640 1 -- 172.21.15.121:0/2360973361 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2ba0059330 con 0x7f2bb815eec0 2024-04-20T10:21:40.845 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.845+0000 7f2bbca00640 1 -- 172.21.15.121:0/2360973361 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2bb80b5220 con 0x7f2bb815eec0 2024-04-20T10:21:40.846 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.847+0000 7f2bafe00640 1 -- 172.21.15.121:0/2360973361 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5204 (secure 0 0 0) 0x7f2ba00010c0 con 0x7f2bb815eec0 2024-04-20T10:21:40.847 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:21:40.847 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1376743826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1376743826},{"type":"v1","addr":"172.21.15.123:6833","nonce":1376743826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/999167058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":999167058},{"type":"v1","addr":"172.21.15.121:6835","nonce":999167058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2160736935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2160736935},{"type":"v1","addr":"172.21.15.121:6837","nonce":2160736935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:21:02.048590+0000","modified":"2024-04-20T10:21:29.775269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6835/3336775250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3336775250},{"type":"v1","addr":"172.21.15.123:6835","nonce":3336775250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1}]} 2024-04-20T10:21:40.847 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 8 2024-04-20T10:21:40.849 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.849+0000 7f2bbca00640 1 -- 172.21.15.121:0/2360973361 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f2b9406c8f0 msgr2=0x7f2b9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:40.849 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.849+0000 7f2bbca00640 1 --2- 172.21.15.121:0/2360973361 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f2b9406c8f0 0x7f2b9406edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8036000 tx=0x7f2ba803c040 comp rx=0 tx=0).stop 2024-04-20T10:21:40.849 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.849+0000 7f2bbca00640 1 -- 172.21.15.121:0/2360973361 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bb815eec0 msgr2=0x7f2bb806be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:40.849 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.849+0000 7f2bbca00640 1 --2- 172.21.15.121:0/2360973361 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bb815eec0 0x7f2bb806be40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2ba000c440 tx=0x7f2ba000c900 comp rx=0 tx=0).stop 2024-04-20T10:21:40.849 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.849+0000 7f2bbca00640 1 -- 172.21.15.121:0/2360973361 shutdown_connections 2024-04-20T10:21:40.849 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.849+0000 7f2bbca00640 1 --2- 172.21.15.121:0/2360973361 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2bb815fa40 0x7f2bb8157590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:40.849 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.849+0000 7f2bbca00640 1 --2- 172.21.15.121:0/2360973361 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2bb815eec0 0x7f2bb806be40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:40.850 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.849+0000 7f2bbca00640 1 --2- 172.21.15.121:0/2360973361 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f2b9406c8f0 0x7f2b9406edb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:40.850 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.849+0000 7f2bbca00640 1 --2- 172.21.15.121:0/2360973361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bb815b400 0x7f2bb806b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:40.850 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.849+0000 7f2bbca00640 1 -- 172.21.15.121:0/2360973361 >> 172.21.15.121:0/2360973361 conn(0x7f2bb80d0040 msgr2=0x7f2bb815d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:40.850 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.850+0000 7f2bbca00640 1 -- 172.21.15.121:0/2360973361 shutdown_connections 2024-04-20T10:21:40.850 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:40.850+0000 7f2bbca00640 1 -- 172.21.15.121:0/2360973361 wait complete. 2024-04-20T10:21:40.867 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:21:41.055 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.055+0000 7f31ce400640 1 Processor -- start 2024-04-20T10:21:41.055 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.055+0000 7f31ce400640 1 -- start start 2024-04-20T10:21:41.056 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.057+0000 7f31ce400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f31c80b2f70 0x7f31c80b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:41.056 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.057+0000 7f31ce400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31c806b900 0x7f31c806bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:41.056 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.057+0000 7f31ce400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31c8071a80 con 0x7f31c80adf60 2024-04-20T10:21:41.056 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.057+0000 7f31ce400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f31c80ae3f0 con 0x7f31c80b2f70 2024-04-20T10:21:41.056 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.057+0000 7f31ce400640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f31c806c240 con 0x7f31c806b900 2024-04-20T10:21:41.058 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.057+0000 7f31c7e00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f31c80adf60 0x7f31c815f5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:47654/0 (socket says 172.21.15.121:47654) 2024-04-20T10:21:41.059 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.057+0000 7f31c7e00640 1 -- 172.21.15.121:0/2670035519 learned_addr learned my addr 172.21.15.121:0/2670035519 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:41.059 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.057+0000 7f31cda00640 1 --2- 172.21.15.121:0/2670035519 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31c806b900 0x7f31c806bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:41.059 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.057+0000 7f31cd000640 1 --2- 172.21.15.121:0/2670035519 >> v2:172.21.15.123:3300/0 conn(0x7f31c80b2f70 0x7f31c80b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:41.059 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.058+0000 7f31c7400640 1 -- 172.21.15.121:0/2670035519 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 888489069 0 0) 0x7f31c8071a80 con 0x7f31c80adf60 2024-04-20T10:21:41.059 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.058+0000 7f31c7400640 1 -- 172.21.15.121:0/2670035519 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f31b0003520 con 0x7f31c80adf60 2024-04-20T10:21:41.059 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.058+0000 7f31c7400640 1 -- 172.21.15.121:0/2670035519 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3837860815 0 0) 0x7f31b0003520 con 0x7f31c80adf60 2024-04-20T10:21:41.059 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.058+0000 7f31c7400640 1 -- 172.21.15.121:0/2670035519 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31c806b900 msgr2=0x7f31c806bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:41.059 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.058+0000 7f31c7400640 1 --2- 172.21.15.121:0/2670035519 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31c806b900 0x7f31c806bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:41.059 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.058+0000 7f31c7400640 1 -- 172.21.15.121:0/2670035519 >> v2:172.21.15.123:3300/0 conn(0x7f31c80b2f70 msgr2=0x7f31c80b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:41.059 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.058+0000 7f31c7400640 1 --2- 172.21.15.121:0/2670035519 >> v2:172.21.15.123:3300/0 conn(0x7f31c80b2f70 0x7f31c80b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:41.059 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.058+0000 7f31c7400640 1 -- 172.21.15.121:0/2670035519 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c8071cf0 con 0x7f31c80adf60 2024-04-20T10:21:41.059 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.058+0000 7f31c7400640 1 -- 172.21.15.121:0/2670035519 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f31b4001670 con 0x7f31c80adf60 2024-04-20T10:21:41.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.058+0000 7f31c7400640 1 -- 172.21.15.121:0/2670035519 >> v1:172.21.15.121:6789/0 conn(0x7f31c80adf60 legacy=0x7f31c815f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:41.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.058+0000 7f31c7400640 1 --2- 172.21.15.121:0/2670035519 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31b00039b0 0x7f31b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:41.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.058+0000 7f31c7400640 1 --2- 172.21.15.121:0/2670035519 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31c806b900 0x7f31b0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:41.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.058+0000 7f31c7400640 1 --2- 172.21.15.121:0/2670035519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31b0004d40 0x7f31b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:41.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.058+0000 7f31c7400640 1 -- 172.21.15.121:0/2670035519 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f31c80ae3f0 con 0x7f31b00039b0 2024-04-20T10:21:41.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.058+0000 7f31c7400640 1 -- 172.21.15.121:0/2670035519 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f31c806c240 con 0x7f31b0004d40 2024-04-20T10:21:41.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.058+0000 7f31c7400640 1 -- 172.21.15.121:0/2670035519 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f31b000b9c0 con 0x7f31c806b900 2024-04-20T10:21:41.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.059+0000 7f31cd000640 1 --2- 172.21.15.121:0/2670035519 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31b00039b0 0x7f31b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:41.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.059+0000 7f31c7e00640 1 --2- 172.21.15.121:0/2670035519 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31c806b900 0x7f31b0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:41.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.059+0000 7f31cd000640 1 -- 172.21.15.121:0/2670035519 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31c806b900 msgr2=0x7f31b0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:41.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.059+0000 7f31cda00640 1 --2- 172.21.15.121:0/2670035519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31b0004d40 0x7f31b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:41.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.059+0000 7f31cd000640 1 --2- 172.21.15.121:0/2670035519 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31c806b900 0x7f31b0004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:41.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.059+0000 7f31cd000640 1 -- 172.21.15.121:0/2670035519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31b0004d40 msgr2=0x7f31b00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:41.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.059+0000 7f31cd000640 1 --2- 172.21.15.121:0/2670035519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31b0004d40 0x7f31b00051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:41.061 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.059+0000 7f31cd000640 1 -- 172.21.15.121:0/2670035519 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31b0003520 con 0x7f31b00039b0 2024-04-20T10:21:41.061 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.060+0000 7f31cd000640 1 --2- 172.21.15.121:0/2670035519 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31b00039b0 0x7f31b0003dd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f31bc009ac0 tx=0x7f31b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccb9569f84ea0b17 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:41.062 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.060+0000 7f31c7400640 1 -- 172.21.15.121:0/2670035519 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31bc039070 con 0x7f31b00039b0 2024-04-20T10:21:41.062 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.060+0000 7f31c7400640 1 -- 172.21.15.121:0/2670035519 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31bc02c710 con 0x7f31b00039b0 2024-04-20T10:21:41.062 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.060+0000 7f31c7400640 1 -- 172.21.15.121:0/2670035519 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31bc009690 con 0x7f31b00039b0 2024-04-20T10:21:41.062 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.061+0000 7f31ce400640 1 -- 172.21.15.121:0/2670035519 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31b00039b0 msgr2=0x7f31b0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:41.062 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.061+0000 7f31ce400640 1 --2- 172.21.15.121:0/2670035519 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31b00039b0 0x7f31b0003dd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f31bc009ac0 tx=0x7f31b0005830 comp rx=0 tx=0).stop 2024-04-20T10:21:41.062 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.061+0000 7f31cda00640 1 -- 172.21.15.121:0/2670035519 reap_dead start 2024-04-20T10:21:41.062 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.061+0000 7f31ce400640 1 -- 172.21.15.121:0/2670035519 shutdown_connections 2024-04-20T10:21:41.062 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.061+0000 7f31ce400640 1 -- 172.21.15.121:0/2670035519 >> 172.21.15.121:0/2670035519 conn(0x7f31c80d0040 msgr2=0x7f31c80d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:41.062 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.061+0000 7f31ce400640 1 -- 172.21.15.121:0/2670035519 shutdown_connections 2024-04-20T10:21:41.062 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.061+0000 7f31ce400640 1 -- 172.21.15.121:0/2670035519 wait complete. 2024-04-20T10:21:41.064 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.063+0000 7f31ce400640 1 Processor -- start 2024-04-20T10:21:41.064 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.063+0000 7f31ce400640 1 -- start start 2024-04-20T10:21:41.064 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.064+0000 7f31ce400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31b00039b0 0x7f31c815d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:41.064 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.064+0000 7f31ce400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31c81e4e80 0x7f31c814ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:41.064 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.064+0000 7f31ce400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31c81599f0 0x7f31c8159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:41.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.064+0000 7f31ce400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f31c80736c0 con 0x7f31c81e4e80 2024-04-20T10:21:41.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.064+0000 7f31ce400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f31c80739c0 con 0x7f31b00039b0 2024-04-20T10:21:41.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.064+0000 7f31ce400640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f31c815fcc0 con 0x7f31c81599f0 2024-04-20T10:21:41.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.064+0000 7f31c7e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31c81e4e80 0x7f31c814ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:41.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.064+0000 7f31c7e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31c81e4e80 0x7f31c814ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58732/0 (socket says 172.21.15.121:58732) 2024-04-20T10:21:41.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.064+0000 7f31c7e00640 1 -- 172.21.15.121:0/2828108208 learned_addr learned my addr 172.21.15.121:0/2828108208 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:41.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.064+0000 7f31cda00640 1 --2- 172.21.15.121:0/2828108208 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31c81599f0 0x7f31c8159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:41.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.064+0000 7f31c7e00640 1 -- 172.21.15.121:0/2828108208 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31c81599f0 msgr2=0x7f31c8159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:41.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.064+0000 7f31c7e00640 1 --2- 172.21.15.121:0/2828108208 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31c81599f0 0x7f31c8159ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:41.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.064+0000 7f31c7e00640 1 -- 172.21.15.121:0/2828108208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31b00039b0 msgr2=0x7f31c815d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:41.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.064+0000 7f31c7e00640 1 --2- 172.21.15.121:0/2828108208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31b00039b0 0x7f31c815d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:41.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.064+0000 7f31c7e00640 1 -- 172.21.15.121:0/2828108208 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c815dc80 con 0x7f31c81e4e80 2024-04-20T10:21:41.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.065+0000 7f31c7e00640 1 --2- 172.21.15.121:0/2828108208 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31c81e4e80 0x7f31c814ca90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f31b400d7d0 tx=0x7f31b400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:41.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.065+0000 7f31c4c00640 1 -- 172.21.15.121:0/2828108208 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b4019070 con 0x7f31c81e4e80 2024-04-20T10:21:41.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.065+0000 7f31c4c00640 1 -- 172.21.15.121:0/2828108208 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31b40074e0 con 0x7f31c81e4e80 2024-04-20T10:21:41.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.065+0000 7f31ce400640 1 -- 172.21.15.121:0/2828108208 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31c815df60 con 0x7f31c81e4e80 2024-04-20T10:21:41.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.065+0000 7f31c4c00640 1 -- 172.21.15.121:0/2828108208 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b4004a20 con 0x7f31c81e4e80 2024-04-20T10:21:41.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.065+0000 7f31ce400640 1 -- 172.21.15.121:0/2828108208 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31c815e520 con 0x7f31c81e4e80 2024-04-20T10:21:41.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.066+0000 7f31ce400640 1 -- 172.21.15.121:0/2828108208 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31c8075b60 con 0x7f31c81e4e80 2024-04-20T10:21:41.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.067+0000 7f31c4c00640 1 -- 172.21.15.121:0/2828108208 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f31b401c070 con 0x7f31c81e4e80 2024-04-20T10:21:41.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.068+0000 7f31c4c00640 1 --2- 172.21.15.121:0/2828108208 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f31a806c920 0x7f31a806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:41.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.068+0000 7f31c4c00640 1 -- 172.21.15.121:0/2828108208 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f31b408b930 con 0x7f31c81e4e80 2024-04-20T10:21:41.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.068+0000 7f31cd000640 1 --2- 172.21.15.121:0/2828108208 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f31a806c920 0x7f31a806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:41.068 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.069+0000 7f31cd000640 1 --2- 172.21.15.121:0/2828108208 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f31a806c920 0x7f31a806ede0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f31bc036000 tx=0x7f31bc02e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:41.072 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.072+0000 7f31c4c00640 1 -- 172.21.15.121:0/2828108208 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f31b4059980 con 0x7f31c81e4e80 2024-04-20T10:21:41.274 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.275+0000 7f31ce400640 1 -- 172.21.15.121:0/2828108208 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f31c806d220 con 0x7f31c81e4e80 2024-04-20T10:21:41.276 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.277+0000 7f31c4c00640 1 -- 172.21.15.121:0/2828108208 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7f31b4059280 con 0x7f31c81e4e80 2024-04-20T10:21:41.276 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:21:41.276 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":25,"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","created":"2024-04-20T10:20:38.076076+0000","modified":"2024-04-20T10:21:07.336542+0000","last_up_change":"2024-04-20T10:20:50.162700+0000","last_in_change":"2024-04-20T10:20:41.510382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:20:51.141365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:21:00.436937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:21:01.026957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:21:04.862693+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"629a7b8c-a40a-4a2a-a3cc-d9f357774310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6809","nonce":1044588835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6811","nonce":1044588835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6815","nonce":1044588835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6813","nonce":1044588835}]},"public_addr":"172.21.15.121:6809/1044588835","cluster_addr":"172.21.15.121:6811/1044588835","heartbeat_back_addr":"172.21.15.121:6815/1044588835","heartbeat_front_addr":"172.21.15.121:6813/1044588835","state":["exists","up"]},{"osd":1,"uuid":"f2570399-aa50-4ff3-bf5d-3d09207290db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6801","nonce":3505199624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6803","nonce":3505199624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6807","nonce":3505199624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6805","nonce":3505199624}]},"public_addr":"172.21.15.121:6801/3505199624","cluster_addr":"172.21.15.121:6803/3505199624","heartbeat_back_addr":"172.21.15.121:6807/3505199624","heartbeat_front_addr":"172.21.15.121:6805/3505199624","state":["exists","up"]},{"osd":2,"uuid":"50893807-dc30-49f4-b41f-739366626658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6817","nonce":3656724207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6819","nonce":3656724207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6827","nonce":3656724207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6822","nonce":3656724207}]},"public_addr":"172.21.15.121:6817/3656724207","cluster_addr":"172.21.15.121:6819/3656724207","heartbeat_back_addr":"172.21.15.121:6827/3656724207","heartbeat_front_addr":"172.21.15.121:6822/3656724207","state":["exists","up"]},{"osd":3,"uuid":"dadb01bc-5491-4ffc-8824-d48a057240b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6823","nonce":2232342832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6826","nonce":2232342832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6831","nonce":2232342832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6829","nonce":2232342832}]},"public_addr":"172.21.15.121:6823/2232342832","cluster_addr":"172.21.15.121:6826/2232342832","heartbeat_back_addr":"172.21.15.121:6831/2232342832","heartbeat_front_addr":"172.21.15.121:6829/2232342832","state":["exists","up"]},{"osd":4,"uuid":"b3f3b4a1-b375-4526-b69e-344ab24c2da1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6801","nonce":99353111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6803","nonce":99353111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6807","nonce":99353111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6805","nonce":99353111}]},"public_addr":"172.21.15.123:6801/99353111","cluster_addr":"172.21.15.123:6803/99353111","heartbeat_back_addr":"172.21.15.123:6807/99353111","heartbeat_front_addr":"172.21.15.123:6805/99353111","state":["exists","up"]},{"osd":5,"uuid":"2ee24a08-1624-4961-9642-d7b510282484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6818","nonce":2504374688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6822","nonce":2504374688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6830","nonce":2504374688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6826","nonce":2504374688}]},"public_addr":"172.21.15.123:6818/2504374688","cluster_addr":"172.21.15.123:6822/2504374688","heartbeat_back_addr":"172.21.15.123:6830/2504374688","heartbeat_front_addr":"172.21.15.123:6826/2504374688","state":["exists","up"]},{"osd":6,"uuid":"186118cc-6128-49de-acca-55260945c1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6819","nonce":4139299599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6823","nonce":4139299599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6829","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6831","nonce":4139299599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6827","nonce":4139299599}]},"public_addr":"172.21.15.123:6819/4139299599","cluster_addr":"172.21.15.123:6823/4139299599","heartbeat_back_addr":"172.21.15.123:6831/4139299599","heartbeat_front_addr":"172.21.15.123:6827/4139299599","state":["exists","up"]},{"osd":7,"uuid":"a00aac6e-4bd9-415a-a93d-ae85210865ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6809","nonce":4043981069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6811","nonce":4043981069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6815","nonce":4043981069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6813","nonce":4043981069}]},"public_addr":"172.21.15.123:6809/4043981069","cluster_addr":"172.21.15.123:6811/4043981069","heartbeat_back_addr":"172.21.15.123:6815/4043981069","heartbeat_front_addr":"172.21.15.123:6813/4043981069","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.904540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.958585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.616367+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.885244+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.103127+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.167248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.884348+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.862356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:21:41.279 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.279+0000 7f31ce400640 1 -- 172.21.15.121:0/2828108208 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f31a806c920 msgr2=0x7f31a806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:41.279 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.279+0000 7f31ce400640 1 --2- 172.21.15.121:0/2828108208 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f31a806c920 0x7f31a806ede0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f31bc036000 tx=0x7f31bc02e000 comp rx=0 tx=0).stop 2024-04-20T10:21:41.279 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.279+0000 7f31ce400640 1 -- 172.21.15.121:0/2828108208 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31c81e4e80 msgr2=0x7f31c814ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:41.279 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.279+0000 7f31ce400640 1 --2- 172.21.15.121:0/2828108208 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31c81e4e80 0x7f31c814ca90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f31b400d7d0 tx=0x7f31b400dc90 comp rx=0 tx=0).stop 2024-04-20T10:21:41.280 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.279+0000 7f31ce400640 1 -- 172.21.15.121:0/2828108208 shutdown_connections 2024-04-20T10:21:41.280 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.279+0000 7f31ce400640 1 --2- 172.21.15.121:0/2828108208 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f31c81599f0 0x7f31c8159ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:41.280 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.279+0000 7f31ce400640 1 --2- 172.21.15.121:0/2828108208 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f31c81e4e80 0x7f31c814ca90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:41.280 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.279+0000 7f31ce400640 1 --2- 172.21.15.121:0/2828108208 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f31a806c920 0x7f31a806ede0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:41.280 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.279+0000 7f31ce400640 1 --2- 172.21.15.121:0/2828108208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f31b00039b0 0x7f31c815d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:41.280 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.279+0000 7f31ce400640 1 -- 172.21.15.121:0/2828108208 >> 172.21.15.121:0/2828108208 conn(0x7f31c80d0040 msgr2=0x7f31c809d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:41.280 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.280+0000 7f31ce400640 1 -- 172.21.15.121:0/2828108208 shutdown_connections 2024-04-20T10:21:41.280 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.280+0000 7f31ce400640 1 -- 172.21.15.121:0/2828108208 wait complete. 2024-04-20T10:21:41.297 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:20:51.141365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:21:00.436937+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:21:01.026957+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T10:21:04.862693+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T10:21:41.297 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:21:41.498 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.499+0000 7f7afba00640 1 Processor -- start 2024-04-20T10:21:41.498 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.499+0000 7f7afba00640 1 -- start start 2024-04-20T10:21:41.500 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.500+0000 7f7afba00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7af415e8a0 0x7f7af406e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:41.500 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.500+0000 7f7afba00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7af406dbb0 0x7f7af40d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:41.500 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.500+0000 7f7afba00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7af406f060 con 0x7f7af40d08a0 2024-04-20T10:21:41.500 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.500+0000 7f7afba00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f7af406e7c0 con 0x7f7af415e8a0 2024-04-20T10:21:41.500 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.500+0000 7f7afba00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f7af406eaa0 con 0x7f7af406dbb0 2024-04-20T10:21:41.500 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.501+0000 7f7af9a00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f7af40d08a0 0x7f7af406b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:47662/0 (socket says 172.21.15.121:47662) 2024-04-20T10:21:41.500 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.501+0000 7f7af9a00640 1 -- 172.21.15.121:0/616941830 learned_addr learned my addr 172.21.15.121:0/616941830 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:41.502 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.501+0000 7f7aebe00640 1 --2- 172.21.15.121:0/616941830 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7af406dbb0 0x7f7af40d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:41.502 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.501+0000 7f7af9000640 1 --2- 172.21.15.121:0/616941830 >> v2:172.21.15.123:3300/0 conn(0x7f7af415e8a0 0x7f7af406e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:41.502 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.501+0000 7f7af3e00640 1 -- 172.21.15.121:0/616941830 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1486342121 0 0) 0x7f7af406f060 con 0x7f7af40d08a0 2024-04-20T10:21:41.502 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.501+0000 7f7af3e00640 1 -- 172.21.15.121:0/616941830 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ad8003520 con 0x7f7af40d08a0 2024-04-20T10:21:41.502 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.501+0000 7f7af3e00640 1 -- 172.21.15.121:0/616941830 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1292252047 0 0) 0x7f7ad8003520 con 0x7f7af40d08a0 2024-04-20T10:21:41.502 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.501+0000 7f7af3e00640 1 -- 172.21.15.121:0/616941830 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7af406dbb0 msgr2=0x7f7af40d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:41.502 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.501+0000 7f7af3e00640 1 --2- 172.21.15.121:0/616941830 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7af406dbb0 0x7f7af40d0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:41.502 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.502+0000 7f7af3e00640 1 -- 172.21.15.121:0/616941830 >> v2:172.21.15.123:3300/0 conn(0x7f7af415e8a0 msgr2=0x7f7af406e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:41.502 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.502+0000 7f7af3e00640 1 --2- 172.21.15.121:0/616941830 >> v2:172.21.15.123:3300/0 conn(0x7f7af415e8a0 0x7f7af406e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:41.502 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.502+0000 7f7af3e00640 1 -- 172.21.15.121:0/616941830 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af406ed80 con 0x7f7af40d08a0 2024-04-20T10:21:41.502 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.502+0000 7f7af3e00640 1 -- 172.21.15.121:0/616941830 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f7aec0019f0 con 0x7f7af40d08a0 2024-04-20T10:21:41.503 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.502+0000 7f7af3e00640 1 -- 172.21.15.121:0/616941830 >> v1:172.21.15.121:6789/0 conn(0x7f7af40d08a0 legacy=0x7f7af406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:41.503 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.502+0000 7f7af3e00640 1 --2- 172.21.15.121:0/616941830 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7af406dbb0 0x7f7ad80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:41.503 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.502+0000 7f7af3e00640 1 --2- 172.21.15.121:0/616941830 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ad8003f30 0x7f7ad8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:41.503 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.502+0000 7f7af3e00640 1 --2- 172.21.15.121:0/616941830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ad8004d40 0x7f7ad80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:41.503 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.502+0000 7f7af3e00640 1 -- 172.21.15.121:0/616941830 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7af406e7c0 con 0x7f7ad8003f30 2024-04-20T10:21:41.503 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.502+0000 7f7af3e00640 1 -- 172.21.15.121:0/616941830 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7af406eaa0 con 0x7f7ad8004d40 2024-04-20T10:21:41.503 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.502+0000 7f7af3e00640 1 -- 172.21.15.121:0/616941830 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f7ad800ba10 con 0x7f7af406dbb0 2024-04-20T10:21:41.503 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.502+0000 7f7aebe00640 1 --2- 172.21.15.121:0/616941830 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ad8003f30 0x7f7ad8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:41.503 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.502+0000 7f7aebe00640 1 -- 172.21.15.121:0/616941830 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7af406dbb0 msgr2=0x7f7ad80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:41.503 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.502+0000 7f7aebe00640 1 --2- 172.21.15.121:0/616941830 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7af406dbb0 0x7f7ad80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:41.503 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.502+0000 7f7aebe00640 1 -- 172.21.15.121:0/616941830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ad8004d40 msgr2=0x7f7ad80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:41.503 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.502+0000 7f7aebe00640 1 --2- 172.21.15.121:0/616941830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ad8004d40 0x7f7ad80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:41.503 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.502+0000 7f7aebe00640 1 -- 172.21.15.121:0/616941830 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad8003520 con 0x7f7ad8003f30 2024-04-20T10:21:41.504 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.504+0000 7f7aebe00640 1 --2- 172.21.15.121:0/616941830 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ad8003f30 0x7f7ad8004800 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7adc00ba30 tx=0x7f7ad8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89521c6336453604 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:41.504 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.504+0000 7f7af3e00640 1 -- 172.21.15.121:0/616941830 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc036020 con 0x7f7ad8003f30 2024-04-20T10:21:41.505 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.504+0000 7f7af3e00640 1 -- 172.21.15.121:0/616941830 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7adc032680 con 0x7f7ad8003f30 2024-04-20T10:21:41.505 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.504+0000 7f7af3e00640 1 -- 172.21.15.121:0/616941830 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc00b600 con 0x7f7ad8003f30 2024-04-20T10:21:41.505 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.505+0000 7f7afba00640 1 -- 172.21.15.121:0/616941830 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ad8003f30 msgr2=0x7f7ad8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:41.505 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.505+0000 7f7afba00640 1 --2- 172.21.15.121:0/616941830 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ad8003f30 0x7f7ad8004800 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7adc00ba30 tx=0x7f7ad8005830 comp rx=0 tx=0).stop 2024-04-20T10:21:41.505 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.505+0000 7f7af9a00640 1 -- 172.21.15.121:0/616941830 reap_dead start 2024-04-20T10:21:41.505 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.505+0000 7f7afba00640 1 -- 172.21.15.121:0/616941830 shutdown_connections 2024-04-20T10:21:41.505 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.505+0000 7f7afba00640 1 -- 172.21.15.121:0/616941830 >> 172.21.15.121:0/616941830 conn(0x7f7af40b3350 msgr2=0x7f7af40b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:41.505 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.505+0000 7f7afba00640 1 -- 172.21.15.121:0/616941830 shutdown_connections 2024-04-20T10:21:41.505 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.505+0000 7f7afba00640 1 -- 172.21.15.121:0/616941830 wait complete. 2024-04-20T10:21:41.506 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.507+0000 7f7afba00640 1 Processor -- start 2024-04-20T10:21:41.506 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.507+0000 7f7afba00640 1 -- start start 2024-04-20T10:21:41.507 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.508+0000 7f7afba00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ad8003f30 0x7f7af4156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:41.507 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.508+0000 7f7afba00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7af415d9a0 0x7f7af4144c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:41.507 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.508+0000 7f7afba00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7af415de20 0x7f7af41451d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:41.507 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.508+0000 7f7afba00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7af406fef0 con 0x7f7ad8003f30 2024-04-20T10:21:41.508 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.508+0000 7f7afba00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7af406fd70 con 0x7f7af415d9a0 2024-04-20T10:21:41.508 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.508+0000 7f7afba00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f7af40ae690 con 0x7f7af415de20 2024-04-20T10:21:41.508 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.508+0000 7f7af9000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ad8003f30 0x7f7af4156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:41.508 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.508+0000 7f7af9000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ad8003f30 0x7f7af4156360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58736/0 (socket says 172.21.15.121:58736) 2024-04-20T10:21:41.508 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.508+0000 7f7af9000640 1 -- 172.21.15.121:0/3123980565 learned_addr learned my addr 172.21.15.121:0/3123980565 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:41.509 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.508+0000 7f7af9a00640 1 --2- 172.21.15.121:0/3123980565 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7af415de20 0x7f7af41451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:41.509 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.508+0000 7f7aebe00640 1 --2- 172.21.15.121:0/3123980565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7af415d9a0 0x7f7af4144c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:41.509 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.508+0000 7f7af9000640 1 -- 172.21.15.121:0/3123980565 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7af415de20 msgr2=0x7f7af41451d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:41.509 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.508+0000 7f7af9000640 1 --2- 172.21.15.121:0/3123980565 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7af415de20 0x7f7af41451d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:41.509 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.508+0000 7f7af9000640 1 -- 172.21.15.121:0/3123980565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7af415d9a0 msgr2=0x7f7af4144c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:41.509 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.508+0000 7f7af9000640 1 --2- 172.21.15.121:0/3123980565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7af415d9a0 0x7f7af4144c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:41.509 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.508+0000 7f7af9000640 1 -- 172.21.15.121:0/3123980565 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af4145830 con 0x7f7ad8003f30 2024-04-20T10:21:41.509 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.509+0000 7f7af9000640 1 --2- 172.21.15.121:0/3123980565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ad8003f30 0x7f7af4156360 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7ae400e890 tx=0x7f7ae400ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:41.510 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.509+0000 7f7af1600640 1 -- 172.21.15.121:0/3123980565 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4017070 con 0x7f7ad8003f30 2024-04-20T10:21:41.510 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.509+0000 7f7af1600640 1 -- 172.21.15.121:0/3123980565 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ae400c940 con 0x7f7ad8003f30 2024-04-20T10:21:41.510 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.509+0000 7f7af1600640 1 -- 172.21.15.121:0/3123980565 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae400cd10 con 0x7f7ad8003f30 2024-04-20T10:21:41.510 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.509+0000 7f7afba00640 1 -- 172.21.15.121:0/3123980565 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7af4145b10 con 0x7f7ad8003f30 2024-04-20T10:21:41.510 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.509+0000 7f7afba00640 1 -- 172.21.15.121:0/3123980565 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7af40aee40 con 0x7f7ad8003f30 2024-04-20T10:21:41.510 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.510+0000 7f7afba00640 1 -- 172.21.15.121:0/3123980565 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7af4068980 con 0x7f7ad8003f30 2024-04-20T10:21:41.514 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.511+0000 7f7af1600640 1 -- 172.21.15.121:0/3123980565 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ae401a070 con 0x7f7ad8003f30 2024-04-20T10:21:41.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.512+0000 7f7af1600640 1 --2- 172.21.15.121:0/3123980565 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7ad006c8f0 0x7f7ad006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:41.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.512+0000 7f7af1600640 1 -- 172.21.15.121:0/3123980565 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7ae408c0e0 con 0x7f7ad8003f30 2024-04-20T10:21:41.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.513+0000 7f7aebe00640 1 --2- 172.21.15.121:0/3123980565 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7ad006c8f0 0x7f7ad006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:41.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.513+0000 7f7aebe00640 1 --2- 172.21.15.121:0/3123980565 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7ad006c8f0 0x7f7ad006edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7adc00ba00 tx=0x7f7adc030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:41.516 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.517+0000 7f7af1600640 1 -- 172.21.15.121:0/3123980565 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7ae405a260 con 0x7f7ad8003f30 2024-04-20T10:21:41.719 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.719+0000 7f7afba00640 1 -- 172.21.15.121:0/3123980565 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7af406d160 con 0x7f7ad8003f30 2024-04-20T10:21:41.719 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.720+0000 7f7af1600640 1 -- 172.21.15.121:0/3123980565 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f7ae4059b60 con 0x7f7ad8003f30 2024-04-20T10:21:41.719 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2024-04-20T10:21:41.722 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.722+0000 7f7afba00640 1 -- 172.21.15.121:0/3123980565 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7ad006c8f0 msgr2=0x7f7ad006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:41.722 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.722+0000 7f7afba00640 1 --2- 172.21.15.121:0/3123980565 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7ad006c8f0 0x7f7ad006edb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7adc00ba00 tx=0x7f7adc030000 comp rx=0 tx=0).stop 2024-04-20T10:21:41.722 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.722+0000 7f7afba00640 1 -- 172.21.15.121:0/3123980565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ad8003f30 msgr2=0x7f7af4156360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:41.723 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.722+0000 7f7afba00640 1 --2- 172.21.15.121:0/3123980565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ad8003f30 0x7f7af4156360 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7ae400e890 tx=0x7f7ae400ed50 comp rx=0 tx=0).stop 2024-04-20T10:21:41.723 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.723+0000 7f7afba00640 1 -- 172.21.15.121:0/3123980565 shutdown_connections 2024-04-20T10:21:41.723 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.723+0000 7f7afba00640 1 --2- 172.21.15.121:0/3123980565 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7af415de20 0x7f7af41451d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:41.723 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.723+0000 7f7afba00640 1 --2- 172.21.15.121:0/3123980565 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7ad006c8f0 0x7f7ad006edb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:41.723 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.723+0000 7f7afba00640 1 --2- 172.21.15.121:0/3123980565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7af415d9a0 0x7f7af4144c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:41.723 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.723+0000 7f7afba00640 1 --2- 172.21.15.121:0/3123980565 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ad8003f30 0x7f7af4156360 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:41.723 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.723+0000 7f7afba00640 1 -- 172.21.15.121:0/3123980565 >> 172.21.15.121:0/3123980565 conn(0x7f7af40b3350 msgr2=0x7f7af4160c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:41.723 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.723+0000 7f7afba00640 1 -- 172.21.15.121:0/3123980565 shutdown_connections 2024-04-20T10:21:41.723 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.723+0000 7f7afba00640 1 -- 172.21.15.121:0/3123980565 wait complete. 2024-04-20T10:21:41.741 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:21:41.933 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.933+0000 7fdcc1000640 1 Processor -- start 2024-04-20T10:21:41.933 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.934+0000 7fdcc1000640 1 -- start start 2024-04-20T10:21:41.935 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.935+0000 7fdcc1000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdcbc071b50 0x7fdcbc0adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:41.935 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.935+0000 7fdcc1000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcbc06e840 con 0x7fdcbc15abb0 2024-04-20T10:21:41.935 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.935+0000 7fdcc1000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcbc15b350 con 0x7fdcbc072cc0 2024-04-20T10:21:41.935 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.935+0000 7fdcc1000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fdcbc06eb00 con 0x7fdcbc071b50 2024-04-20T10:21:41.935 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.935+0000 7fdcb9e00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fdcbc15abb0 0x7fdcbc0725b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:47672/0 (socket says 172.21.15.121:47672) 2024-04-20T10:21:41.935 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.935+0000 7fdcb9e00640 1 -- 172.21.15.121:0/3056442529 learned_addr learned my addr 172.21.15.121:0/3056442529 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:41.935 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.935+0000 7fdcba800640 1 --2- 172.21.15.121:0/3056442529 >> v2:172.21.15.121:3300/0 conn(0x7fdcbc071b50 0x7fdcbc0adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:41.936 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.936+0000 7fdcb8a00640 1 -- 172.21.15.121:0/3056442529 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3740400313 0 0) 0x7fdcbc06e840 con 0x7fdcbc15abb0 2024-04-20T10:21:41.936 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.936+0000 7fdcb8a00640 1 -- 172.21.15.121:0/3056442529 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdca0003440 con 0x7fdcbc15abb0 2024-04-20T10:21:41.936 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.936+0000 7fdcba800640 1 -- 172.21.15.121:0/3056442529 >> v1:172.21.15.123:6789/0 conn(0x7fdcbc072cc0 legacy=0x7fdcbc0ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:41.936 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.936+0000 7fdcba800640 1 -- 172.21.15.121:0/3056442529 >> v1:172.21.15.121:6789/0 conn(0x7fdcbc15abb0 legacy=0x7fdcbc0725b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:41.936 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.936+0000 7fdcba800640 1 -- 172.21.15.121:0/3056442529 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcbc15b630 con 0x7fdcbc071b50 2024-04-20T10:21:41.937 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.937+0000 7fdcba800640 1 --2- 172.21.15.121:0/3056442529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdcbc071b50 0x7fdcbc0adfb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4009d90 tx=0x7fdcb402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da1c59458e087a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:41.937 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.937+0000 7fdcb8a00640 1 -- 172.21.15.121:0/3056442529 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4039070 con 0x7fdcbc071b50 2024-04-20T10:21:41.937 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.937+0000 7fdcb8a00640 1 -- 172.21.15.121:0/3056442529 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcb402cd00 con 0x7fdcbc071b50 2024-04-20T10:21:41.937 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.937+0000 7fdcb8a00640 1 -- 172.21.15.121:0/3056442529 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb4034660 con 0x7fdcbc071b50 2024-04-20T10:21:41.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.938+0000 7fdcc1000640 1 -- 172.21.15.121:0/3056442529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdcbc071b50 msgr2=0x7fdcbc0adfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:41.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.938+0000 7fdcc1000640 1 --2- 172.21.15.121:0/3056442529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdcbc071b50 0x7fdcbc0adfb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4009d90 tx=0x7fdcb402bf50 comp rx=0 tx=0).stop 2024-04-20T10:21:41.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.938+0000 7fdcc1000640 1 -- 172.21.15.121:0/3056442529 shutdown_connections 2024-04-20T10:21:41.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.938+0000 7fdcc1000640 1 --2- 172.21.15.121:0/3056442529 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdcbc071b50 0x7fdcbc0adfb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:41.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.938+0000 7fdcc1000640 1 -- 172.21.15.121:0/3056442529 >> 172.21.15.121:0/3056442529 conn(0x7fdcbc0b3350 msgr2=0x7fdcbc0b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:41.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.938+0000 7fdcc1000640 1 -- 172.21.15.121:0/3056442529 shutdown_connections 2024-04-20T10:21:41.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.938+0000 7fdcc1000640 1 -- 172.21.15.121:0/3056442529 wait complete. 2024-04-20T10:21:41.939 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.940+0000 7fdcc1000640 1 Processor -- start 2024-04-20T10:21:41.939 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.940+0000 7fdcc1000640 1 -- start start 2024-04-20T10:21:41.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.941+0000 7fdcc1000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcbc071b50 0x7fdcbc15dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:41.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.941+0000 7fdcc1000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdcbc072cc0 0x7fdcbc0ce8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:41.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.941+0000 7fdcc1000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdcbc15abb0 0x7fdcbc09d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:41.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.941+0000 7fdcc1000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdcbc15bbf0 con 0x7fdcbc072cc0 2024-04-20T10:21:41.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.941+0000 7fdcc1000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdcbc15bf60 con 0x7fdcbc071b50 2024-04-20T10:21:41.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.941+0000 7fdcc1000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fdcbc15af90 con 0x7fdcbc15abb0 2024-04-20T10:21:41.941 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.941+0000 7fdcb9400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdcbc072cc0 0x7fdcbc0ce8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:41.941 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.941+0000 7fdcb9400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdcbc072cc0 0x7fdcbc0ce8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58758/0 (socket says 172.21.15.121:58758) 2024-04-20T10:21:41.941 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.941+0000 7fdcb9400640 1 -- 172.21.15.121:0/825571193 learned_addr learned my addr 172.21.15.121:0/825571193 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:41.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.941+0000 7fdcb9400640 1 -- 172.21.15.121:0/825571193 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdcbc15abb0 msgr2=0x7fdcbc09d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:41.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.941+0000 7fdcb9400640 1 --2- 172.21.15.121:0/825571193 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdcbc15abb0 0x7fdcbc09d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:41.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.941+0000 7fdcb9400640 1 -- 172.21.15.121:0/825571193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcbc071b50 msgr2=0x7fdcbc15dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:41.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.941+0000 7fdcb9400640 1 --2- 172.21.15.121:0/825571193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcbc071b50 0x7fdcbc15dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:41.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.941+0000 7fdcb9400640 1 -- 172.21.15.121:0/825571193 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcbc09dc40 con 0x7fdcbc072cc0 2024-04-20T10:21:41.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.942+0000 7fdcb9400640 1 --2- 172.21.15.121:0/825571193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdcbc072cc0 0x7fdcbc0ce8d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdca800c400 tx=0x7fdca800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:41.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.942+0000 7fdcb2000640 1 -- 172.21.15.121:0/825571193 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca80052d0 con 0x7fdcbc072cc0 2024-04-20T10:21:41.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.942+0000 7fdcb2000640 1 -- 172.21.15.121:0/825571193 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdca8005490 con 0x7fdcbc072cc0 2024-04-20T10:21:41.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.942+0000 7fdcc1000640 1 -- 172.21.15.121:0/825571193 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcbc09df20 con 0x7fdcbc072cc0 2024-04-20T10:21:41.943 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.942+0000 7fdcb2000640 1 -- 172.21.15.121:0/825571193 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdca8007260 con 0x7fdcbc072cc0 2024-04-20T10:21:41.943 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.942+0000 7fdcc1000640 1 -- 172.21.15.121:0/825571193 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcbc15d690 con 0x7fdcbc072cc0 2024-04-20T10:21:41.943 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.943+0000 7fdcc1000640 1 -- 172.21.15.121:0/825571193 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcbc068980 con 0x7fdcbc072cc0 2024-04-20T10:21:41.943 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.944+0000 7fdcb2000640 1 -- 172.21.15.121:0/825571193 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdca8007400 con 0x7fdcbc072cc0 2024-04-20T10:21:41.945 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.945+0000 7fdcb2000640 1 --2- 172.21.15.121:0/825571193 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fdc9c06c8f0 0x7fdc9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:41.945 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.946+0000 7fdcb2000640 1 -- 172.21.15.121:0/825571193 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fdca808b420 con 0x7fdcbc072cc0 2024-04-20T10:21:41.945 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.946+0000 7fdcb9e00640 1 --2- 172.21.15.121:0/825571193 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fdc9c06c8f0 0x7fdc9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:41.945 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.946+0000 7fdcb9e00640 1 --2- 172.21.15.121:0/825571193 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fdc9c06c8f0 0x7fdc9c06edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdcac001570 tx=0x7fdcac007930 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:41.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:41.950+0000 7fdcb2000640 1 -- 172.21.15.121:0/825571193 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdca8059510 con 0x7fdcbc072cc0 2024-04-20T10:21:42.148 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.148+0000 7fdcc1000640 1 -- 172.21.15.121:0/825571193 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdcbc0b2bd0 con 0x7fdcbc072cc0 2024-04-20T10:21:42.149 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.149+0000 7fdcb2000640 1 -- 172.21.15.121:0/825571193 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7fdca8058e10 con 0x7fdcbc072cc0 2024-04-20T10:21:42.149 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 32 2024-04-20T10:21:42.151 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.152+0000 7fdcc1000640 1 -- 172.21.15.121:0/825571193 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fdc9c06c8f0 msgr2=0x7fdc9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:42.151 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.152+0000 7fdcc1000640 1 --2- 172.21.15.121:0/825571193 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fdc9c06c8f0 0x7fdc9c06edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdcac001570 tx=0x7fdcac007930 comp rx=0 tx=0).stop 2024-04-20T10:21:42.152 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.152+0000 7fdcc1000640 1 -- 172.21.15.121:0/825571193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdcbc072cc0 msgr2=0x7fdcbc0ce8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:42.152 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.152+0000 7fdcc1000640 1 --2- 172.21.15.121:0/825571193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdcbc072cc0 0x7fdcbc0ce8d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdca800c400 tx=0x7fdca800c8c0 comp rx=0 tx=0).stop 2024-04-20T10:21:42.152 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.152+0000 7fdcc1000640 1 -- 172.21.15.121:0/825571193 shutdown_connections 2024-04-20T10:21:42.152 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.152+0000 7fdcc1000640 1 --2- 172.21.15.121:0/825571193 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdcbc15abb0 0x7fdcbc09d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:42.152 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.152+0000 7fdcc1000640 1 --2- 172.21.15.121:0/825571193 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdcbc072cc0 0x7fdcbc0ce8d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:42.152 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.152+0000 7fdcc1000640 1 --2- 172.21.15.121:0/825571193 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fdc9c06c8f0 0x7fdc9c06edb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:42.152 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.152+0000 7fdcc1000640 1 --2- 172.21.15.121:0/825571193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcbc071b50 0x7fdcbc15dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:42.152 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.152+0000 7fdcc1000640 1 -- 172.21.15.121:0/825571193 >> 172.21.15.121:0/825571193 conn(0x7fdcbc0b3350 msgr2=0x7fdcbc06b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:42.152 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.152+0000 7fdcc1000640 1 -- 172.21.15.121:0/825571193 shutdown_connections 2024-04-20T10:21:42.152 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.152+0000 7fdcc1000640 1 -- 172.21.15.121:0/825571193 wait complete. 2024-04-20T10:21:42.171 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:21:42.357 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.358+0000 7fb210600640 1 Processor -- start 2024-04-20T10:21:42.357 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.358+0000 7fb210600640 1 -- start start 2024-04-20T10:21:42.359 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.359+0000 7fb210600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb20815b3b0 0x7fb20806eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:42.359 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.359+0000 7fb210600640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb20806dcb0 0x7fb208071b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:42.360 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.359+0000 7fb210600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb208072090 0x7fb2080d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:42.360 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.359+0000 7fb210600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fb2080ade40 con 0x7fb208072090 2024-04-20T10:21:42.360 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.359+0000 7fb210600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fb2080ae0e0 con 0x7fb20815b3b0 2024-04-20T10:21:42.360 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.359+0000 7fb210600640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fb20815b790 con 0x7fb20806dcb0 2024-04-20T10:21:42.360 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.360+0000 7fb20e600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb208072090 0x7fb2080d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:42.360 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.360+0000 7fb20e600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb208072090 0x7fb2080d0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58764/0 (socket says 172.21.15.121:58764) 2024-04-20T10:21:42.360 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.360+0000 7fb20e600640 1 -- 172.21.15.121:0/4058807631 learned_addr learned my addr 172.21.15.121:0/4058807631 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:42.360 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.360+0000 7fb20d200640 1 --2- 172.21.15.121:0/4058807631 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb20806dcb0 0x7fb208071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:42.360 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.360+0000 7fb20dc00640 1 --2- 172.21.15.121:0/4058807631 >> v2:172.21.15.123:3300/0 conn(0x7fb20815b3b0 0x7fb20806eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:42.360 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.360+0000 7fb20e600640 1 -- 172.21.15.121:0/4058807631 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb20806dcb0 msgr2=0x7fb208071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:42.360 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.360+0000 7fb20e600640 1 --2- 172.21.15.121:0/4058807631 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb20806dcb0 0x7fb208071b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:42.360 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.360+0000 7fb20e600640 1 -- 172.21.15.121:0/4058807631 >> v2:172.21.15.123:3300/0 conn(0x7fb20815b3b0 msgr2=0x7fb20806eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:42.360 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.360+0000 7fb20e600640 1 --2- 172.21.15.121:0/4058807631 >> v2:172.21.15.123:3300/0 conn(0x7fb20815b3b0 0x7fb20806eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:42.360 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.360+0000 7fb20e600640 1 -- 172.21.15.121:0/4058807631 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2080ae380 con 0x7fb208072090 2024-04-20T10:21:42.361 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.361+0000 7fb20e600640 1 --2- 172.21.15.121:0/4058807631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb208072090 0x7fb2080d0360 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb20806cfa0 tx=0x7fb20002e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a76aef06210019 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:42.362 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.361+0000 7fb207e00640 1 -- 172.21.15.121:0/4058807631 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20002ea80 con 0x7fb208072090 2024-04-20T10:21:42.362 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.361+0000 7fb207e00640 1 -- 172.21.15.121:0/4058807631 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb20002ec20 con 0x7fb208072090 2024-04-20T10:21:42.362 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.361+0000 7fb207e00640 1 -- 172.21.15.121:0/4058807631 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb200009600 con 0x7fb208072090 2024-04-20T10:21:42.362 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.362+0000 7fb210600640 1 -- 172.21.15.121:0/4058807631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb208072090 msgr2=0x7fb2080d0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:42.362 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.362+0000 7fb210600640 1 --2- 172.21.15.121:0/4058807631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb208072090 0x7fb2080d0360 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb20806cfa0 tx=0x7fb20002e0e0 comp rx=0 tx=0).stop 2024-04-20T10:21:42.362 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.362+0000 7fb210600640 1 -- 172.21.15.121:0/4058807631 shutdown_connections 2024-04-20T10:21:42.362 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.362+0000 7fb210600640 1 --2- 172.21.15.121:0/4058807631 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb208072090 0x7fb2080d0360 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:42.362 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.362+0000 7fb210600640 1 --2- 172.21.15.121:0/4058807631 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb20806dcb0 0x7fb208071b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:42.362 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.362+0000 7fb210600640 1 --2- 172.21.15.121:0/4058807631 >> v2:172.21.15.123:3300/0 conn(0x7fb20815b3b0 0x7fb20806eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:42.362 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.362+0000 7fb210600640 1 -- 172.21.15.121:0/4058807631 >> 172.21.15.121:0/4058807631 conn(0x7fb2080b3350 msgr2=0x7fb2080b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:42.362 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.362+0000 7fb210600640 1 -- 172.21.15.121:0/4058807631 shutdown_connections 2024-04-20T10:21:42.362 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.362+0000 7fb210600640 1 -- 172.21.15.121:0/4058807631 wait complete. 2024-04-20T10:21:42.363 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.364+0000 7fb210600640 1 Processor -- start 2024-04-20T10:21:42.363 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.364+0000 7fb210600640 1 -- start start 2024-04-20T10:21:42.366 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.365+0000 7fb210600640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb20806dcb0 0x7fb20815cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:42.366 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.365+0000 7fb210600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb20815b3b0 0x7fb2081e4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:42.366 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.365+0000 7fb210600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2081494c0 0x7fb2081568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:42.367 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.365+0000 7fb210600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb2080aec20 con 0x7fb20815b3b0 2024-04-20T10:21:42.367 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.365+0000 7fb210600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb2080ae940 con 0x7fb2081494c0 2024-04-20T10:21:42.367 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.365+0000 7fb210600640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fb20806efe0 con 0x7fb20806dcb0 2024-04-20T10:21:42.367 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.365+0000 7fb20d200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb20815b3b0 0x7fb2081e4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:42.367 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.365+0000 7fb20d200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb20815b3b0 0x7fb2081e4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58778/0 (socket says 172.21.15.121:58778) 2024-04-20T10:21:42.367 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.365+0000 7fb20d200640 1 -- 172.21.15.121:0/3206543624 learned_addr learned my addr 172.21.15.121:0/3206543624 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:42.367 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.365+0000 7fb20e600640 1 --2- 172.21.15.121:0/3206543624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2081494c0 0x7fb2081568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:42.367 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.366+0000 7fb20d200640 1 -- 172.21.15.121:0/3206543624 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb20806dcb0 msgr2=0x7fb20815cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:42.367 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.366+0000 7fb20d200640 1 --2- 172.21.15.121:0/3206543624 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb20806dcb0 0x7fb20815cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:42.367 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.366+0000 7fb20d200640 1 -- 172.21.15.121:0/3206543624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2081494c0 msgr2=0x7fb2081568f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:42.367 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.366+0000 7fb20d200640 1 --2- 172.21.15.121:0/3206543624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2081494c0 0x7fb2081568f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:42.367 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.366+0000 7fb20d200640 1 -- 172.21.15.121:0/3206543624 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb20815d960 con 0x7fb20815b3b0 2024-04-20T10:21:42.367 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.366+0000 7fb20e600640 1 --2- 172.21.15.121:0/3206543624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2081494c0 0x7fb2081568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:42.368 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.366+0000 7fb20d200640 1 --2- 172.21.15.121:0/3206543624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb20815b3b0 0x7fb2081e4d70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb1f400d800 tx=0x7fb1f400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:42.368 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.366+0000 7fb205600640 1 -- 172.21.15.121:0/3206543624 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f4018020 con 0x7fb20815b3b0 2024-04-20T10:21:42.368 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.366+0000 7fb205600640 1 -- 172.21.15.121:0/3206543624 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1f40096a0 con 0x7fb20815b3b0 2024-04-20T10:21:42.368 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.366+0000 7fb205600640 1 -- 172.21.15.121:0/3206543624 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f4009a70 con 0x7fb20815b3b0 2024-04-20T10:21:42.368 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.366+0000 7fb210600640 1 -- 172.21.15.121:0/3206543624 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb20815dc40 con 0x7fb20815b3b0 2024-04-20T10:21:42.368 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.366+0000 7fb210600640 1 -- 172.21.15.121:0/3206543624 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb20815e200 con 0x7fb20815b3b0 2024-04-20T10:21:42.369 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.367+0000 7fb210600640 1 -- 172.21.15.121:0/3206543624 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb208068980 con 0x7fb20815b3b0 2024-04-20T10:21:42.369 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.368+0000 7fb205600640 1 -- 172.21.15.121:0/3206543624 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb1f401b020 con 0x7fb20815b3b0 2024-04-20T10:21:42.369 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.369+0000 7fb205600640 1 --2- 172.21.15.121:0/3206543624 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fb1e806c920 0x7fb1e806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:42.369 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.369+0000 7fb205600640 1 -- 172.21.15.121:0/3206543624 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb1f408c0e0 con 0x7fb20815b3b0 2024-04-20T10:21:42.369 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.369+0000 7fb20dc00640 1 --2- 172.21.15.121:0/3206543624 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fb1e806c920 0x7fb1e806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:42.370 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.370+0000 7fb20dc00640 1 --2- 172.21.15.121:0/3206543624 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fb1e806c920 0x7fb1e806ede0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc006fd0 tx=0x7fb1fc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:42.373 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.373+0000 7fb205600640 1 -- 172.21.15.121:0/3206543624 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb1f4022030 con 0x7fb20815b3b0 2024-04-20T10:21:42.564 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.565+0000 7fb210600640 1 -- 172.21.15.121:0/3206543624 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb2080738d0 con 0x7fb20815b3b0 2024-04-20T10:21:42.565 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.566+0000 7fb205600640 1 -- 172.21.15.121:0/3206543624 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7fb1f405a1c0 con 0x7fb20815b3b0 2024-04-20T10:21:42.565 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2024-04-20T10:21:42.568 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.568+0000 7fb210600640 1 -- 172.21.15.121:0/3206543624 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fb1e806c920 msgr2=0x7fb1e806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:42.568 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.568+0000 7fb210600640 1 --2- 172.21.15.121:0/3206543624 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fb1e806c920 0x7fb1e806ede0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc006fd0 tx=0x7fb1fc008040 comp rx=0 tx=0).stop 2024-04-20T10:21:42.568 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.568+0000 7fb210600640 1 -- 172.21.15.121:0/3206543624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb20815b3b0 msgr2=0x7fb2081e4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:42.568 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.568+0000 7fb210600640 1 --2- 172.21.15.121:0/3206543624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb20815b3b0 0x7fb2081e4d70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fb1f400d800 tx=0x7fb1f400dcc0 comp rx=0 tx=0).stop 2024-04-20T10:21:42.568 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.568+0000 7fb210600640 1 -- 172.21.15.121:0/3206543624 shutdown_connections 2024-04-20T10:21:42.568 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.568+0000 7fb210600640 1 --2- 172.21.15.121:0/3206543624 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fb1e806c920 0x7fb1e806ede0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:42.568 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.568+0000 7fb210600640 1 --2- 172.21.15.121:0/3206543624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2081494c0 0x7fb2081568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:42.568 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.568+0000 7fb210600640 1 --2- 172.21.15.121:0/3206543624 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb20815b3b0 0x7fb2081e4d70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:42.569 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.568+0000 7fb210600640 1 --2- 172.21.15.121:0/3206543624 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb20806dcb0 0x7fb20815cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:42.569 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.568+0000 7fb210600640 1 -- 172.21.15.121:0/3206543624 >> 172.21.15.121:0/3206543624 conn(0x7fb2080b3350 msgr2=0x7fb208158180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:42.569 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.568+0000 7fb210600640 1 -- 172.21.15.121:0/3206543624 shutdown_connections 2024-04-20T10:21:42.569 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.569+0000 7fb210600640 1 -- 172.21.15.121:0/3206543624 wait complete. 2024-04-20T10:21:42.586 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T10:21:42.791 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.792+0000 7f54ea000640 1 Processor -- start 2024-04-20T10:21:42.791 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.792+0000 7f54ea000640 1 -- start start 2024-04-20T10:21:42.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.793+0000 7f54ea000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f54e4165190 0x7f54e4167580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:42.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.793+0000 7f54ea000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54e406b900 con 0x7f54e415d4f0 2024-04-20T10:21:42.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.793+0000 7f54ea000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54e406bbe0 con 0x7f54e40aee60 2024-04-20T10:21:42.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.793+0000 7f54ea000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f54e406c510 con 0x7f54e4165190 2024-04-20T10:21:42.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.794+0000 7f54dbe00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f54e415d4f0 0x7f54e40ad0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:47686/0 (socket says 172.21.15.121:47686) 2024-04-20T10:21:42.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.794+0000 7f54dbe00640 1 -- 172.21.15.121:0/3720927517 learned_addr learned my addr 172.21.15.121:0/3720927517 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:42.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.794+0000 7f54e9600640 1 --2- 172.21.15.121:0/3720927517 >> v2:172.21.15.121:3300/0 conn(0x7f54e4165190 0x7f54e4167580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:42.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.794+0000 7f54e3e00640 1 -- 172.21.15.121:0/3720927517 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1682250382 0 0) 0x7f54e406b900 con 0x7f54e415d4f0 2024-04-20T10:21:42.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.794+0000 7f54e3e00640 1 -- 172.21.15.121:0/3720927517 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f54c8003520 con 0x7f54e415d4f0 2024-04-20T10:21:42.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.794+0000 7f54e9600640 1 -- 172.21.15.121:0/3720927517 >> v1:172.21.15.123:6789/0 conn(0x7f54e40aee60 legacy=0x7f54e40af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:42.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.794+0000 7f54e9600640 1 -- 172.21.15.121:0/3720927517 >> v1:172.21.15.121:6789/0 conn(0x7f54e415d4f0 legacy=0x7f54e40ad0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:42.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.794+0000 7f54e9600640 1 -- 172.21.15.121:0/3720927517 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e406bec0 con 0x7f54e4165190 2024-04-20T10:21:42.795 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.795+0000 7f54e9600640 1 --2- 172.21.15.121:0/3720927517 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54e4165190 0x7f54e4167580 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f54dc009d90 tx=0x7f54dc02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5198973fab829126 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:42.795 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.796+0000 7f54e3e00640 1 -- 172.21.15.121:0/3720927517 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc039070 con 0x7f54e4165190 2024-04-20T10:21:42.795 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.796+0000 7f54e3e00640 1 -- 172.21.15.121:0/3720927517 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54dc02cd00 con 0x7f54e4165190 2024-04-20T10:21:42.795 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.796+0000 7f54e3e00640 1 -- 172.21.15.121:0/3720927517 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc034660 con 0x7f54e4165190 2024-04-20T10:21:42.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.796+0000 7f54ea000640 1 -- 172.21.15.121:0/3720927517 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54e4165190 msgr2=0x7f54e4167580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:42.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.796+0000 7f54ea000640 1 --2- 172.21.15.121:0/3720927517 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54e4165190 0x7f54e4167580 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f54dc009d90 tx=0x7f54dc02bf50 comp rx=0 tx=0).stop 2024-04-20T10:21:42.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.796+0000 7f54ea000640 1 -- 172.21.15.121:0/3720927517 shutdown_connections 2024-04-20T10:21:42.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.796+0000 7f54ea000640 1 --2- 172.21.15.121:0/3720927517 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54e4165190 0x7f54e4167580 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:42.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.796+0000 7f54ea000640 1 -- 172.21.15.121:0/3720927517 >> 172.21.15.121:0/3720927517 conn(0x7f54e40b5270 msgr2=0x7f54e40b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:42.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.797+0000 7f54ea000640 1 -- 172.21.15.121:0/3720927517 shutdown_connections 2024-04-20T10:21:42.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.797+0000 7f54ea000640 1 -- 172.21.15.121:0/3720927517 wait complete. 2024-04-20T10:21:42.798 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.798+0000 7f54ea000640 1 Processor -- start 2024-04-20T10:21:42.798 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.798+0000 7f54ea000640 1 -- start start 2024-04-20T10:21:42.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.799+0000 7f54ea000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54e40aee60 0x7f54e414b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:42.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.799+0000 7f54ea000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54e415d4f0 0x7f54e4167520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:42.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.799+0000 7f54ea000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f54e4165190 0x7f54e4099560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:42.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.799+0000 7f54ea000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f54e406c9a0 con 0x7f54e415d4f0 2024-04-20T10:21:42.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.799+0000 7f54ea000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f54e406f070 con 0x7f54e40aee60 2024-04-20T10:21:42.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.799+0000 7f54ea000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f54e4167ac0 con 0x7f54e4165190 2024-04-20T10:21:42.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.799+0000 7f54dbe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54e415d4f0 0x7f54e4167520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:42.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.800+0000 7f54dbe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54e415d4f0 0x7f54e4167520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58792/0 (socket says 172.21.15.121:58792) 2024-04-20T10:21:42.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.800+0000 7f54dbe00640 1 -- 172.21.15.121:0/244043611 learned_addr learned my addr 172.21.15.121:0/244043611 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:42.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.800+0000 7f54e9600640 1 --2- 172.21.15.121:0/244043611 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f54e4165190 0x7f54e4099560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:42.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.800+0000 7f54e8c00640 1 --2- 172.21.15.121:0/244043611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54e40aee60 0x7f54e414b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:42.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.800+0000 7f54dbe00640 1 -- 172.21.15.121:0/244043611 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f54e4165190 msgr2=0x7f54e4099560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:42.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.800+0000 7f54dbe00640 1 --2- 172.21.15.121:0/244043611 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f54e4165190 0x7f54e4099560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:42.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.800+0000 7f54dbe00640 1 -- 172.21.15.121:0/244043611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54e40aee60 msgr2=0x7f54e414b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:42.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.800+0000 7f54dbe00640 1 --2- 172.21.15.121:0/244043611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54e40aee60 0x7f54e414b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:42.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.800+0000 7f54dbe00640 1 -- 172.21.15.121:0/244043611 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e4166480 con 0x7f54e415d4f0 2024-04-20T10:21:42.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.800+0000 7f54e8c00640 1 --2- 172.21.15.121:0/244043611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54e40aee60 0x7f54e414b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:42.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.800+0000 7f54dbe00640 1 --2- 172.21.15.121:0/244043611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54e415d4f0 0x7f54e4167520 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f54cc00c500 tx=0x7f54cc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:42.801 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.801+0000 7f54e1600640 1 -- 172.21.15.121:0/244043611 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc017070 con 0x7f54e415d4f0 2024-04-20T10:21:42.801 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.801+0000 7f54e1600640 1 -- 172.21.15.121:0/244043611 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54cc005ce0 con 0x7f54e415d4f0 2024-04-20T10:21:42.801 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.801+0000 7f54e1600640 1 -- 172.21.15.121:0/244043611 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc007330 con 0x7f54e415d4f0 2024-04-20T10:21:42.801 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.801+0000 7f54ea000640 1 -- 172.21.15.121:0/244043611 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54e4166760 con 0x7f54e415d4f0 2024-04-20T10:21:42.801 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.801+0000 7f54ea000640 1 -- 172.21.15.121:0/244043611 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54e4166d20 con 0x7f54e415d4f0 2024-04-20T10:21:42.801 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.802+0000 7f54ea000640 1 -- 172.21.15.121:0/244043611 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54e4075b30 con 0x7f54e415d4f0 2024-04-20T10:21:42.802 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.803+0000 7f54e1600640 1 -- 172.21.15.121:0/244043611 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f54cc01a070 con 0x7f54e415d4f0 2024-04-20T10:21:42.803 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.804+0000 7f54e1600640 1 --2- 172.21.15.121:0/244043611 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f54c006c920 0x7f54c006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:42.803 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.804+0000 7f54e1600640 1 -- 172.21.15.121:0/244043611 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f54cc08b340 con 0x7f54e415d4f0 2024-04-20T10:21:42.803 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.804+0000 7f54e8c00640 1 --2- 172.21.15.121:0/244043611 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f54c006c920 0x7f54c006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:42.804 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.805+0000 7f54e8c00640 1 --2- 172.21.15.121:0/244043611 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f54c006c920 0x7f54c006ede0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f54d40015e0 tx=0x7f54d4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:42.807 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.808+0000 7f54e1600640 1 -- 172.21.15.121:0/244043611 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f54cc0594e0 con 0x7f54e415d4f0 2024-04-20T10:21:42.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.993+0000 7f54ea000640 1 -- 172.21.15.121:0/244043611 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f54e406de40 con 0x7f54e415d4f0 2024-04-20T10:21:42.993 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.994+0000 7f54e1600640 1 -- 172.21.15.121:0/244043611 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f54cc058de0 con 0x7f54e415d4f0 2024-04-20T10:21:42.994 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 32 2024-04-20T10:21:42.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.996+0000 7f54ea000640 1 -- 172.21.15.121:0/244043611 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f54c006c920 msgr2=0x7f54c006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:42.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.996+0000 7f54ea000640 1 --2- 172.21.15.121:0/244043611 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f54c006c920 0x7f54c006ede0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f54d40015e0 tx=0x7f54d4009040 comp rx=0 tx=0).stop 2024-04-20T10:21:42.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.996+0000 7f54ea000640 1 -- 172.21.15.121:0/244043611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54e415d4f0 msgr2=0x7f54e4167520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:42.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.996+0000 7f54ea000640 1 --2- 172.21.15.121:0/244043611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54e415d4f0 0x7f54e4167520 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f54cc00c500 tx=0x7f54cc00c9c0 comp rx=0 tx=0).stop 2024-04-20T10:21:42.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.996+0000 7f54ea000640 1 -- 172.21.15.121:0/244043611 shutdown_connections 2024-04-20T10:21:42.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.996+0000 7f54ea000640 1 --2- 172.21.15.121:0/244043611 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f54e4165190 0x7f54e4099560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:42.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.996+0000 7f54ea000640 1 --2- 172.21.15.121:0/244043611 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54e415d4f0 0x7f54e4167520 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:42.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.996+0000 7f54ea000640 1 --2- 172.21.15.121:0/244043611 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f54c006c920 0x7f54c006ede0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:42.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.996+0000 7f54ea000640 1 --2- 172.21.15.121:0/244043611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54e40aee60 0x7f54e414b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:42.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.996+0000 7f54ea000640 1 -- 172.21.15.121:0/244043611 >> 172.21.15.121:0/244043611 conn(0x7f54e40b5270 msgr2=0x7f54e40b04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:42.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.997+0000 7f54ea000640 1 -- 172.21.15.121:0/244043611 shutdown_connections 2024-04-20T10:21:42.997 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:42.997+0000 7f54ea000640 1 -- 172.21.15.121:0/244043611 wait complete. 2024-04-20T10:21:43.013 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:21:43.213 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.214+0000 7f7ddc800640 1 Processor -- start 2024-04-20T10:21:43.214 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.214+0000 7f7ddc800640 1 -- start start 2024-04-20T10:21:43.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.215+0000 7f7ddc800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7dd4156c20 0x7f7dd406b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:43.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.215+0000 7f7ddc800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7dd415ca40 0x7f7dd40ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:43.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.215+0000 7f7ddc800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7dd40ad8e0 0x7f7dd40d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:43.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.215+0000 7f7ddc800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f7dd4141c60 con 0x7f7dd40ad8e0 2024-04-20T10:21:43.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.215+0000 7f7ddc800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f7dd415cf90 con 0x7f7dd415ca40 2024-04-20T10:21:43.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.215+0000 7f7ddc800640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f7dd415d270 con 0x7f7dd4156c20 2024-04-20T10:21:43.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.216+0000 7f7dda800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7dd40ad8e0 0x7f7dd40d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:43.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.216+0000 7f7dda800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7dd40ad8e0 0x7f7dd40d0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58808/0 (socket says 172.21.15.121:58808) 2024-04-20T10:21:43.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.216+0000 7f7dda800640 1 -- 172.21.15.121:0/1125794261 learned_addr learned my addr 172.21.15.121:0/1125794261 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:43.216 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.216+0000 7f7dd9e00640 1 --2- 172.21.15.121:0/1125794261 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7dd4156c20 0x7f7dd406b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:43.216 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.216+0000 7f7dda800640 1 -- 172.21.15.121:0/1125794261 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7dd4156c20 msgr2=0x7f7dd406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:43.217 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.216+0000 7f7dd9400640 1 --2- 172.21.15.121:0/1125794261 >> v2:172.21.15.123:3300/0 conn(0x7f7dd415ca40 0x7f7dd40ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:43.217 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.216+0000 7f7dda800640 1 --2- 172.21.15.121:0/1125794261 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7dd4156c20 0x7f7dd406b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:43.217 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.216+0000 7f7dda800640 1 -- 172.21.15.121:0/1125794261 >> v2:172.21.15.123:3300/0 conn(0x7f7dd415ca40 msgr2=0x7f7dd40ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:43.217 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.216+0000 7f7dda800640 1 --2- 172.21.15.121:0/1125794261 >> v2:172.21.15.123:3300/0 conn(0x7f7dd415ca40 0x7f7dd40ad3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:43.217 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.216+0000 7f7dda800640 1 -- 172.21.15.121:0/1125794261 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd415d550 con 0x7f7dd40ad8e0 2024-04-20T10:21:43.217 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.216+0000 7f7dd9e00640 1 --2- 172.21.15.121:0/1125794261 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7dd4156c20 0x7f7dd406b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:43.217 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.217+0000 7f7dda800640 1 --2- 172.21.15.121:0/1125794261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7dd40ad8e0 0x7f7dd40d0360 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7dd406fe70 tx=0x7f7dd002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=874b721b37352320 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:43.218 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.218+0000 7f7dd8a00640 1 -- 172.21.15.121:0/1125794261 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd002ca70 con 0x7f7dd40ad8e0 2024-04-20T10:21:43.218 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.218+0000 7f7dd8a00640 1 -- 172.21.15.121:0/1125794261 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dd002cc10 con 0x7f7dd40ad8e0 2024-04-20T10:21:43.218 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.218+0000 7f7dd8a00640 1 -- 172.21.15.121:0/1125794261 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd00343f0 con 0x7f7dd40ad8e0 2024-04-20T10:21:43.218 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.218+0000 7f7ddc800640 1 -- 172.21.15.121:0/1125794261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7dd40ad8e0 msgr2=0x7f7dd40d0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:43.218 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.218+0000 7f7ddc800640 1 --2- 172.21.15.121:0/1125794261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7dd40ad8e0 0x7f7dd40d0360 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7dd406fe70 tx=0x7f7dd002bed0 comp rx=0 tx=0).stop 2024-04-20T10:21:43.218 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.218+0000 7f7ddc800640 1 -- 172.21.15.121:0/1125794261 shutdown_connections 2024-04-20T10:21:43.219 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.218+0000 7f7ddc800640 1 --2- 172.21.15.121:0/1125794261 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7dd40ad8e0 0x7f7dd40d0360 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:43.219 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.218+0000 7f7ddc800640 1 --2- 172.21.15.121:0/1125794261 >> v2:172.21.15.123:3300/0 conn(0x7f7dd415ca40 0x7f7dd40ad3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:43.219 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.218+0000 7f7ddc800640 1 --2- 172.21.15.121:0/1125794261 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7dd4156c20 0x7f7dd406b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:43.219 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.218+0000 7f7ddc800640 1 -- 172.21.15.121:0/1125794261 >> 172.21.15.121:0/1125794261 conn(0x7f7dd40b3350 msgr2=0x7f7dd40b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:43.219 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.218+0000 7f7ddc800640 1 -- 172.21.15.121:0/1125794261 shutdown_connections 2024-04-20T10:21:43.219 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.218+0000 7f7ddc800640 1 -- 172.21.15.121:0/1125794261 wait complete. 2024-04-20T10:21:43.219 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.220+0000 7f7ddc800640 1 Processor -- start 2024-04-20T10:21:43.219 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.220+0000 7f7ddc800640 1 -- start start 2024-04-20T10:21:43.221 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.221+0000 7f7ddc800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7dd40ad8e0 0x7f7dd409a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:43.221 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.221+0000 7f7ddc800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dd4156c20 0x7f7dd4073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:43.221 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.221+0000 7f7ddc800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7dd415ca40 0x7f7dd4073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:43.221 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.221+0000 7f7ddc800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd406bfc0 con 0x7f7dd40ad8e0 2024-04-20T10:21:43.221 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.221+0000 7f7ddc800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd406be40 con 0x7f7dd4156c20 2024-04-20T10:21:43.221 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.221+0000 7f7ddc800640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f7dd40d08a0 con 0x7f7dd415ca40 2024-04-20T10:21:43.221 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.221+0000 7f7dd9e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7dd40ad8e0 0x7f7dd409a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:43.222 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.221+0000 7f7dd9e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7dd40ad8e0 0x7f7dd409a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58814/0 (socket says 172.21.15.121:58814) 2024-04-20T10:21:43.222 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.221+0000 7f7dd9e00640 1 -- 172.21.15.121:0/1567125812 learned_addr learned my addr 172.21.15.121:0/1567125812 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:43.222 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.222+0000 7f7dd9400640 1 --2- 172.21.15.121:0/1567125812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dd4156c20 0x7f7dd4073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:43.222 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.222+0000 7f7dda800640 1 --2- 172.21.15.121:0/1567125812 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7dd415ca40 0x7f7dd4073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:43.222 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.222+0000 7f7dd9e00640 1 -- 172.21.15.121:0/1567125812 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7dd415ca40 msgr2=0x7f7dd4073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:43.222 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.222+0000 7f7dd9e00640 1 --2- 172.21.15.121:0/1567125812 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7dd415ca40 0x7f7dd4073680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:43.222 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.222+0000 7f7dd9e00640 1 -- 172.21.15.121:0/1567125812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dd4156c20 msgr2=0x7f7dd4073140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:43.222 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.222+0000 7f7dd9e00640 1 --2- 172.21.15.121:0/1567125812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dd4156c20 0x7f7dd4073140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:43.222 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.222+0000 7f7dd9e00640 1 -- 172.21.15.121:0/1567125812 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd409db20 con 0x7f7dd40ad8e0 2024-04-20T10:21:43.222 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.222+0000 7f7dd9400640 1 --2- 172.21.15.121:0/1567125812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dd4156c20 0x7f7dd4073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:43.223 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.222+0000 7f7dd9e00640 1 --2- 172.21.15.121:0/1567125812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7dd40ad8e0 0x7f7dd409a2d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7dc800c7c0 tx=0x7f7dc800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:43.223 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.222+0000 7f7dce000640 1 -- 172.21.15.121:0/1567125812 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8015070 con 0x7f7dd40ad8e0 2024-04-20T10:21:43.223 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.222+0000 7f7dce000640 1 -- 172.21.15.121:0/1567125812 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dc8007a30 con 0x7f7dd40ad8e0 2024-04-20T10:21:43.223 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.222+0000 7f7dce000640 1 -- 172.21.15.121:0/1567125812 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc8007e00 con 0x7f7dd40ad8e0 2024-04-20T10:21:43.223 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.222+0000 7f7ddc800640 1 -- 172.21.15.121:0/1567125812 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dd409de00 con 0x7f7dd40ad8e0 2024-04-20T10:21:43.224 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.223+0000 7f7ddc800640 1 -- 172.21.15.121:0/1567125812 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dd40ce4c0 con 0x7f7dd40ad8e0 2024-04-20T10:21:43.224 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.224+0000 7f7ddc800640 1 -- 172.21.15.121:0/1567125812 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dd4068980 con 0x7f7dd40ad8e0 2024-04-20T10:21:43.224 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.225+0000 7f7dce000640 1 -- 172.21.15.121:0/1567125812 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7dc8018070 con 0x7f7dd40ad8e0 2024-04-20T10:21:43.225 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.226+0000 7f7dce000640 1 --2- 172.21.15.121:0/1567125812 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7db806c8f0 0x7f7db806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:43.226 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.226+0000 7f7dce000640 1 -- 172.21.15.121:0/1567125812 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7dc808bce0 con 0x7f7dd40ad8e0 2024-04-20T10:21:43.226 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.226+0000 7f7dd9400640 1 --2- 172.21.15.121:0/1567125812 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7db806c8f0 0x7f7db806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:43.226 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.227+0000 7f7dd9400640 1 --2- 172.21.15.121:0/1567125812 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7db806c8f0 0x7f7db806edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7dd40af140 tx=0x7f7dc4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:43.230 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.231+0000 7f7dce000640 1 -- 172.21.15.121:0/1567125812 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7dc8059e60 con 0x7f7dd40ad8e0 2024-04-20T10:21:43.474 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.474+0000 7f7ddc800640 1 -- 172.21.15.121:0/1567125812 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7dd406c140 con 0x7f7dd40ad8e0 2024-04-20T10:21:43.474 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.475+0000 7f7dce000640 1 -- 172.21.15.121:0/1567125812 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5204 (secure 0 0 0) 0x7f7dc8059760 con 0x7f7dd40ad8e0 2024-04-20T10:21:43.475 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:21:43.475 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1376743826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1376743826},{"type":"v1","addr":"172.21.15.123:6833","nonce":1376743826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/999167058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":999167058},{"type":"v1","addr":"172.21.15.121:6835","nonce":999167058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2160736935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2160736935},{"type":"v1","addr":"172.21.15.121:6837","nonce":2160736935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:21:02.048590+0000","modified":"2024-04-20T10:21:29.775269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6835/3336775250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3336775250},{"type":"v1","addr":"172.21.15.123:6835","nonce":3336775250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1}]} 2024-04-20T10:21:43.475 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 8 2024-04-20T10:21:43.478 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.477+0000 7f7ddc800640 1 -- 172.21.15.121:0/1567125812 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7db806c8f0 msgr2=0x7f7db806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:43.478 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.477+0000 7f7ddc800640 1 --2- 172.21.15.121:0/1567125812 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7db806c8f0 0x7f7db806edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7dd40af140 tx=0x7f7dc4008040 comp rx=0 tx=0).stop 2024-04-20T10:21:43.478 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.477+0000 7f7ddc800640 1 -- 172.21.15.121:0/1567125812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7dd40ad8e0 msgr2=0x7f7dd409a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:43.478 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.477+0000 7f7ddc800640 1 --2- 172.21.15.121:0/1567125812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7dd40ad8e0 0x7f7dd409a2d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7dc800c7c0 tx=0x7f7dc800cc80 comp rx=0 tx=0).stop 2024-04-20T10:21:43.478 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.477+0000 7f7ddc800640 1 -- 172.21.15.121:0/1567125812 shutdown_connections 2024-04-20T10:21:43.478 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.477+0000 7f7ddc800640 1 --2- 172.21.15.121:0/1567125812 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7dd415ca40 0x7f7dd4073680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:43.478 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.477+0000 7f7ddc800640 1 --2- 172.21.15.121:0/1567125812 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7db806c8f0 0x7f7db806edb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:43.478 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.477+0000 7f7ddc800640 1 --2- 172.21.15.121:0/1567125812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dd4156c20 0x7f7dd4073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:43.479 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.477+0000 7f7ddc800640 1 --2- 172.21.15.121:0/1567125812 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7dd40ad8e0 0x7f7dd409a2d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:43.479 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.477+0000 7f7ddc800640 1 -- 172.21.15.121:0/1567125812 >> 172.21.15.121:0/1567125812 conn(0x7f7dd40b3350 msgr2=0x7f7dd40b0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:43.479 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.478+0000 7f7ddc800640 1 -- 172.21.15.121:0/1567125812 shutdown_connections 2024-04-20T10:21:43.479 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.478+0000 7f7ddc800640 1 -- 172.21.15.121:0/1567125812 wait complete. 2024-04-20T10:21:43.496 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:21:43.688 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.688+0000 7f0137600640 1 Processor -- start 2024-04-20T10:21:43.688 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.689+0000 7f0137600640 1 -- start start 2024-04-20T10:21:43.690 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.690+0000 7f0137600640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0130162c00 0x7f01300d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:43.690 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.690+0000 7f0137600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f013006fc30 0x7f01300d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:43.690 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.690+0000 7f0137600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f013006df70 con 0x7f013006f090 2024-04-20T10:21:43.690 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.690+0000 7f0137600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f0130162fe0 con 0x7f013006fc30 2024-04-20T10:21:43.690 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.690+0000 7f0137600640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f013006ec00 con 0x7f0130162c00 2024-04-20T10:21:43.691 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.691+0000 7f0134c00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0130162c00 0x7f01300d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:43.691 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.691+0000 7f0134c00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0130162c00 0x7f01300d0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.121:49974/0 (socket says 172.21.15.121:49974) 2024-04-20T10:21:43.691 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.691+0000 7f0134c00640 1 -- 172.21.15.121:0/2294990513 learned_addr learned my addr 172.21.15.121:0/2294990513 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:43.691 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.691+0000 7f0135600640 1 --2- 172.21.15.121:0/2294990513 >> v2:172.21.15.123:3300/0 conn(0x7f013006fc30 0x7f01300d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:43.691 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.692+0000 7f0134c00640 1 -- 172.21.15.121:0/2294990513 >> v2:172.21.15.123:3300/0 conn(0x7f013006fc30 msgr2=0x7f01300d08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:43.691 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.692+0000 7f0134c00640 1 --2- 172.21.15.121:0/2294990513 >> v2:172.21.15.123:3300/0 conn(0x7f013006fc30 0x7f01300d08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:43.691 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.692+0000 7f0134c00640 1 -- 172.21.15.121:0/2294990513 >> v1:172.21.15.123:6789/0 conn(0x7f013006f090 legacy=0x7f013006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:43.691 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.692+0000 7f0134c00640 1 -- 172.21.15.121:0/2294990513 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f013006e310 con 0x7f0130162c00 2024-04-20T10:21:43.692 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.693+0000 7f0134c00640 1 --2- 172.21.15.121:0/2294990513 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0130162c00 0x7f01300d0360 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0120009870 tx=0x7f012002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6649367d880da9f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:43.693 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.693+0000 7f012fe00640 1 -- 172.21.15.121:0/2294990513 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0120035020 con 0x7f0130162c00 2024-04-20T10:21:43.693 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.693+0000 7f012fe00640 1 -- 172.21.15.121:0/2294990513 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0120031640 con 0x7f0130162c00 2024-04-20T10:21:43.693 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.693+0000 7f012fe00640 1 -- 172.21.15.121:0/2294990513 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012002d040 con 0x7f0130162c00 2024-04-20T10:21:43.694 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.694+0000 7f0137600640 1 -- 172.21.15.121:0/2294990513 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0130162c00 msgr2=0x7f01300d0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:43.694 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.694+0000 7f0137600640 1 --2- 172.21.15.121:0/2294990513 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0130162c00 0x7f01300d0360 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0120009870 tx=0x7f012002b8f0 comp rx=0 tx=0).stop 2024-04-20T10:21:43.694 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.694+0000 7f0137600640 1 -- 172.21.15.121:0/2294990513 shutdown_connections 2024-04-20T10:21:43.694 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.694+0000 7f0137600640 1 --2- 172.21.15.121:0/2294990513 >> v2:172.21.15.123:3300/0 conn(0x7f013006fc30 0x7f01300d08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:43.694 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.694+0000 7f0137600640 1 --2- 172.21.15.121:0/2294990513 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0130162c00 0x7f01300d0360 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:43.694 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.694+0000 7f0137600640 1 -- 172.21.15.121:0/2294990513 >> 172.21.15.121:0/2294990513 conn(0x7f01300b3350 msgr2=0x7f01300b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:43.694 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.694+0000 7f0137600640 1 -- 172.21.15.121:0/2294990513 shutdown_connections 2024-04-20T10:21:43.694 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.694+0000 7f0137600640 1 -- 172.21.15.121:0/2294990513 wait complete. 2024-04-20T10:21:43.695 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.696+0000 7f0137600640 1 Processor -- start 2024-04-20T10:21:43.695 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.696+0000 7f0137600640 1 -- start start 2024-04-20T10:21:43.696 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.697+0000 7f0137600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f013006f090 0x7f0130165ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:43.696 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.697+0000 7f0137600640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f013006fc30 0x7f01300cd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:43.697 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.697+0000 7f0137600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0130162c00 0x7f01300cc170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:43.697 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.697+0000 7f0137600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f013006dbb0 con 0x7f0130162c00 2024-04-20T10:21:43.697 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.697+0000 7f0137600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f013006e890 con 0x7f013006f090 2024-04-20T10:21:43.697 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.697+0000 7f0137600640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f013006dd30 con 0x7f013006fc30 2024-04-20T10:21:43.698 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.697+0000 7f0135600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0130162c00 0x7f01300cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:43.698 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.698+0000 7f0135600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0130162c00 0x7f01300cc170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58830/0 (socket says 172.21.15.121:58830) 2024-04-20T10:21:43.698 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.698+0000 7f0135600640 1 -- 172.21.15.121:0/1595781419 learned_addr learned my addr 172.21.15.121:0/1595781419 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:43.699 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.698+0000 7f0127e00640 1 --2- 172.21.15.121:0/1595781419 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f013006fc30 0x7f01300cd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:43.699 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.698+0000 7f0134c00640 1 --2- 172.21.15.121:0/1595781419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f013006f090 0x7f0130165ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:43.699 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.698+0000 7f0135600640 1 -- 172.21.15.121:0/1595781419 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f013006fc30 msgr2=0x7f01300cd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:43.699 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.698+0000 7f0135600640 1 --2- 172.21.15.121:0/1595781419 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f013006fc30 0x7f01300cd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:43.699 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.698+0000 7f0135600640 1 -- 172.21.15.121:0/1595781419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f013006f090 msgr2=0x7f0130165ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:43.699 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.698+0000 7f0135600640 1 --2- 172.21.15.121:0/1595781419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f013006f090 0x7f0130165ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:43.699 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.698+0000 7f0135600640 1 -- 172.21.15.121:0/1595781419 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01300cc910 con 0x7f0130162c00 2024-04-20T10:21:43.699 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.698+0000 7f0134c00640 1 --2- 172.21.15.121:0/1595781419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f013006f090 0x7f0130165ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:21:43.699 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.698+0000 7f0135600640 1 --2- 172.21.15.121:0/1595781419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0130162c00 0x7f01300cc170 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f012800efc0 tx=0x7f012800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:43.699 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.699+0000 7f012d600640 1 -- 172.21.15.121:0/1595781419 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0128018020 con 0x7f0130162c00 2024-04-20T10:21:43.699 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.699+0000 7f012d600640 1 -- 172.21.15.121:0/1595781419 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0128014480 con 0x7f0130162c00 2024-04-20T10:21:43.699 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.699+0000 7f0137600640 1 -- 172.21.15.121:0/1595781419 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01300ccbf0 con 0x7f0130162c00 2024-04-20T10:21:43.699 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.699+0000 7f012d600640 1 -- 172.21.15.121:0/1595781419 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0128014870 con 0x7f0130162c00 2024-04-20T10:21:43.699 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.699+0000 7f0137600640 1 -- 172.21.15.121:0/1595781419 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01300cd1b0 con 0x7f0130162c00 2024-04-20T10:21:43.700 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.700+0000 7f0137600640 1 -- 172.21.15.121:0/1595781419 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0130068980 con 0x7f0130162c00 2024-04-20T10:21:43.700 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.701+0000 7f012d600640 1 -- 172.21.15.121:0/1595781419 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0128005d10 con 0x7f0130162c00 2024-04-20T10:21:43.701 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.702+0000 7f012d600640 1 --2- 172.21.15.121:0/1595781419 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f010c06c920 0x7f010c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:43.701 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.702+0000 7f012d600640 1 -- 172.21.15.121:0/1595781419 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f012808cb70 con 0x7f0130162c00 2024-04-20T10:21:43.702 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.702+0000 7f0134c00640 1 --2- 172.21.15.121:0/1595781419 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f010c06c920 0x7f010c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:43.702 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.703+0000 7f0134c00640 1 --2- 172.21.15.121:0/1595781419 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f010c06c920 0x7f010c06ede0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0120036000 tx=0x7f012003c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:43.706 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.706+0000 7f012d600640 1 -- 172.21.15.121:0/1595781419 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0128022030 con 0x7f0130162c00 2024-04-20T10:21:43.918 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.918+0000 7f0137600640 1 -- 172.21.15.121:0/1595781419 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0130072ae0 con 0x7f0130162c00 2024-04-20T10:21:43.920 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.920+0000 7f012d600640 1 -- 172.21.15.121:0/1595781419 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+18940 (secure 0 0 0) 0x7f0128010040 con 0x7f0130162c00 2024-04-20T10:21:43.920 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:21:43.920 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":26,"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","created":"2024-04-20T10:20:38.076076+0000","modified":"2024-04-20T10:21:43.707286+0000","last_up_change":"2024-04-20T10:20:50.162700+0000","last_in_change":"2024-04-20T10:20:41.510382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:20:51.141365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:21:00.436937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:21:01.026957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:21:04.862693+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"629a7b8c-a40a-4a2a-a3cc-d9f357774310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6809","nonce":1044588835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6811","nonce":1044588835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6815","nonce":1044588835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6813","nonce":1044588835}]},"public_addr":"172.21.15.121:6809/1044588835","cluster_addr":"172.21.15.121:6811/1044588835","heartbeat_back_addr":"172.21.15.121:6815/1044588835","heartbeat_front_addr":"172.21.15.121:6813/1044588835","state":["exists","up"]},{"osd":1,"uuid":"f2570399-aa50-4ff3-bf5d-3d09207290db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6801","nonce":3505199624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6803","nonce":3505199624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6807","nonce":3505199624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6805","nonce":3505199624}]},"public_addr":"172.21.15.121:6801/3505199624","cluster_addr":"172.21.15.121:6803/3505199624","heartbeat_back_addr":"172.21.15.121:6807/3505199624","heartbeat_front_addr":"172.21.15.121:6805/3505199624","state":["exists","up"]},{"osd":2,"uuid":"50893807-dc30-49f4-b41f-739366626658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6817","nonce":3656724207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6819","nonce":3656724207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6827","nonce":3656724207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6822","nonce":3656724207}]},"public_addr":"172.21.15.121:6817/3656724207","cluster_addr":"172.21.15.121:6819/3656724207","heartbeat_back_addr":"172.21.15.121:6827/3656724207","heartbeat_front_addr":"172.21.15.121:6822/3656724207","state":["exists","up"]},{"osd":3,"uuid":"dadb01bc-5491-4ffc-8824-d48a057240b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6823","nonce":2232342832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6826","nonce":2232342832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6831","nonce":2232342832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6829","nonce":2232342832}]},"public_addr":"172.21.15.121:6823/2232342832","cluster_addr":"172.21.15.121:6826/2232342832","heartbeat_back_addr":"172.21.15.121:6831/2232342832","heartbeat_front_addr":"172.21.15.121:6829/2232342832","state":["exists","up"]},{"osd":4,"uuid":"b3f3b4a1-b375-4526-b69e-344ab24c2da1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6801","nonce":99353111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6803","nonce":99353111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6807","nonce":99353111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6805","nonce":99353111}]},"public_addr":"172.21.15.123:6801/99353111","cluster_addr":"172.21.15.123:6803/99353111","heartbeat_back_addr":"172.21.15.123:6807/99353111","heartbeat_front_addr":"172.21.15.123:6805/99353111","state":["exists","up"]},{"osd":5,"uuid":"2ee24a08-1624-4961-9642-d7b510282484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6818","nonce":2504374688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6822","nonce":2504374688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6830","nonce":2504374688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6826","nonce":2504374688}]},"public_addr":"172.21.15.123:6818/2504374688","cluster_addr":"172.21.15.123:6822/2504374688","heartbeat_back_addr":"172.21.15.123:6830/2504374688","heartbeat_front_addr":"172.21.15.123:6826/2504374688","state":["exists","up"]},{"osd":6,"uuid":"186118cc-6128-49de-acca-55260945c1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6819","nonce":4139299599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6823","nonce":4139299599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6829","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6831","nonce":4139299599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6827","nonce":4139299599}]},"public_addr":"172.21.15.123:6819/4139299599","cluster_addr":"172.21.15.123:6823/4139299599","heartbeat_back_addr":"172.21.15.123:6831/4139299599","heartbeat_front_addr":"172.21.15.123:6827/4139299599","state":["exists","up"]},{"osd":7,"uuid":"a00aac6e-4bd9-415a-a93d-ae85210865ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6809","nonce":4043981069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6811","nonce":4043981069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6815","nonce":4043981069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6813","nonce":4043981069}]},"public_addr":"172.21.15.123:6809/4043981069","cluster_addr":"172.21.15.123:6811/4043981069","heartbeat_back_addr":"172.21.15.123:6815/4043981069","heartbeat_front_addr":"172.21.15.123:6813/4043981069","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.904540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.958585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.616367+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.885244+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.103127+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.167248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.884348+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.862356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.23","mappings":[{"from":3,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:21:43.922 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.923+0000 7f0137600640 1 -- 172.21.15.121:0/1595781419 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f010c06c920 msgr2=0x7f010c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:43.922 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.923+0000 7f0137600640 1 --2- 172.21.15.121:0/1595781419 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f010c06c920 0x7f010c06ede0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0120036000 tx=0x7f012003c040 comp rx=0 tx=0).stop 2024-04-20T10:21:43.923 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.923+0000 7f0137600640 1 -- 172.21.15.121:0/1595781419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0130162c00 msgr2=0x7f01300cc170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:43.923 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.923+0000 7f0137600640 1 --2- 172.21.15.121:0/1595781419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0130162c00 0x7f01300cc170 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f012800efc0 tx=0x7f012800c490 comp rx=0 tx=0).stop 2024-04-20T10:21:43.923 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.923+0000 7f0137600640 1 -- 172.21.15.121:0/1595781419 shutdown_connections 2024-04-20T10:21:43.923 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.923+0000 7f0137600640 1 --2- 172.21.15.121:0/1595781419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0130162c00 0x7f01300cc170 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:43.923 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.923+0000 7f0137600640 1 --2- 172.21.15.121:0/1595781419 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f013006fc30 0x7f01300cd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:43.923 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.923+0000 7f0137600640 1 --2- 172.21.15.121:0/1595781419 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f010c06c920 0x7f010c06ede0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:43.923 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.923+0000 7f0137600640 1 --2- 172.21.15.121:0/1595781419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f013006f090 0x7f0130165ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:43.923 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.923+0000 7f0137600640 1 -- 172.21.15.121:0/1595781419 >> 172.21.15.121:0/1595781419 conn(0x7f01300b3350 msgr2=0x7f01300af060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:43.923 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.923+0000 7f0137600640 1 -- 172.21.15.121:0/1595781419 shutdown_connections 2024-04-20T10:21:43.923 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:43.923+0000 7f0137600640 1 -- 172.21.15.121:0/1595781419 wait complete. 2024-04-20T10:21:43.941 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:21:44.143 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.143+0000 7f834ce00640 1 Processor -- start 2024-04-20T10:21:44.143 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.144+0000 7f834ce00640 1 -- start start 2024-04-20T10:21:44.144 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.145+0000 7f834ce00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f834815c450 0x7f834815ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:44.145 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.145+0000 7f834ce00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f834806ed90 0x7f834806dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:44.145 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.145+0000 7f834ce00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f834815f2e0 con 0x7f834815c070 2024-04-20T10:21:44.145 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.145+0000 7f834ce00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f834815f5c0 con 0x7f834806ed90 2024-04-20T10:21:44.145 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.145+0000 7f834ce00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f834815f8a0 con 0x7f834815c450 2024-04-20T10:21:44.145 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.146+0000 7f8345400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f834815c450 0x7f834815ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:44.146 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.146+0000 7f8345400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f834815c450 0x7f834815ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.121:49996/0 (socket says 172.21.15.121:49996) 2024-04-20T10:21:44.146 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.146+0000 7f8345400640 1 -- 172.21.15.121:0/2529913682 learned_addr learned my addr 172.21.15.121:0/2529913682 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:44.146 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.146+0000 7f8346800640 1 --2- 172.21.15.121:0/2529913682 >> v2:172.21.15.123:3300/0 conn(0x7f834806ed90 0x7f834806dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:44.146 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.147+0000 7f8345400640 1 -- 172.21.15.121:0/2529913682 >> v2:172.21.15.123:3300/0 conn(0x7f834806ed90 msgr2=0x7f834806dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:44.146 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.147+0000 7f8345400640 1 --2- 172.21.15.121:0/2529913682 >> v2:172.21.15.123:3300/0 conn(0x7f834806ed90 0x7f834806dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:44.147 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.147+0000 7f8345400640 1 -- 172.21.15.121:0/2529913682 >> v1:172.21.15.123:6789/0 conn(0x7f834815c070 legacy=0x7f83480cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:44.147 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.147+0000 7f8345400640 1 -- 172.21.15.121:0/2529913682 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f834806d770 con 0x7f834815c450 2024-04-20T10:21:44.147 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.148+0000 7f8345400640 1 --2- 172.21.15.121:0/2529913682 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f834815c450 0x7f834815ecb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8334009870 tx=0x7f833402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4a1e6c38fe68c14 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:44.148 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.149+0000 7f8344a00640 1 -- 172.21.15.121:0/2529913682 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8334034070 con 0x7f834815c450 2024-04-20T10:21:44.148 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.149+0000 7f8344a00640 1 -- 172.21.15.121:0/2529913682 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8334030840 con 0x7f834815c450 2024-04-20T10:21:44.148 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.149+0000 7f8344a00640 1 -- 172.21.15.121:0/2529913682 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8334030c10 con 0x7f834815c450 2024-04-20T10:21:44.149 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.150+0000 7f834ce00640 1 -- 172.21.15.121:0/2529913682 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f834815c450 msgr2=0x7f834815ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:44.149 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.150+0000 7f834ce00640 1 --2- 172.21.15.121:0/2529913682 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f834815c450 0x7f834815ecb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8334009870 tx=0x7f833402b980 comp rx=0 tx=0).stop 2024-04-20T10:21:44.149 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.150+0000 7f834ce00640 1 -- 172.21.15.121:0/2529913682 shutdown_connections 2024-04-20T10:21:44.149 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.150+0000 7f834ce00640 1 --2- 172.21.15.121:0/2529913682 >> v2:172.21.15.123:3300/0 conn(0x7f834806ed90 0x7f834806dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:44.149 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.150+0000 7f834ce00640 1 --2- 172.21.15.121:0/2529913682 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f834815c450 0x7f834815ecb0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:44.149 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.150+0000 7f834ce00640 1 -- 172.21.15.121:0/2529913682 >> 172.21.15.121:0/2529913682 conn(0x7f83480b3fb0 msgr2=0x7f83480b4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:44.150 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.150+0000 7f834ce00640 1 -- 172.21.15.121:0/2529913682 shutdown_connections 2024-04-20T10:21:44.150 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.150+0000 7f834ce00640 1 -- 172.21.15.121:0/2529913682 wait complete. 2024-04-20T10:21:44.151 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.152+0000 7f834ce00640 1 Processor -- start 2024-04-20T10:21:44.151 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.152+0000 7f834ce00640 1 -- start start 2024-04-20T10:21:44.153 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.153+0000 7f834ce00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f834806ed90 0x7f834814b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:44.153 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.153+0000 7f834ce00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f834815c070 0x7f83481f80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:44.153 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.153+0000 7f834ce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83481f8630 0x7f83481e5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:44.154 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.153+0000 7f834ce00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f834806d380 con 0x7f834806ed90 2024-04-20T10:21:44.154 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.153+0000 7f834ce00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f834815fb80 con 0x7f83481f8630 2024-04-20T10:21:44.154 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.153+0000 7f834ce00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f834806e1d0 con 0x7f834815c070 2024-04-20T10:21:44.154 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.154+0000 7f8345e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f834806ed90 0x7f834814b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:44.154 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.154+0000 7f8345e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f834806ed90 0x7f834814b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58836/0 (socket says 172.21.15.121:58836) 2024-04-20T10:21:44.154 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.154+0000 7f8345e00640 1 -- 172.21.15.121:0/2699772672 learned_addr learned my addr 172.21.15.121:0/2699772672 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:44.154 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.154+0000 7f8345400640 1 --2- 172.21.15.121:0/2699772672 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f834815c070 0x7f83481f80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:44.154 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.154+0000 7f8346800640 1 --2- 172.21.15.121:0/2699772672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83481f8630 0x7f83481e5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:44.155 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.154+0000 7f8345e00640 1 -- 172.21.15.121:0/2699772672 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f834815c070 msgr2=0x7f83481f80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:44.155 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.154+0000 7f8345e00640 1 --2- 172.21.15.121:0/2699772672 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f834815c070 0x7f83481f80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:44.155 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.154+0000 7f8345e00640 1 -- 172.21.15.121:0/2699772672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83481f8630 msgr2=0x7f83481e5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:44.155 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.154+0000 7f8345e00640 1 --2- 172.21.15.121:0/2699772672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83481f8630 0x7f83481e5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:44.156 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.154+0000 7f8345e00640 1 -- 172.21.15.121:0/2699772672 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83481f8b40 con 0x7f834806ed90 2024-04-20T10:21:44.156 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.154+0000 7f8346800640 1 --2- 172.21.15.121:0/2699772672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83481f8630 0x7f83481e5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:21:44.156 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.154+0000 7f8345400640 1 --2- 172.21.15.121:0/2699772672 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f834815c070 0x7f83481f80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:21:44.156 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.155+0000 7f8345e00640 1 --2- 172.21.15.121:0/2699772672 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f834806ed90 0x7f834814b720 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f833800c400 tx=0x7f833800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:44.156 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.155+0000 7f833e000640 1 -- 172.21.15.121:0/2699772672 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83380052d0 con 0x7f834806ed90 2024-04-20T10:21:44.156 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.155+0000 7f833e000640 1 -- 172.21.15.121:0/2699772672 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8338005490 con 0x7f834806ed90 2024-04-20T10:21:44.156 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.155+0000 7f833e000640 1 -- 172.21.15.121:0/2699772672 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8338007260 con 0x7f834806ed90 2024-04-20T10:21:44.156 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.155+0000 7f834ce00640 1 -- 172.21.15.121:0/2699772672 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83481f8e20 con 0x7f834806ed90 2024-04-20T10:21:44.156 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.155+0000 7f834ce00640 1 -- 172.21.15.121:0/2699772672 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f834814bc60 con 0x7f834806ed90 2024-04-20T10:21:44.157 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.157+0000 7f834ce00640 1 -- 172.21.15.121:0/2699772672 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8348069090 con 0x7f834806ed90 2024-04-20T10:21:44.157 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.157+0000 7f833e000640 1 -- 172.21.15.121:0/2699772672 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8338007400 con 0x7f834806ed90 2024-04-20T10:21:44.158 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.158+0000 7f833e000640 1 --2- 172.21.15.121:0/2699772672 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f832806c920 0x7f832806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:44.158 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.158+0000 7f833e000640 1 -- 172.21.15.121:0/2699772672 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6735+0+0 (secure 0 0 0) 0x7f833808b3d0 con 0x7f834806ed90 2024-04-20T10:21:44.158 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.159+0000 7f8345400640 1 --2- 172.21.15.121:0/2699772672 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f832806c920 0x7f832806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:44.158 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.159+0000 7f8345400640 1 --2- 172.21.15.121:0/2699772672 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f832806c920 0x7f832806ede0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8334036000 tx=0x7f833403b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:44.162 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.163+0000 7f833e000640 1 -- 172.21.15.121:0/2699772672 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8338059490 con 0x7f834806ed90 2024-04-20T10:21:44.405 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.405+0000 7f834ce00640 1 -- 172.21.15.121:0/2699772672 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83480b3960 con 0x7f834806ed90 2024-04-20T10:21:44.405 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.406+0000 7f833e000640 1 -- 172.21.15.121:0/2699772672 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5204 (secure 0 0 0) 0x7f8338058d90 con 0x7f834806ed90 2024-04-20T10:21:44.406 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:21:44.406 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1376743826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1376743826},{"type":"v1","addr":"172.21.15.123:6833","nonce":1376743826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/999167058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":999167058},{"type":"v1","addr":"172.21.15.121:6835","nonce":999167058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2160736935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2160736935},{"type":"v1","addr":"172.21.15.121:6837","nonce":2160736935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:21:02.048590+0000","modified":"2024-04-20T10:21:29.775269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6835/3336775250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3336775250},{"type":"v1","addr":"172.21.15.123:6835","nonce":3336775250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1}]} 2024-04-20T10:21:44.406 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 8 2024-04-20T10:21:44.409 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.409+0000 7f834ce00640 1 -- 172.21.15.121:0/2699772672 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f832806c920 msgr2=0x7f832806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:44.409 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.409+0000 7f834ce00640 1 --2- 172.21.15.121:0/2699772672 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f832806c920 0x7f832806ede0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8334036000 tx=0x7f833403b040 comp rx=0 tx=0).stop 2024-04-20T10:21:44.409 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.409+0000 7f834ce00640 1 -- 172.21.15.121:0/2699772672 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f834806ed90 msgr2=0x7f834814b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:44.409 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.409+0000 7f834ce00640 1 --2- 172.21.15.121:0/2699772672 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f834806ed90 0x7f834814b720 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f833800c400 tx=0x7f833800c8c0 comp rx=0 tx=0).stop 2024-04-20T10:21:44.409 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.409+0000 7f834ce00640 1 -- 172.21.15.121:0/2699772672 shutdown_connections 2024-04-20T10:21:44.409 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.409+0000 7f834ce00640 1 --2- 172.21.15.121:0/2699772672 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f832806c920 0x7f832806ede0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:44.409 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.409+0000 7f834ce00640 1 --2- 172.21.15.121:0/2699772672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83481f8630 0x7f83481e5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:44.409 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.409+0000 7f834ce00640 1 --2- 172.21.15.121:0/2699772672 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f834815c070 0x7f83481f80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:44.409 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.409+0000 7f834ce00640 1 --2- 172.21.15.121:0/2699772672 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f834806ed90 0x7f834814b720 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:44.409 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.409+0000 7f834ce00640 1 -- 172.21.15.121:0/2699772672 >> 172.21.15.121:0/2699772672 conn(0x7f83480b3fb0 msgr2=0x7f83480676c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:44.410 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.409+0000 7f834ce00640 1 -- 172.21.15.121:0/2699772672 shutdown_connections 2024-04-20T10:21:44.410 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:44.409+0000 7f834ce00640 1 -- 172.21.15.121:0/2699772672 wait complete. 2024-04-20T10:21:44.831 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T10:21:02.048590+0000', 'modified': '2024-04-20T10:21:29.775269+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4281}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4281': {'gid': 4281, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.123:6835/3336775250', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6834', 'nonce': 3336775250}, {'type': 'v1', 'addr': '172.21.15.123:6835', 'nonce': 3336775250}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4281, 'qdb_cluster': [4281]} 2024-04-20T10:21:44.831 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T10:21:44.831 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-20T10:21:45.043 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.043+0000 7f1fa2000640 1 Processor -- start 2024-04-20T10:21:45.043 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.043+0000 7f1fa2000640 1 -- start start 2024-04-20T10:21:45.045 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.045+0000 7f1fa2000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1f9c06c010 0x7f1f9c0cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:45.045 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.045+0000 7f1fa2000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1f9c0709c0 0x7f1f9c0b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:45.045 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.045+0000 7f1fa2000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f9c06dd00 con 0x7f1f9c15e740 2024-04-20T10:21:45.045 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.045+0000 7f1fa2000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f1f9c0b4240 con 0x7f1f9c0709c0 2024-04-20T10:21:45.045 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.045+0000 7f1fa2000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f1f9c0cfac0 con 0x7f1f9c06c010 2024-04-20T10:21:45.045 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.045+0000 7f1f9be00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1f9c0709c0 0x7f1f9c0b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:45.045 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.045+0000 7f1f9be00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1f9c0709c0 0x7f1f9c0b46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58844/0 (socket says 172.21.15.121:58844) 2024-04-20T10:21:45.045 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.045+0000 7f1f9be00640 1 -- 172.21.15.121:0/2599844544 learned_addr learned my addr 172.21.15.121:0/2599844544 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:45.046 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.046+0000 7f1f9aa00640 1 --2- 172.21.15.121:0/2599844544 >> v2:172.21.15.123:3300/0 conn(0x7f1f9c06c010 0x7f1f9c0cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:45.046 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.046+0000 7f1f9be00640 1 -- 172.21.15.121:0/2599844544 >> v2:172.21.15.123:3300/0 conn(0x7f1f9c06c010 msgr2=0x7f1f9c0cf3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:45.046 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.046+0000 7f1f9be00640 1 --2- 172.21.15.121:0/2599844544 >> v2:172.21.15.123:3300/0 conn(0x7f1f9c06c010 0x7f1f9c0cf3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.046+0000 7f1f9be00640 1 -- 172.21.15.121:0/2599844544 >> v1:172.21.15.123:6789/0 conn(0x7f1f9c15e740 legacy=0x7f1f9c06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:45.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.046+0000 7f1f9be00640 1 -- 172.21.15.121:0/2599844544 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9c0cfda0 con 0x7f1f9c0709c0 2024-04-20T10:21:45.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.046+0000 7f1f9aa00640 1 --2- 172.21.15.121:0/2599844544 >> v2:172.21.15.123:3300/0 conn(0x7f1f9c06c010 0x7f1f9c0cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:45.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.047+0000 7f1f9be00640 1 --2- 172.21.15.121:0/2599844544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f9c0709c0 0x7f1f9c0b46d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c0ce470 tx=0x7f1f9402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=582ea694dcbbc71f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:45.048 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.047+0000 7f1f9a000640 1 -- 172.21.15.121:0/2599844544 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f94035020 con 0x7f1f9c0709c0 2024-04-20T10:21:45.048 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.048+0000 7f1f9a000640 1 -- 172.21.15.121:0/2599844544 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f94031870 con 0x7f1f9c0709c0 2024-04-20T10:21:45.048 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.048+0000 7f1f9a000640 1 -- 172.21.15.121:0/2599844544 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f94031c40 con 0x7f1f9c0709c0 2024-04-20T10:21:45.048 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.048+0000 7f1fa2000640 1 -- 172.21.15.121:0/2599844544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f9c0709c0 msgr2=0x7f1f9c0b46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:45.048 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.048+0000 7f1fa2000640 1 --2- 172.21.15.121:0/2599844544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f9c0709c0 0x7f1f9c0b46d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c0ce470 tx=0x7f1f9402b910 comp rx=0 tx=0).stop 2024-04-20T10:21:45.048 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.048+0000 7f1fa2000640 1 -- 172.21.15.121:0/2599844544 shutdown_connections 2024-04-20T10:21:45.048 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.048+0000 7f1fa2000640 1 --2- 172.21.15.121:0/2599844544 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f9c0709c0 0x7f1f9c0b46d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.048 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.048+0000 7f1fa2000640 1 --2- 172.21.15.121:0/2599844544 >> v2:172.21.15.123:3300/0 conn(0x7f1f9c06c010 0x7f1f9c0cf3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.048 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.048+0000 7f1fa2000640 1 -- 172.21.15.121:0/2599844544 >> 172.21.15.121:0/2599844544 conn(0x7f1f9c0cd530 msgr2=0x7f1f9c0cd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:45.049 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.048+0000 7f1fa2000640 1 -- 172.21.15.121:0/2599844544 shutdown_connections 2024-04-20T10:21:45.049 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.048+0000 7f1fa2000640 1 -- 172.21.15.121:0/2599844544 wait complete. 2024-04-20T10:21:45.050 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.050+0000 7f1fa2000640 1 Processor -- start 2024-04-20T10:21:45.050 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.051+0000 7f1fa2000640 1 -- start start 2024-04-20T10:21:45.051 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.052+0000 7f1fa2000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f9c06c010 0x7f1f9c149110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:45.051 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.052+0000 7f1fa2000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f9c0709c0 0x7f1f9c160260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:45.051 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.052+0000 7f1fa2000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f9c15e740 0x7f1f9c0b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:45.052 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.052+0000 7f1fa2000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c0b4c10 con 0x7f1f9c06c010 2024-04-20T10:21:45.052 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.052+0000 7f1fa2000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c15eb20 con 0x7f1f9c15e740 2024-04-20T10:21:45.052 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.052+0000 7f1fa2000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f1f9c0b4d90 con 0x7f1f9c0709c0 2024-04-20T10:21:45.052 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.052+0000 7f1f9b400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f9c06c010 0x7f1f9c149110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:45.052 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.052+0000 7f1f9b400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f9c06c010 0x7f1f9c149110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58846/0 (socket says 172.21.15.121:58846) 2024-04-20T10:21:45.052 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.052+0000 7f1f9b400640 1 -- 172.21.15.121:0/1701853533 learned_addr learned my addr 172.21.15.121:0/1701853533 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:45.053 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.052+0000 7f1f9be00640 1 --2- 172.21.15.121:0/1701853533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f9c15e740 0x7f1f9c0b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:45.053 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.052+0000 7f1f9b400640 1 -- 172.21.15.121:0/1701853533 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f9c0709c0 msgr2=0x7f1f9c160260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:45.053 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.052+0000 7f1f9aa00640 1 --2- 172.21.15.121:0/1701853533 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f9c0709c0 0x7f1f9c160260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:45.053 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.053+0000 7f1f9b400640 1 --2- 172.21.15.121:0/1701853533 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f9c0709c0 0x7f1f9c160260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.053 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.053+0000 7f1f9b400640 1 -- 172.21.15.121:0/1701853533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f9c15e740 msgr2=0x7f1f9c0b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:45.054 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.053+0000 7f1f9b400640 1 --2- 172.21.15.121:0/1701853533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f9c15e740 0x7f1f9c0b10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.054 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.053+0000 7f1f9b400640 1 -- 172.21.15.121:0/1701853533 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9c15f6b0 con 0x7f1f9c06c010 2024-04-20T10:21:45.054 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.053+0000 7f1f9be00640 1 --2- 172.21.15.121:0/1701853533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f9c15e740 0x7f1f9c0b10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:45.054 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.053+0000 7f1f9b400640 1 --2- 172.21.15.121:0/1701853533 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f9c06c010 0x7f1f9c149110 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c00b420 tx=0x7f1f8c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:45.054 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.053+0000 7f1f93400640 1 -- 172.21.15.121:0/1701853533 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c011830 con 0x7f1f9c06c010 2024-04-20T10:21:45.054 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.053+0000 7f1f93400640 1 -- 172.21.15.121:0/1701853533 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f8c016070 con 0x7f1f9c06c010 2024-04-20T10:21:45.054 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.053+0000 7f1fa2000640 1 -- 172.21.15.121:0/1701853533 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f9c15f990 con 0x7f1f9c06c010 2024-04-20T10:21:45.054 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.053+0000 7f1f93400640 1 -- 172.21.15.121:0/1701853533 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c011c10 con 0x7f1f9c06c010 2024-04-20T10:21:45.054 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.053+0000 7f1fa2000640 1 -- 172.21.15.121:0/1701853533 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f9c149cb0 con 0x7f1f9c06c010 2024-04-20T10:21:45.055 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.056+0000 7f1f93400640 1 -- 172.21.15.121:0/1701853533 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1f8c01b460 con 0x7f1f9c06c010 2024-04-20T10:21:45.056 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.056+0000 7f1f93400640 1 --2- 172.21.15.121:0/1701853533 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f1f7806c920 0x7f1f7806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:45.056 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.057+0000 7f1f93400640 1 -- 172.21.15.121:0/1701853533 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f1f8c08bce0 con 0x7f1f9c06c010 2024-04-20T10:21:45.056 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.057+0000 7f1f9aa00640 1 --2- 172.21.15.121:0/1701853533 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f1f7806c920 0x7f1f7806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:45.057 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.057+0000 7f1fa4d85740 1 Processor -- start 2024-04-20T10:21:45.057 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.057+0000 7f1fa4d85740 1 -- start start 2024-04-20T10:21:45.057 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.058+0000 7f1f9aa00640 1 --2- 172.21.15.121:0/1701853533 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f1f7806c920 0x7f1f7806ede0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c0cce90 tx=0x7f1f840084f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:45.059 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.058+0000 7f1fa4d85740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1b31d3460 0x55e1b318aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:45.059 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.058+0000 7f1fa4d85740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55e1b31894e0 0x55e1b318afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:45.059 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.058+0000 7f1fa4d85740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e1b3185930 0x55e1b31b5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:45.059 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.058+0000 7f1fa4d85740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e1b30f19d0 con 0x55e1b31d3460 2024-04-20T10:21:45.059 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.058+0000 7f1fa4d85740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e1b3098270 con 0x55e1b3185930 2024-04-20T10:21:45.059 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.058+0000 7f1fa4d85740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x55e1b30ed990 con 0x55e1b31894e0 2024-04-20T10:21:45.059 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.059+0000 7f1f9b400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1b31d3460 0x55e1b318aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:45.059 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.059+0000 7f1f9b400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1b31d3460 0x55e1b318aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58850/0 (socket says 172.21.15.121:58850) 2024-04-20T10:21:45.059 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.059+0000 7f1f9b400640 1 -- 172.21.15.121:0/797097145 learned_addr learned my addr 172.21.15.121:0/797097145 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:45.059 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.059+0000 7f1f9aa00640 1 --2- 172.21.15.121:0/797097145 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55e1b31894e0 0x55e1b318afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:45.059 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.059+0000 7f1f9b400640 1 -- 172.21.15.121:0/797097145 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55e1b31894e0 msgr2=0x55e1b318afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:45.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.059+0000 7f1f9be00640 1 --2- 172.21.15.121:0/797097145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e1b3185930 0x55e1b31b5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:45.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.059+0000 7f1f9b400640 1 --2- 172.21.15.121:0/797097145 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55e1b31894e0 0x55e1b318afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.059+0000 7f1f9b400640 1 -- 172.21.15.121:0/797097145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e1b3185930 msgr2=0x55e1b31b5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:45.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.059+0000 7f1f9b400640 1 --2- 172.21.15.121:0/797097145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e1b3185930 0x55e1b31b5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.059+0000 7f1f9b400640 1 -- 172.21.15.121:0/797097145 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1b2e042d0 con 0x55e1b31d3460 2024-04-20T10:21:45.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.059+0000 7f1f9b400640 1 --2- 172.21.15.121:0/797097145 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1b31d3460 0x55e1b318aa90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c089cb0 tx=0x7f1f8c059630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f094baf526ad32bb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:45.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.059+0000 7f1f8b400640 1 -- 172.21.15.121:0/797097145 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c057610 con 0x55e1b31d3460 2024-04-20T10:21:45.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.060+0000 7f1f8b400640 1 -- 172.21.15.121:0/797097145 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f8c057870 con 0x55e1b31d3460 2024-04-20T10:21:45.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.060+0000 7f1f8b400640 1 -- 172.21.15.121:0/797097145 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c061cc0 con 0x55e1b31d3460 2024-04-20T10:21:45.061 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.060+0000 7f1fa4d85740 1 -- 172.21.15.121:0/797097145 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1b31d3460 msgr2=0x55e1b318aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:45.061 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.060+0000 7f1fa4d85740 1 --2- 172.21.15.121:0/797097145 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1b31d3460 0x55e1b318aa90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c089cb0 tx=0x7f1f8c059630 comp rx=0 tx=0).stop 2024-04-20T10:21:45.061 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.060+0000 7f1fa4d85740 1 -- 172.21.15.121:0/797097145 shutdown_connections 2024-04-20T10:21:45.061 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.060+0000 7f1fa4d85740 1 --2- 172.21.15.121:0/797097145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e1b3185930 0x55e1b31b5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.061 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.060+0000 7f1fa4d85740 1 --2- 172.21.15.121:0/797097145 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55e1b31894e0 0x55e1b318afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.061 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.060+0000 7f1fa4d85740 1 --2- 172.21.15.121:0/797097145 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1b31d3460 0x55e1b318aa90 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.062 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.060+0000 7f1fa4d85740 1 -- 172.21.15.121:0/797097145 >> 172.21.15.121:0/797097145 conn(0x55e1b3187bf0 msgr2=0x55e1b31d0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:45.062 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.060+0000 7f1fa4d85740 1 -- 172.21.15.121:0/797097145 shutdown_connections 2024-04-20T10:21:45.062 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.060+0000 7f1fa4d85740 1 -- 172.21.15.121:0/797097145 wait complete. 2024-04-20T10:21:45.062 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.061+0000 7f1fa4d85740 1 -- start start 2024-04-20T10:21:45.062 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.061+0000 7f1fa4d85740 1 Processor -- start 2024-04-20T10:21:45.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.062+0000 7f1fa4d85740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55e1b31894e0 0x55e1b31b3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:45.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.062+0000 7f1fa4d85740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e1b31d3460 0x55e1b319aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:45.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.062+0000 7f1fa4d85740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1b319b410 0x55e1b31b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:45.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.062+0000 7f1fa4d85740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e1b31899f0 con 0x55e1b319b410 2024-04-20T10:21:45.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.062+0000 7f1fa4d85740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e1b30ab5e0 con 0x55e1b31d3460 2024-04-20T10:21:45.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.062+0000 7f1fa4d85740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x55e1b3189cd0 con 0x55e1b31894e0 2024-04-20T10:21:45.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.062+0000 7f1fa4d85740 10 client.0 _ll_register_callbacks cb 0x55e1b2e03ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:21:45.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.062+0000 7f1f9be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1b319b410 0x55e1b31b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:45.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.063+0000 7f1f9be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1b319b410 0x55e1b31b0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58858/0 (socket says 172.21.15.121:58858) 2024-04-20T10:21:45.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.063+0000 7f1f9be00640 1 -- 172.21.15.121:0/2771197751 learned_addr learned my addr 172.21.15.121:0/2771197751 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:45.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.063+0000 7f1f9aa00640 1 --2- 172.21.15.121:0/2771197751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e1b31d3460 0x55e1b319aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:45.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.063+0000 7f1f9b400640 1 --2- 172.21.15.121:0/2771197751 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55e1b31894e0 0x55e1b31b3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:45.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.063+0000 7f1f9be00640 1 -- 172.21.15.121:0/2771197751 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55e1b31894e0 msgr2=0x55e1b31b3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:45.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.063+0000 7f1f9be00640 1 --2- 172.21.15.121:0/2771197751 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55e1b31894e0 0x55e1b31b3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.063+0000 7f1f9be00640 1 -- 172.21.15.121:0/2771197751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e1b31d3460 msgr2=0x55e1b319aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:45.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.063+0000 7f1f9be00640 1 --2- 172.21.15.121:0/2771197751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e1b31d3460 0x55e1b319aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.063+0000 7f1f9be00640 1 -- 172.21.15.121:0/2771197751 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1b2ece6b0 con 0x55e1b319b410 2024-04-20T10:21:45.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.063+0000 7f1f9b400640 1 --2- 172.21.15.121:0/2771197751 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55e1b31894e0 0x55e1b31b3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:45.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.063+0000 7f1f9be00640 1 --2- 172.21.15.121:0/2771197751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1b319b410 0x55e1b31b0a70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1f9402bf20 tx=0x7f1f9402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d27a94f391c05a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:45.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.063+0000 7f1f83e00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-04-20T10:21:45.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.064+0000 7f1f83e00640 1 -- 172.21.15.121:0/2771197751 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f94049060 con 0x55e1b319b410 2024-04-20T10:21:45.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.064+0000 7f1f83e00640 1 -- 172.21.15.121:0/2771197751 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f1f9403a630 con 0x55e1b319b410 2024-04-20T10:21:45.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.064+0000 7f1fa4d85740 1 -- 172.21.15.121:0/2771197751 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f8c057870 con 0x55e1b319b410 2024-04-20T10:21:45.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.064+0000 7f1f83e00640 1 -- 172.21.15.121:0/2771197751 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f1f9404f050 con 0x55e1b319b410 2024-04-20T10:21:45.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.064+0000 7f1fa4d85740 10 client.4857 fetch_fsmap learned FSMap version 8 2024-04-20T10:21:45.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.064+0000 7f1fa4d85740 1 -- 172.21.15.121:0/2771197751 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55e1b31993d0 con 0x55e1b319b410 2024-04-20T10:21:45.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.064+0000 7f1f83e00640 1 -- 172.21.15.121:0/2771197751 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f8c057870 con 0x55e1b319b410 2024-04-20T10:21:45.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.064+0000 7f1f83e00640 1 -- 172.21.15.121:0/2771197751 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 8) ==== 3503+0+0 (secure 0 0 0) 0x7f1f94052020 con 0x55e1b319b410 2024-04-20T10:21:45.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.064+0000 7f1fa4d85740 10 client.4857 fetch_fsmap finished waiting for FSMap version 8 2024-04-20T10:21:45.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.064+0000 7f1fa4d85740 10 client.4857 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-20T10:21:45.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.064+0000 7f1fa4d85740 20 client.4857 populate_metadata read hostname 'smithi121' 2024-04-20T10:21:45.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.064+0000 7f1fa4d85740 1 --2- 172.21.15.121:0/2771197751 >> [v2:172.21.15.123:6834/3336775250,v1:172.21.15.123:6835/3336775250] conn(0x55e1b31d5650 0x55e1b319a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:45.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.064+0000 7f1fa4d85740 4 client.4857 mds_command: new command op to 4281 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:21:45.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.064+0000 7f1fa4d85740 1 -- 172.21.15.121:0/2771197751 --> [v2:172.21.15.123:6834/3336775250,v1:172.21.15.123:6835/3336775250] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1b30ff3f0 con 0x55e1b31d5650 2024-04-20T10:21:45.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.065+0000 7f1f9b400640 1 --2- 172.21.15.121:0/2771197751 >> [v2:172.21.15.123:6834/3336775250,v1:172.21.15.123:6835/3336775250] conn(0x55e1b31d5650 0x55e1b319a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:45.068 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.066+0000 7f1f9b400640 1 --2- 172.21.15.121:0/2771197751 >> [v2:172.21.15.123:6834/3336775250,v1:172.21.15.123:6835/3336775250] conn(0x55e1b31d5650 0x55e1b319a510 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a8d12157d08e9a9 server_cookie=87fbde50b193cbcc in_seq=0 out_seq=0 2024-04-20T10:21:45.069 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.066+0000 7f1f83e00640 10 client.4857 ms_handle_connect on v2:172.21.15.123:6834/3336775250 2024-04-20T10:21:45.069 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.068+0000 7f1f83e00640 1 -- 172.21.15.121:0/2771197751 <== mds.0 v2:172.21.15.123:6834/3336775250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e1b30ff3f0 con 0x55e1b31d5650 2024-04-20T10:21:45.069 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.068+0000 7f1f83e00640 10 client.4857 handle_command_reply: tid=0 2024-04-20T10:21:45.069 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.068+0000 7f1fa4d85740 1 client.4857 shutdown 2024-04-20T10:21:45.069 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.068+0000 7f1fa4d85740 1 -- 172.21.15.121:0/2771197751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1b319b410 msgr2=0x55e1b31b0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:45.069 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.068+0000 7f1fa4d85740 1 --2- 172.21.15.121:0/2771197751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1b319b410 0x55e1b31b0a70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1f9402bf20 tx=0x7f1f9402e000 comp rx=0 tx=0).stop 2024-04-20T10:21:45.069 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.068+0000 7f1fa4d85740 1 -- 172.21.15.121:0/2771197751 shutdown_connections 2024-04-20T10:21:45.069 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.068+0000 7f1fa4d85740 1 --2- 172.21.15.121:0/2771197751 >> [v2:172.21.15.123:6834/3336775250,v1:172.21.15.123:6835/3336775250] conn(0x55e1b31d5650 0x55e1b319a510 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.069 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.068+0000 7f1fa4d85740 1 --2- 172.21.15.121:0/2771197751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1b319b410 0x55e1b31b0a70 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.069 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.068+0000 7f1fa4d85740 1 --2- 172.21.15.121:0/2771197751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e1b31d3460 0x55e1b319aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.069 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.068+0000 7f1fa4d85740 1 --2- 172.21.15.121:0/2771197751 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55e1b31894e0 0x55e1b31b3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.069 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.068+0000 7f1f83e00640 0 client.4857 ms_handle_reset on v2:172.21.15.123:6834/3336775250 2024-04-20T10:21:45.069 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.068+0000 7f1fa4d85740 1 -- 172.21.15.121:0/2771197751 >> 172.21.15.121:0/2771197751 conn(0x55e1b3185930 msgr2=0x55e1b318b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:45.070 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.068+0000 7f1fa4d85740 1 -- 172.21.15.121:0/2771197751 shutdown_connections 2024-04-20T10:21:45.070 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.068+0000 7f1fa4d85740 1 -- 172.21.15.121:0/2771197751 wait complete. 2024-04-20T10:21:45.070 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.068+0000 7f1fa4d85740 20 client.4857 trim_cache size 0 max 16384 2024-04-20T10:21:45.088 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.088+0000 7f1fa4d85740 1 Processor -- start 2024-04-20T10:21:45.088 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.088+0000 7f1fa4d85740 1 -- start start 2024-04-20T10:21:45.089 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.089+0000 7f1fa4d85740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e1b31894e0 0x55e1b31ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:45.089 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.089+0000 7f1fa4d85740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55e1b319b410 0x55e1b319a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:45.089 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.089+0000 7f1fa4d85740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1b31d3460 0x55e1b319ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:45.089 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.089+0000 7f1fa4d85740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c201510 con 0x55e1b31d3460 2024-04-20T10:21:45.089 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.089+0000 7f1fa4d85740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c2009c0 con 0x55e1b31894e0 2024-04-20T10:21:45.090 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.089+0000 7f1fa4d85740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f1f9c200ef0 con 0x55e1b319b410 2024-04-20T10:21:45.090 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.089+0000 7f1f9be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1b31d3460 0x55e1b319ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:45.090 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.089+0000 7f1f9be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1b31d3460 0x55e1b319ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58870/0 (socket says 172.21.15.121:58870) 2024-04-20T10:21:45.090 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.089+0000 7f1f9be00640 1 -- 172.21.15.121:0/2711513346 learned_addr learned my addr 172.21.15.121:0/2711513346 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:45.090 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.090+0000 7f1f9aa00640 1 --2- 172.21.15.121:0/2711513346 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55e1b319b410 0x55e1b319a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:45.090 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.090+0000 7f1f9be00640 1 -- 172.21.15.121:0/2711513346 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55e1b319b410 msgr2=0x55e1b319a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:45.090 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.090+0000 7f1f9be00640 1 --2- 172.21.15.121:0/2711513346 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55e1b319b410 0x55e1b319a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.090 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.090+0000 7f1f9be00640 1 -- 172.21.15.121:0/2711513346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e1b31894e0 msgr2=0x55e1b31ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:45.090 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.090+0000 7f1f9be00640 1 --2- 172.21.15.121:0/2711513346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e1b31894e0 0x55e1b31ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.090 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.090+0000 7f1f9be00640 1 -- 172.21.15.121:0/2711513346 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1b31b0de0 con 0x55e1b31d3460 2024-04-20T10:21:45.090 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.090+0000 7f1f9be00640 1 --2- 172.21.15.121:0/2711513346 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1b31d3460 0x55e1b319ae70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1f9402c0d0 tx=0x7f1f94031000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c37a58c49ccd8f4d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:45.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.090+0000 7f1f8aa00640 1 -- 172.21.15.121:0/2711513346 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f94035040 con 0x55e1b31d3460 2024-04-20T10:21:45.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.090+0000 7f1f8aa00640 1 -- 172.21.15.121:0/2711513346 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f94009490 con 0x55e1b31d3460 2024-04-20T10:21:45.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.090+0000 7f1f8aa00640 1 -- 172.21.15.121:0/2711513346 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9403acb0 con 0x55e1b31d3460 2024-04-20T10:21:45.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.091+0000 7f1fa4d85740 1 -- 172.21.15.121:0/2711513346 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1b31d3460 msgr2=0x55e1b319ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:45.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.091+0000 7f1fa4d85740 1 --2- 172.21.15.121:0/2711513346 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1b31d3460 0x55e1b319ae70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1f9402c0d0 tx=0x7f1f94031000 comp rx=0 tx=0).stop 2024-04-20T10:21:45.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.091+0000 7f1fa4d85740 1 -- 172.21.15.121:0/2711513346 shutdown_connections 2024-04-20T10:21:45.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.091+0000 7f1fa4d85740 1 --2- 172.21.15.121:0/2711513346 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1b31d3460 0x55e1b319ae70 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.091+0000 7f1fa4d85740 1 --2- 172.21.15.121:0/2711513346 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55e1b319b410 0x55e1b319a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.091+0000 7f1fa4d85740 1 --2- 172.21.15.121:0/2711513346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e1b31894e0 0x55e1b31ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.091+0000 7f1fa4d85740 1 -- 172.21.15.121:0/2711513346 >> 172.21.15.121:0/2711513346 conn(0x55e1b3185930 msgr2=0x55e1b31b96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:45.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.091+0000 7f1fa4d85740 1 -- 172.21.15.121:0/2711513346 shutdown_connections 2024-04-20T10:21:45.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.091+0000 7f1fa4d85740 1 -- 172.21.15.121:0/2711513346 wait complete. 2024-04-20T10:21:45.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.092+0000 7f1fa4d85740 1 -- start start 2024-04-20T10:21:45.093 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.092+0000 7f1fa4d85740 1 Processor -- start 2024-04-20T10:21:45.095 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.093+0000 7f1fa4d85740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55e1b31894e0 0x55e1b31abcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:45.095 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.093+0000 7f1fa4d85740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e1b319b410 0x55e1b31ac200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:45.095 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.093+0000 7f1fa4d85740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1b31d3460 0x55e1b31b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:45.095 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.093+0000 7f1fa4d85740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x55e1b31d54d0 con 0x55e1b31d3460 2024-04-20T10:21:45.096 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.093+0000 7f1fa4d85740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e1b31d62f0 con 0x55e1b319b410 2024-04-20T10:21:45.096 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.093+0000 7f1fa4d85740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x55e1b31d5b20 con 0x55e1b31894e0 2024-04-20T10:21:45.096 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.093+0000 7f1fa4d85740 10 client.0 _ll_register_callbacks cb 0x7f1f940096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:21:45.096 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.093+0000 7f1f9be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1b31d3460 0x55e1b31b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:45.096 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.093+0000 7f1f9be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1b31d3460 0x55e1b31b3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58876/0 (socket says 172.21.15.121:58876) 2024-04-20T10:21:45.096 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.093+0000 7f1f9be00640 1 -- 172.21.15.121:0/2472237639 learned_addr learned my addr 172.21.15.121:0/2472237639 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:45.096 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.094+0000 7f1f9b400640 1 --2- 172.21.15.121:0/2472237639 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55e1b31894e0 0x55e1b31abcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:45.096 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.094+0000 7f1f9be00640 1 -- 172.21.15.121:0/2472237639 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55e1b31894e0 msgr2=0x55e1b31abcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:45.096 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.094+0000 7f1f9be00640 1 --2- 172.21.15.121:0/2472237639 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55e1b31894e0 0x55e1b31abcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.096 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.094+0000 7f1f9be00640 1 -- 172.21.15.121:0/2472237639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e1b319b410 msgr2=0x55e1b31ac200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:45.096 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.094+0000 7f1f9be00640 1 --2- 172.21.15.121:0/2472237639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e1b319b410 0x55e1b31ac200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.096 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.094+0000 7f1f9be00640 1 -- 172.21.15.121:0/2472237639 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1b3235810 con 0x55e1b31d3460 2024-04-20T10:21:45.096 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.094+0000 7f1f9b400640 1 --2- 172.21.15.121:0/2472237639 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55e1b31894e0 0x55e1b31abcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:45.096 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.094+0000 7f1f9be00640 1 --2- 172.21.15.121:0/2472237639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1b31d3460 0x55e1b31b3ca0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1f94001000 tx=0x7f1f94009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b72facd25c1177 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:45.096 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.094+0000 7f1f83e00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-04-20T10:21:45.097 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.094+0000 7f1f83e00640 1 -- 172.21.15.121:0/2472237639 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f94057030 con 0x55e1b31d3460 2024-04-20T10:21:45.097 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.094+0000 7f1f83e00640 1 -- 172.21.15.121:0/2472237639 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f1f94004240 con 0x55e1b31d3460 2024-04-20T10:21:45.097 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.094+0000 7f1fa4d85740 1 -- 172.21.15.121:0/2472237639 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f94009490 con 0x55e1b31d3460 2024-04-20T10:21:45.097 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.094+0000 7f1f83e00640 1 -- 172.21.15.121:0/2472237639 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f1f94060050 con 0x55e1b31d3460 2024-04-20T10:21:45.097 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.095+0000 7f1fa4d85740 10 client.4863 fetch_fsmap learned FSMap version 8 2024-04-20T10:21:45.097 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.095+0000 7f1fa4d85740 1 -- 172.21.15.121:0/2472237639 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55e1b31977c0 con 0x55e1b31d3460 2024-04-20T10:21:45.097 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.095+0000 7f1f83e00640 1 -- 172.21.15.121:0/2472237639 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f94009490 con 0x55e1b31d3460 2024-04-20T10:21:45.097 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.095+0000 7f1f83e00640 1 -- 172.21.15.121:0/2472237639 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 8) ==== 3503+0+0 (secure 0 0 0) 0x7f1f94063020 con 0x55e1b31d3460 2024-04-20T10:21:45.097 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.095+0000 7f1fa4d85740 10 client.4863 fetch_fsmap finished waiting for FSMap version 8 2024-04-20T10:21:45.097 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.095+0000 7f1fa4d85740 10 client.4863 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-20T10:21:45.097 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.095+0000 7f1fa4d85740 20 client.4863 populate_metadata read hostname 'smithi121' 2024-04-20T10:21:45.097 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.095+0000 7f1fa4d85740 1 --2- 172.21.15.121:0/2472237639 >> [v2:172.21.15.123:6834/3336775250,v1:172.21.15.123:6835/3336775250] conn(0x55e1b31d19e0 0x55e1b31d1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:45.097 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.095+0000 7f1fa4d85740 4 client.4863 mds_command: new command op to 4281 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T10:21:45.097 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.095+0000 7f1fa4d85740 1 -- 172.21.15.121:0/2472237639 --> [v2:172.21.15.123:6834/3336775250,v1:172.21.15.123:6835/3336775250] -- command(tid 0: {"prefix": "status"}) -- 0x55e1b31d32b0 con 0x55e1b31d19e0 2024-04-20T10:21:45.098 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.096+0000 7f1f9b400640 1 --2- 172.21.15.121:0/2472237639 >> [v2:172.21.15.123:6834/3336775250,v1:172.21.15.123:6835/3336775250] conn(0x55e1b31d19e0 0x55e1b31d1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:45.099 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.097+0000 7f1f9b400640 1 --2- 172.21.15.121:0/2472237639 >> [v2:172.21.15.123:6834/3336775250,v1:172.21.15.123:6835/3336775250] conn(0x55e1b31d19e0 0x55e1b31d1dc0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57f7cfc63cf603b6 server_cookie=12abedabe0520c86 in_seq=0 out_seq=0 2024-04-20T10:21:45.099 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.097+0000 7f1f83e00640 10 client.4863 ms_handle_connect on v2:172.21.15.123:6834/3336775250 2024-04-20T10:21:45.099 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.097+0000 7f1f83e00640 1 -- 172.21.15.121:0/2472237639 <== mds.0 v2:172.21.15.123:6834/3336775250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55e1b31d32b0 con 0x55e1b31d19e0 2024-04-20T10:21:45.099 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.097+0000 7f1f83e00640 10 client.4863 handle_command_reply: tid=0 2024-04-20T10:21:45.099 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.097+0000 7f1fa4d85740 1 client.4863 shutdown 2024-04-20T10:21:45.099 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.098+0000 7f1fa4d85740 1 -- 172.21.15.121:0/2472237639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1b31d3460 msgr2=0x55e1b31b3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:45.099 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.098+0000 7f1fa4d85740 1 --2- 172.21.15.121:0/2472237639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1b31d3460 0x55e1b31b3ca0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1f94001000 tx=0x7f1f94009240 comp rx=0 tx=0).stop 2024-04-20T10:21:45.099 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.098+0000 7f1fa4d85740 1 -- 172.21.15.121:0/2472237639 shutdown_connections 2024-04-20T10:21:45.099 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.098+0000 7f1fa4d85740 1 --2- 172.21.15.121:0/2472237639 >> [v2:172.21.15.123:6834/3336775250,v1:172.21.15.123:6835/3336775250] conn(0x55e1b31d19e0 0x55e1b31d1dc0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.099 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.098+0000 7f1fa4d85740 1 --2- 172.21.15.121:0/2472237639 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x55e1b31d3460 0x55e1b31b3ca0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.100 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.098+0000 7f1fa4d85740 1 --2- 172.21.15.121:0/2472237639 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e1b319b410 0x55e1b31ac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.100 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.098+0000 7f1fa4d85740 1 --2- 172.21.15.121:0/2472237639 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x55e1b31894e0 0x55e1b31abcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.100 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.098+0000 7f1f83e00640 0 client.4863 ms_handle_reset on v2:172.21.15.123:6834/3336775250 2024-04-20T10:21:45.100 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.098+0000 7f1fa4d85740 1 -- 172.21.15.121:0/2472237639 >> 172.21.15.121:0/2472237639 conn(0x55e1b3185930 msgr2=0x55e1b31d0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:45.100 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.098+0000 7f1fa4d85740 1 -- 172.21.15.121:0/2472237639 shutdown_connections 2024-04-20T10:21:45.100 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.098+0000 7f1fa4d85740 1 -- 172.21.15.121:0/2472237639 wait complete. 2024-04-20T10:21:45.100 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.099+0000 7f1fa4d85740 20 client.4863 trim_cache size 0 max 16384 2024-04-20T10:21:45.100 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T10:21:45.100 INFO:teuthology.orchestra.run.smithi121.stdout: "cluster_fsid": "ceaac016-4ab2-472d-a36e-7bfe9497c762", 2024-04-20T10:21:45.100 INFO:teuthology.orchestra.run.smithi121.stdout: "whoami": 0, 2024-04-20T10:21:45.100 INFO:teuthology.orchestra.run.smithi121.stdout: "id": 4281, 2024-04-20T10:21:45.101 INFO:teuthology.orchestra.run.smithi121.stdout: "want_state": "up:active", 2024-04-20T10:21:45.101 INFO:teuthology.orchestra.run.smithi121.stdout: "state": "up:active", 2024-04-20T10:21:45.101 INFO:teuthology.orchestra.run.smithi121.stdout: "fs_name": "cephfs", 2024-04-20T10:21:45.101 INFO:teuthology.orchestra.run.smithi121.stdout: "rank_uptime": 42.495787638000003, 2024-04-20T10:21:45.101 INFO:teuthology.orchestra.run.smithi121.stdout: "mdsmap_epoch": 8, 2024-04-20T10:21:45.101 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch": 27, 2024-04-20T10:21:45.101 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch_barrier": 25, 2024-04-20T10:21:45.101 INFO:teuthology.orchestra.run.smithi121.stdout: "uptime": 47.537011151999998 2024-04-20T10:21:45.101 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T10:21:45.102 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.099+0000 7f1fa2000640 1 -- 172.21.15.121:0/1701853533 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f1f7806c920 msgr2=0x7f1f7806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:45.102 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.099+0000 7f1fa2000640 1 --2- 172.21.15.121:0/1701853533 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f1f7806c920 0x7f1f7806ede0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c0cce90 tx=0x7f1f840084f0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.102 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.099+0000 7f1fa2000640 1 -- 172.21.15.121:0/1701853533 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f9c06c010 msgr2=0x7f1f9c149110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:45.102 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.099+0000 7f1fa2000640 1 --2- 172.21.15.121:0/1701853533 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f9c06c010 0x7f1f9c149110 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c00b420 tx=0x7f1f8c00b8e0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.102 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.099+0000 7f1fa2000640 1 -- 172.21.15.121:0/1701853533 shutdown_connections 2024-04-20T10:21:45.102 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.099+0000 7f1fa2000640 1 --2- 172.21.15.121:0/1701853533 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f1f7806c920 0x7f1f7806ede0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.102 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.099+0000 7f1fa2000640 1 --2- 172.21.15.121:0/1701853533 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f9c15e740 0x7f1f9c0b10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.102 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.099+0000 7f1fa2000640 1 --2- 172.21.15.121:0/1701853533 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1f9c0709c0 0x7f1f9c160260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.102 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.099+0000 7f1fa2000640 1 --2- 172.21.15.121:0/1701853533 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1f9c06c010 0x7f1f9c149110 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.103 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.099+0000 7f1fa2000640 1 -- 172.21.15.121:0/1701853533 >> 172.21.15.121:0/1701853533 conn(0x7f1f9c0cd530 msgr2=0x7f1f9c06e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:45.103 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.099+0000 7f1fa2000640 1 -- 172.21.15.121:0/1701853533 shutdown_connections 2024-04-20T10:21:45.103 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.099+0000 7f1fa2000640 1 -- 172.21.15.121:0/1701853533 wait complete. 2024-04-20T10:21:45.116 INFO:teuthology.run_tasks:Running task sequential... 2024-04-20T10:21:45.125 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-20T10:21:45.125 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-20T10:21:45.125 INFO:teuthology.task.print:Enabling mgr modules 2024-04-20T10:21:45.125 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-20T10:21:45.145 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-20T10:21:45.146 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-20T10:21:45.146 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-20T10:21:45.146 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-20T10:21:45.146 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T10:21:45.146 INFO:tasks.ceph_fuse:client.2 config is {} 2024-04-20T10:21:45.146 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-20T10:21:45.146 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-20T10:21:45.146 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-20T10:21:45.146 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-20T10:21:45.147 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-20T10:21:45.147 INFO:tasks.ceph_fuse:client.3 config is {} 2024-04-20T10:21:45.147 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-20T10:21:45.147 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-20T10:21:45.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:45.147 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2024-04-20T10:21:45.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:45.206 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link delete ceph-brx 2024-04-20T10:21:45.295 INFO:teuthology.orchestra.run.smithi123.stderr:Cannot find device "ceph-brx" 2024-04-20T10:21:45.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:21:45.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:45.299 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2024-04-20T10:21:45.357 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-20T10:21:45.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:45.358 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-20T10:21:45.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:45.449 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link delete ceph-brx 2024-04-20T10:21:45.551 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-20T10:21:45.551 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T10:21:45.727 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.727+0000 7f9e25a00640 1 Processor -- start 2024-04-20T10:21:45.727 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.728+0000 7f9e25a00640 1 -- start start 2024-04-20T10:21:45.729 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.729+0000 7f9e25a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9e2006dcb0 0x7f9e20071b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:45.729 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.729+0000 7f9e25a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e200ade40 con 0x7f9e2015b3b0 2024-04-20T10:21:45.729 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.729+0000 7f9e25a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e200ae0e0 con 0x7f9e20072090 2024-04-20T10:21:45.729 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.729+0000 7f9e25a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f9e2006f1b0 con 0x7f9e2006dcb0 2024-04-20T10:21:45.729 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.729+0000 7f9e1ea00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f9e2015b3b0 0x7f9e2006eaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:47696/0 (socket says 172.21.15.121:47696) 2024-04-20T10:21:45.729 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.729+0000 7f9e1ea00640 1 -- 172.21.15.121:0/197519427 learned_addr learned my addr 172.21.15.121:0/197519427 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:45.729 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.729+0000 7f9e16000640 1 --2- 172.21.15.121:0/197519427 >> v2:172.21.15.121:3300/0 conn(0x7f9e2006dcb0 0x7f9e20071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:45.730 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.730+0000 7f9e1e000640 1 -- 172.21.15.121:0/197519427 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3754828211 0 0) 0x7f9e200ade40 con 0x7f9e2015b3b0 2024-04-20T10:21:45.730 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.730+0000 7f9e16000640 1 -- 172.21.15.121:0/197519427 >> v1:172.21.15.123:6789/0 conn(0x7f9e20072090 legacy=0x7f9e200d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:45.730 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.730+0000 7f9e16000640 1 -- 172.21.15.121:0/197519427 >> v1:172.21.15.121:6789/0 conn(0x7f9e2015b3b0 legacy=0x7f9e2006eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:45.730 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.730+0000 7f9e16000640 1 -- 172.21.15.121:0/197519427 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e2015b790 con 0x7f9e2006dcb0 2024-04-20T10:21:45.731 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.731+0000 7f9e16000640 1 --2- 172.21.15.121:0/197519427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e2006dcb0 0x7f9e20071b50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9e200ad8d0 tx=0x7f9e0802b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=688af37352bfebb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:45.731 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.731+0000 7f9e1e000640 1 -- 172.21.15.121:0/197519427 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e08035020 con 0x7f9e2006dcb0 2024-04-20T10:21:45.731 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.731+0000 7f9e1e000640 1 -- 172.21.15.121:0/197519427 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e08031800 con 0x7f9e2006dcb0 2024-04-20T10:21:45.731 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.731+0000 7f9e1e000640 1 -- 172.21.15.121:0/197519427 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e08031bd0 con 0x7f9e2006dcb0 2024-04-20T10:21:45.732 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.732+0000 7f9e25a00640 1 -- 172.21.15.121:0/197519427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e2006dcb0 msgr2=0x7f9e20071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:45.732 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.732+0000 7f9e25a00640 1 --2- 172.21.15.121:0/197519427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e2006dcb0 0x7f9e20071b50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9e200ad8d0 tx=0x7f9e0802b8a0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.732 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.732+0000 7f9e25a00640 1 -- 172.21.15.121:0/197519427 shutdown_connections 2024-04-20T10:21:45.732 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.732+0000 7f9e25a00640 1 --2- 172.21.15.121:0/197519427 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e2006dcb0 0x7f9e20071b50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.732 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.732+0000 7f9e25a00640 1 -- 172.21.15.121:0/197519427 >> 172.21.15.121:0/197519427 conn(0x7f9e200b3350 msgr2=0x7f9e200b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:45.732 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.732+0000 7f9e25a00640 1 -- 172.21.15.121:0/197519427 shutdown_connections 2024-04-20T10:21:45.732 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.732+0000 7f9e25a00640 1 -- 172.21.15.121:0/197519427 wait complete. 2024-04-20T10:21:45.733 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.734+0000 7f9e25a00640 1 Processor -- start 2024-04-20T10:21:45.733 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.734+0000 7f9e25a00640 1 -- start start 2024-04-20T10:21:45.734 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.735+0000 7f9e25a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e2006dcb0 0x7f9e200af140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:45.735 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.735+0000 7f9e25a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e2015b3b0 0x7f9e20156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:45.735 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.735+0000 7f9e25a00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e20149770 0x7f9e201569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:45.735 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.735+0000 7f9e25a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9e200aec20 con 0x7f9e2006dcb0 2024-04-20T10:21:45.735 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.735+0000 7f9e25a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9e200ae940 con 0x7f9e2015b3b0 2024-04-20T10:21:45.735 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.735+0000 7f9e25a00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f9e200d0a70 con 0x7f9e20149770 2024-04-20T10:21:45.735 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.735+0000 7f9e1ea00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e2006dcb0 0x7f9e200af140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:45.735 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.735+0000 7f9e1ea00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e2006dcb0 0x7f9e200af140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58882/0 (socket says 172.21.15.121:58882) 2024-04-20T10:21:45.735 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.735+0000 7f9e1ea00640 1 -- 172.21.15.121:0/3794394659 learned_addr learned my addr 172.21.15.121:0/3794394659 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:45.736 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.736+0000 7f9e16000640 1 --2- 172.21.15.121:0/3794394659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e2015b3b0 0x7f9e20156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:45.736 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.736+0000 7f9e1f400640 1 --2- 172.21.15.121:0/3794394659 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e20149770 0x7f9e201569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:45.736 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.736+0000 7f9e1ea00640 1 -- 172.21.15.121:0/3794394659 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e20149770 msgr2=0x7f9e201569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:45.736 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.736+0000 7f9e1ea00640 1 --2- 172.21.15.121:0/3794394659 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e20149770 0x7f9e201569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.736 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.736+0000 7f9e1ea00640 1 -- 172.21.15.121:0/3794394659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e2015b3b0 msgr2=0x7f9e20156470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:45.737 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.736+0000 7f9e1ea00640 1 --2- 172.21.15.121:0/3794394659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e2015b3b0 0x7f9e20156470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.737 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.736+0000 7f9e1ea00640 1 -- 172.21.15.121:0/3794394659 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e200af990 con 0x7f9e2006dcb0 2024-04-20T10:21:45.737 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.736+0000 7f9e16000640 1 --2- 172.21.15.121:0/3794394659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e2015b3b0 0x7f9e20156470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:45.737 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.736+0000 7f9e1ea00640 1 --2- 172.21.15.121:0/3794394659 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e2006dcb0 0x7f9e200af140 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9e1000c460 tx=0x7f9e1000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:45.737 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.738+0000 7f9e17400640 1 -- 172.21.15.121:0/3794394659 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e10018070 con 0x7f9e2006dcb0 2024-04-20T10:21:45.738 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.738+0000 7f9e17400640 1 -- 172.21.15.121:0/3794394659 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e10007500 con 0x7f9e2006dcb0 2024-04-20T10:21:45.738 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.738+0000 7f9e25a00640 1 -- 172.21.15.121:0/3794394659 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e200afc70 con 0x7f9e2006dcb0 2024-04-20T10:21:45.738 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.738+0000 7f9e17400640 1 -- 172.21.15.121:0/3794394659 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e10004a20 con 0x7f9e2006dcb0 2024-04-20T10:21:45.738 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.738+0000 7f9e25a00640 1 -- 172.21.15.121:0/3794394659 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e200af680 con 0x7f9e2006dcb0 2024-04-20T10:21:45.739 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.739+0000 7f9e25a00640 1 -- 172.21.15.121:0/3794394659 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e2006ef70 con 0x7f9e2006dcb0 2024-04-20T10:21:45.739 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.740+0000 7f9e17400640 1 -- 172.21.15.121:0/3794394659 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9e1001b070 con 0x7f9e2006dcb0 2024-04-20T10:21:45.741 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.741+0000 7f9e17400640 1 --2- 172.21.15.121:0/3794394659 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9dfc06c8f0 0x7f9dfc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:45.741 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.741+0000 7f9e17400640 1 -- 172.21.15.121:0/3794394659 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f9e1008b170 con 0x7f9e2006dcb0 2024-04-20T10:21:45.741 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.742+0000 7f9e16000640 1 --2- 172.21.15.121:0/3794394659 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9dfc06c8f0 0x7f9dfc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:45.742 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.742+0000 7f9e16000640 1 --2- 172.21.15.121:0/3794394659 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9dfc06c8f0 0x7f9dfc06edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9e08036000 tx=0x7f9e08008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:45.745 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.746+0000 7f9e17400640 1 -- 172.21.15.121:0/3794394659 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9e2006ef70 con 0x7f9e2006dcb0 2024-04-20T10:21:45.991 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.991+0000 7f9e25a00640 1 -- 172.21.15.121:0/3794394659 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9e20068980 con 0x7f9e2006dcb0 2024-04-20T10:21:45.992 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.992+0000 7f9e17400640 1 -- 172.21.15.121:0/3794394659 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7f9e10059230 con 0x7f9e2006dcb0 2024-04-20T10:21:45.992 INFO:teuthology.orchestra.run.smithi121.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-20T10:21:45.994 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.995+0000 7f9e25a00640 1 -- 172.21.15.121:0/3794394659 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9dfc06c8f0 msgr2=0x7f9dfc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:45.995 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.995+0000 7f9e25a00640 1 --2- 172.21.15.121:0/3794394659 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9dfc06c8f0 0x7f9dfc06edb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9e08036000 tx=0x7f9e08008030 comp rx=0 tx=0).stop 2024-04-20T10:21:45.995 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.995+0000 7f9e25a00640 1 -- 172.21.15.121:0/3794394659 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e2006dcb0 msgr2=0x7f9e200af140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:45.995 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.995+0000 7f9e25a00640 1 --2- 172.21.15.121:0/3794394659 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e2006dcb0 0x7f9e200af140 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9e1000c460 tx=0x7f9e1000c920 comp rx=0 tx=0).stop 2024-04-20T10:21:45.995 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.995+0000 7f9e25a00640 1 -- 172.21.15.121:0/3794394659 shutdown_connections 2024-04-20T10:21:45.995 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.995+0000 7f9e25a00640 1 --2- 172.21.15.121:0/3794394659 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9e20149770 0x7f9e201569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.995 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.995+0000 7f9e25a00640 1 --2- 172.21.15.121:0/3794394659 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9dfc06c8f0 0x7f9dfc06edb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.995 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.995+0000 7f9e25a00640 1 --2- 172.21.15.121:0/3794394659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e2015b3b0 0x7f9e20156470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.995 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.995+0000 7f9e25a00640 1 --2- 172.21.15.121:0/3794394659 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9e2006dcb0 0x7f9e200af140 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:45.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.995+0000 7f9e25a00640 1 -- 172.21.15.121:0/3794394659 >> 172.21.15.121:0/3794394659 conn(0x7f9e200b3350 msgr2=0x7f9e20072480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:45.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.996+0000 7f9e25a00640 1 -- 172.21.15.121:0/3794394659 shutdown_connections 2024-04-20T10:21:45.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:45.996+0000 7f9e25a00640 1 -- 172.21.15.121:0/3794394659 wait complete. 2024-04-20T10:21:46.013 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T10:21:46.013 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T10:21:46.013 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2024-04-20T10:21:46.013 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T10:21:46.014 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T10:21:46.014 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T10:21:46.014 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T10:21:46.014 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T10:21:46.014 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T10:21:46.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:46.014 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2024-04-20T10:21:46.060 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T10:21:46.060 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T10:21:46.060 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T10:21:46.060 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:21:46.060 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2024-04-20T10:21:46.060 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:21:46.060 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:21:46.061 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:21:46.061 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2024-04-20T10:21:46.061 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:21:46.061 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:21:46.061 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2024-04-20T10:21:46.061 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:21:46.061 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:21:46.061 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T10:21:46.061 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:21:46.061 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T10:21:46.061 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 42602sec preferred_lft 42602sec 2024-04-20T10:21:46.061 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ec4:7aff:fe8f:cba3/64 scope link 2024-04-20T10:21:46.061 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:21:46.062 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-20T10:21:46.062 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:21:46.063 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-20T10:21:46.063 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link add name ceph-brx type bridge 2024-04-20T10:21:46.063 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip addr flush dev ceph-brx 2024-04-20T10:21:46.063 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set ceph-brx up 2024-04-20T10:21:46.063 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-20T10:21:46.063 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-20T10:21:46.212 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:21:46.339 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:21:46.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:46.352 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T10:21:46.416 INFO:teuthology.orchestra.run.smithi121.stdout:1 2024-04-20T10:21:46.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:46.422 DEBUG:teuthology.orchestra.run.smithi121:> route 2024-04-20T10:21:46.464 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2024-04-20T10:21:46.464 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T10:21:46.464 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T10:21:46.464 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T10:21:46.464 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T10:21:46.464 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:21:46.464 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-20T10:21:46.464 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T10:21:46.465 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T10:21:46.465 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T10:21:46.465 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-20T10:21:46.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:21:46.705 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:21:46.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:46.716 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2024-04-20T10:21:46.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:46.761 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list-id 2024-04-20T10:21:46.855 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:21:46.855 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-20T10:21:46.855 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:21:46.855 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-20T10:21:46.855 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-20T10:21:46.966 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:21:47.033 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:21:47.043 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-20T10:21:47.043 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:21:47.043 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-20T10:21:47.044 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-20T10:21:47.044 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-20T10:21:47.044 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-20T10:21:47.044 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-20T10:21:47.044 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-20T10:21:47.044 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-20T10:21:47.146 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:21:47.298 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:21:47.307 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:21:47.307 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-20T10:21:47.307 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.0 up 2024-04-20T10:21:47.307 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set dev brx.0 master ceph-brx 2024-04-20T10:21:47.307 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-20T10:21:47.401 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:21:47.496 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:21:47.505 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-20T10:21:47.505 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T10:21:47.505 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T10:21:47.547 INFO:teuthology.orchestra.run.smithi121.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T10:21:47.547 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T10:21:47.547 DEBUG:teuthology.orchestra.run.smithi121:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T10:21:47.639 DEBUG:teuthology.orchestra.run.smithi121:> sudo modprobe fuse 2024-04-20T10:21:47.713 DEBUG:teuthology.orchestra.run.smithi121:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T10:21:47.766 INFO:teuthology.orchestra.run.smithi121.stdout:/proc 2024-04-20T10:21:47.766 INFO:teuthology.orchestra.run.smithi121.stdout:/sys 2024-04-20T10:21:47.766 INFO:teuthology.orchestra.run.smithi121.stdout:/dev 2024-04-20T10:21:47.766 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/security 2024-04-20T10:21:47.766 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/shm 2024-04-20T10:21:47.766 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/pts 2024-04-20T10:21:47.766 INFO:teuthology.orchestra.run.smithi121.stdout:/run 2024-04-20T10:21:47.766 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/cgroup 2024-04-20T10:21:47.766 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/pstore 2024-04-20T10:21:47.766 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/bpf 2024-04-20T10:21:47.767 INFO:teuthology.orchestra.run.smithi121.stdout:/ 2024-04-20T10:21:47.767 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/selinux 2024-04-20T10:21:47.767 INFO:teuthology.orchestra.run.smithi121.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T10:21:47.767 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/hugepages 2024-04-20T10:21:47.767 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/mqueue 2024-04-20T10:21:47.767 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/debug 2024-04-20T10:21:47.767 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/tracing 2024-04-20T10:21:47.767 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/fuse/connections 2024-04-20T10:21:47.767 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/config 2024-04-20T10:21:47.767 INFO:teuthology.orchestra.run.smithi121.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T10:21:47.767 INFO:teuthology.orchestra.run.smithi121.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T10:21:47.767 INFO:teuthology.orchestra.run.smithi121.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T10:21:47.767 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T10:21:47.767 INFO:teuthology.orchestra.run.smithi121.stdout:/run/user/1000 2024-04-20T10:21:47.768 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph 2024-04-20T10:21:47.768 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T10:21:47.768 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T10:21:47.768 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T10:21:47.768 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T10:21:47.768 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns 2024-04-20T10:21:47.768 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:21:47.768 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:21:47.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:47.769 DEBUG:teuthology.orchestra.run.smithi121:> ls /sys/fs/fuse/connections 2024-04-20T10:21:47.817 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T10:21:47.817 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-20T10:21:47.822 DEBUG:teuthology.orchestra.run.smithi121:> sudo modprobe fuse 2024-04-20T10:21:47.885 DEBUG:teuthology.orchestra.run.smithi121:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T10:21:47.925 INFO:teuthology.orchestra.run.smithi121.stdout:/proc 2024-04-20T10:21:47.925 INFO:teuthology.orchestra.run.smithi121.stdout:/sys 2024-04-20T10:21:47.925 INFO:teuthology.orchestra.run.smithi121.stdout:/dev 2024-04-20T10:21:47.926 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/security 2024-04-20T10:21:47.926 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/shm 2024-04-20T10:21:47.926 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/pts 2024-04-20T10:21:47.926 INFO:teuthology.orchestra.run.smithi121.stdout:/run 2024-04-20T10:21:47.926 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/cgroup 2024-04-20T10:21:47.926 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/pstore 2024-04-20T10:21:47.926 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/bpf 2024-04-20T10:21:47.926 INFO:teuthology.orchestra.run.smithi121.stdout:/ 2024-04-20T10:21:47.926 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/selinux 2024-04-20T10:21:47.926 INFO:teuthology.orchestra.run.smithi121.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T10:21:47.926 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/hugepages 2024-04-20T10:21:47.926 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/mqueue 2024-04-20T10:21:47.927 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/debug 2024-04-20T10:21:47.927 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/tracing 2024-04-20T10:21:47.927 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/fuse/connections 2024-04-20T10:21:47.927 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/config 2024-04-20T10:21:47.927 INFO:teuthology.orchestra.run.smithi121.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T10:21:47.927 INFO:teuthology.orchestra.run.smithi121.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T10:21:47.927 INFO:teuthology.orchestra.run.smithi121.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T10:21:47.927 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T10:21:47.927 INFO:teuthology.orchestra.run.smithi121.stdout:/run/user/1000 2024-04-20T10:21:47.927 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph 2024-04-20T10:21:47.927 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T10:21:47.927 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T10:21:47.927 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T10:21:47.927 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T10:21:47.927 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns 2024-04-20T10:21:47.928 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:21:47.928 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:21:47.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:47.929 DEBUG:teuthology.orchestra.run.smithi121:> ls /sys/fs/fuse/connections 2024-04-20T10:21:47.985 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi121.stderr:2024-04-20T10:21:47.985+0000 7f93371b8480 -1 init, newargv = 0x558569924590 newargc=15 2024-04-20T10:21:47.985 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi121.stderr:ceph-fuse[33190]: starting ceph client 2024-04-20T10:21:47.997 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi121.stderr:ceph-fuse[33190]: starting fuse 2024-04-20T10:21:48.012 INFO:teuthology.orchestra.run.smithi121.stdout:41 2024-04-20T10:21:48.012 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-20T10:21:48.012 DEBUG:teuthology.orchestra.run.smithi121:> sudo stdin-killer -- python3 -c ' 2024-04-20T10:21:48.013 DEBUG:teuthology.orchestra.run.smithi121:> import glob 2024-04-20T10:21:48.013 DEBUG:teuthology.orchestra.run.smithi121:> import re 2024-04-20T10:21:48.013 DEBUG:teuthology.orchestra.run.smithi121:> import os 2024-04-20T10:21:48.013 DEBUG:teuthology.orchestra.run.smithi121:> import subprocess 2024-04-20T10:21:48.013 DEBUG:teuthology.orchestra.run.smithi121:> 2024-04-20T10:21:48.013 DEBUG:teuthology.orchestra.run.smithi121:> def _find_admin_socket(client_name): 2024-04-20T10:21:48.013 DEBUG:teuthology.orchestra.run.smithi121:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-20T10:21:48.013 DEBUG:teuthology.orchestra.run.smithi121:> files = glob.glob(asok_path) 2024-04-20T10:21:48.013 DEBUG:teuthology.orchestra.run.smithi121:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-20T10:21:48.013 DEBUG:teuthology.orchestra.run.smithi121:> 2024-04-20T10:21:48.013 DEBUG:teuthology.orchestra.run.smithi121:> # Given a non-glob path, it better be there 2024-04-20T10:21:48.013 DEBUG:teuthology.orchestra.run.smithi121:> if "*" not in asok_path: 2024-04-20T10:21:48.013 DEBUG:teuthology.orchestra.run.smithi121:> assert(len(files) == 1) 2024-04-20T10:21:48.013 DEBUG:teuthology.orchestra.run.smithi121:> return files[0] 2024-04-20T10:21:48.013 DEBUG:teuthology.orchestra.run.smithi121:> 2024-04-20T10:21:48.013 DEBUG:teuthology.orchestra.run.smithi121:> for f in files: 2024-04-20T10:21:48.014 DEBUG:teuthology.orchestra.run.smithi121:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T10:21:48.014 DEBUG:teuthology.orchestra.run.smithi121:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T10:21:48.014 DEBUG:teuthology.orchestra.run.smithi121:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T10:21:48.014 DEBUG:teuthology.orchestra.run.smithi121:> contents = proc_f.read() 2024-04-20T10:21:48.014 DEBUG:teuthology.orchestra.run.smithi121:> if mountpoint in contents: 2024-04-20T10:21:48.014 DEBUG:teuthology.orchestra.run.smithi121:> return f 2024-04-20T10:21:48.014 DEBUG:teuthology.orchestra.run.smithi121:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T10:21:48.014 DEBUG:teuthology.orchestra.run.smithi121:> 2024-04-20T10:21:48.014 DEBUG:teuthology.orchestra.run.smithi121:> print(_find_admin_socket("client.0")) 2024-04-20T10:21:48.014 DEBUG:teuthology.orchestra.run.smithi121:> ' 2024-04-20T10:21:48.203 INFO:teuthology.orchestra.run.smithi121.stdout:/var/run/ceph/ceph-client.0.33190.asok 2024-04-20T10:21:48.207 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:21:48.220 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.33190.asok 2024-04-20T10:21:48.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:48.220 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.33190.asok status 2024-04-20T10:21:48.449 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T10:21:48.449 INFO:teuthology.orchestra.run.smithi121.stdout: "metadata": { 2024-04-20T10:21:48.449 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T10:21:48.449 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T10:21:48.449 INFO:teuthology.orchestra.run.smithi121.stdout: "entity_id": "0", 2024-04-20T10:21:48.449 INFO:teuthology.orchestra.run.smithi121.stdout: "hostname": "smithi121", 2024-04-20T10:21:48.449 INFO:teuthology.orchestra.run.smithi121.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-20T10:21:48.450 INFO:teuthology.orchestra.run.smithi121.stdout: "pid": "33190", 2024-04-20T10:21:48.450 INFO:teuthology.orchestra.run.smithi121.stdout: "root": "/" 2024-04-20T10:21:48.450 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T10:21:48.450 INFO:teuthology.orchestra.run.smithi121.stdout: "dentry_count": 0, 2024-04-20T10:21:48.450 INFO:teuthology.orchestra.run.smithi121.stdout: "dentry_pinned_count": 0, 2024-04-20T10:21:48.450 INFO:teuthology.orchestra.run.smithi121.stdout: "id": 4878, 2024-04-20T10:21:48.450 INFO:teuthology.orchestra.run.smithi121.stdout: "inst": { 2024-04-20T10:21:48.450 INFO:teuthology.orchestra.run.smithi121.stdout: "name": { 2024-04-20T10:21:48.450 INFO:teuthology.orchestra.run.smithi121.stdout: "type": "client", 2024-04-20T10:21:48.450 INFO:teuthology.orchestra.run.smithi121.stdout: "num": 4878 2024-04-20T10:21:48.450 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T10:21:48.450 INFO:teuthology.orchestra.run.smithi121.stdout: "addr": { 2024-04-20T10:21:48.450 INFO:teuthology.orchestra.run.smithi121.stdout: "type": "v1", 2024-04-20T10:21:48.451 INFO:teuthology.orchestra.run.smithi121.stdout: "addr": "192.168.0.1:0", 2024-04-20T10:21:48.451 INFO:teuthology.orchestra.run.smithi121.stdout: "nonce": 3605782673 2024-04-20T10:21:48.451 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T10:21:48.451 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T10:21:48.451 INFO:teuthology.orchestra.run.smithi121.stdout: "addr": { 2024-04-20T10:21:48.451 INFO:teuthology.orchestra.run.smithi121.stdout: "type": "v1", 2024-04-20T10:21:48.451 INFO:teuthology.orchestra.run.smithi121.stdout: "addr": "192.168.0.1:0", 2024-04-20T10:21:48.451 INFO:teuthology.orchestra.run.smithi121.stdout: "nonce": 3605782673 2024-04-20T10:21:48.451 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T10:21:48.451 INFO:teuthology.orchestra.run.smithi121.stdout: "inst_str": "client.4878 192.168.0.1:0/3605782673", 2024-04-20T10:21:48.451 INFO:teuthology.orchestra.run.smithi121.stdout: "addr_str": "192.168.0.1:0/3605782673", 2024-04-20T10:21:48.451 INFO:teuthology.orchestra.run.smithi121.stdout: "inode_count": 1, 2024-04-20T10:21:48.451 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_epoch": 8, 2024-04-20T10:21:48.452 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_epoch": 27, 2024-04-20T10:21:48.452 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_epoch_barrier": 0, 2024-04-20T10:21:48.452 INFO:teuthology.orchestra.run.smithi121.stdout: "blocklisted": false, 2024-04-20T10:21:48.452 INFO:teuthology.orchestra.run.smithi121.stdout: "fs_name": "cephfs" 2024-04-20T10:21:48.452 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T10:21:48.463 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T10:21:48.665 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.666+0000 7f1ff3e00640 1 Processor -- start 2024-04-20T10:21:48.666 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.666+0000 7f1ff3e00640 1 -- start start 2024-04-20T10:21:48.667 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.668+0000 7f1ff3e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1ff415b400 0x7f1ff4072d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:48.667 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.668+0000 7f1ff3e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ff415c7f0 con 0x7f1ff415eec0 2024-04-20T10:21:48.667 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.668+0000 7f1ff3e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ff406ccc0 con 0x7f1ff415fa40 2024-04-20T10:21:48.668 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.668+0000 7f1ff3e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f1ff406d280 con 0x7f1ff415b400 2024-04-20T10:21:48.668 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.668+0000 7f1febe00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f1ff415eec0 0x7f1ff415f330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:38296/0 (socket says 172.21.15.121:38296) 2024-04-20T10:21:48.668 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.668+0000 7f1febe00640 1 -- 172.21.15.121:0/111374435 learned_addr learned my addr 172.21.15.121:0/111374435 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:48.670 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.669+0000 7f1ff2a00640 1 --2- 172.21.15.121:0/111374435 >> v2:172.21.15.123:3300/0 conn(0x7f1ff415b400 0x7f1ff4072d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:48.670 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.669+0000 7f1ff2000640 1 -- 172.21.15.121:0/111374435 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3072402902 0 0) 0x7f1ff415c7f0 con 0x7f1ff415eec0 2024-04-20T10:21:48.670 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.669+0000 7f1ff2000640 1 -- 172.21.15.121:0/111374435 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1fd8003520 con 0x7f1ff415eec0 2024-04-20T10:21:48.670 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.669+0000 7f1ff2000640 1 -- 172.21.15.121:0/111374435 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2396125358 0 0) 0x7f1fd8003520 con 0x7f1ff415eec0 2024-04-20T10:21:48.670 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.669+0000 7f1ff2000640 1 -- 172.21.15.121:0/111374435 >> v2:172.21.15.123:3300/0 conn(0x7f1ff415b400 msgr2=0x7f1ff4072d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:48.670 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.669+0000 7f1ff2000640 1 --2- 172.21.15.121:0/111374435 >> v2:172.21.15.123:3300/0 conn(0x7f1ff415b400 0x7f1ff4072d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:48.670 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.669+0000 7f1ff2000640 1 -- 172.21.15.121:0/111374435 >> v1:172.21.15.123:6789/0 conn(0x7f1ff415fa40 legacy=0x7f1ff406ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:48.670 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.669+0000 7f1ff2000640 1 -- 172.21.15.121:0/111374435 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff406cfa0 con 0x7f1ff415eec0 2024-04-20T10:21:48.670 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.669+0000 7f1ff2000640 1 -- 172.21.15.121:0/111374435 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f1fdc001670 con 0x7f1ff415eec0 2024-04-20T10:21:48.670 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.669+0000 7f1ff2000640 1 -- 172.21.15.121:0/111374435 >> v1:172.21.15.121:6789/0 conn(0x7f1ff415eec0 legacy=0x7f1ff415f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:48.670 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.669+0000 7f1ff2000640 1 --2- 172.21.15.121:0/111374435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fd8003a00 0x7f1fd8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:48.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.669+0000 7f1ff2000640 1 --2- 172.21.15.121:0/111374435 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fd8004800 0x7f1fd8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:48.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.669+0000 7f1ff2000640 1 --2- 172.21.15.121:0/111374435 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fd80051c0 0x7f1fd800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:48.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.669+0000 7f1ff2000640 1 -- 172.21.15.121:0/111374435 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff406d280 con 0x7f1fd8004800 2024-04-20T10:21:48.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.669+0000 7f1ff2000640 1 -- 172.21.15.121:0/111374435 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd800c0c0 con 0x7f1fd8003a00 2024-04-20T10:21:48.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.669+0000 7f1ff2000640 1 -- 172.21.15.121:0/111374435 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f1fd800c2c0 con 0x7f1fd80051c0 2024-04-20T10:21:48.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.669+0000 7f1febe00640 1 --2- 172.21.15.121:0/111374435 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fd8004800 0x7f1fd8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:48.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.670+0000 7f1ff3400640 1 --2- 172.21.15.121:0/111374435 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fd80051c0 0x7f1fd800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:48.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.670+0000 7f1febe00640 1 -- 172.21.15.121:0/111374435 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fd80051c0 msgr2=0x7f1fd800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:48.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.670+0000 7f1febe00640 1 --2- 172.21.15.121:0/111374435 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1fd80051c0 0x7f1fd800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:48.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.670+0000 7f1febe00640 1 -- 172.21.15.121:0/111374435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fd8003a00 msgr2=0x7f1fd8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:48.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.670+0000 7f1febe00640 1 --2- 172.21.15.121:0/111374435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fd8003a00 0x7f1fd8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:48.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.670+0000 7f1febe00640 1 -- 172.21.15.121:0/111374435 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fd8003520 con 0x7f1fd8004800 2024-04-20T10:21:48.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.670+0000 7f1ff3400640 1 -- 172.21.15.121:0/111374435 reap_dead start 2024-04-20T10:21:48.672 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.671+0000 7f1febe00640 1 --2- 172.21.15.121:0/111374435 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fd8004800 0x7f1fd8004c80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc00c110 tx=0x7f1fd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f011570844904a58 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:48.673 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.671+0000 7f1ff2000640 1 -- 172.21.15.121:0/111374435 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc007c40 con 0x7f1fd8004800 2024-04-20T10:21:48.673 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.671+0000 7f1ff2000640 1 -- 172.21.15.121:0/111374435 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fdc030070 con 0x7f1fd8004800 2024-04-20T10:21:48.673 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.671+0000 7f1ff2000640 1 -- 172.21.15.121:0/111374435 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc005d70 con 0x7f1fd8004800 2024-04-20T10:21:48.673 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.672+0000 7f1ff3e00640 1 -- 172.21.15.121:0/111374435 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fd8004800 msgr2=0x7f1fd8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:48.673 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.672+0000 7f1ff3e00640 1 --2- 172.21.15.121:0/111374435 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fd8004800 0x7f1fd8004c80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc00c110 tx=0x7f1fd8005830 comp rx=0 tx=0).stop 2024-04-20T10:21:48.673 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.672+0000 7f1ff3e00640 1 -- 172.21.15.121:0/111374435 shutdown_connections 2024-04-20T10:21:48.673 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.672+0000 7f1ff3e00640 1 --2- 172.21.15.121:0/111374435 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1fd8004800 0x7f1fd8004c80 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:48.673 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.672+0000 7f1ff3e00640 1 -- 172.21.15.121:0/111374435 >> 172.21.15.121:0/111374435 conn(0x7f1ff40d0040 msgr2=0x7f1ff40d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:48.673 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.672+0000 7f1ff3e00640 1 -- 172.21.15.121:0/111374435 shutdown_connections 2024-04-20T10:21:48.674 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.672+0000 7f1ff3e00640 1 -- 172.21.15.121:0/111374435 wait complete. 2024-04-20T10:21:48.675 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.674+0000 7f1ff3e00640 1 Processor -- start 2024-04-20T10:21:48.676 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.674+0000 7f1ff3e00640 1 -- start start 2024-04-20T10:21:48.676 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.675+0000 7f1ff3e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fd8004800 0x7f1ff4157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:48.676 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.675+0000 7f1ff3e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ff4155a80 0x7f1ff40b4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:48.676 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.675+0000 7f1ff3e00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ff4155ee0 0x7f1ff40b4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:48.676 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.675+0000 7f1ff3e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff415f330 con 0x7f1ff4155a80 2024-04-20T10:21:48.676 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.675+0000 7f1ff3e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff406e720 con 0x7f1fd8004800 2024-04-20T10:21:48.676 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.675+0000 7f1ff3e00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f1ff415f4b0 con 0x7f1ff4155ee0 2024-04-20T10:21:48.676 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.675+0000 7f1febe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ff4155a80 0x7f1ff40b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:48.676 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.675+0000 7f1febe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ff4155a80 0x7f1ff40b4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37456/0 (socket says 172.21.15.121:37456) 2024-04-20T10:21:48.676 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.675+0000 7f1febe00640 1 -- 172.21.15.121:0/2237714228 learned_addr learned my addr 172.21.15.121:0/2237714228 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:48.676 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.675+0000 7f1ff3400640 1 --2- 172.21.15.121:0/2237714228 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ff4155ee0 0x7f1ff40b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:48.676 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.675+0000 7f1febe00640 1 -- 172.21.15.121:0/2237714228 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ff4155ee0 msgr2=0x7f1ff40b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:48.676 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.675+0000 7f1febe00640 1 --2- 172.21.15.121:0/2237714228 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ff4155ee0 0x7f1ff40b4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:48.676 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.675+0000 7f1febe00640 1 -- 172.21.15.121:0/2237714228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fd8004800 msgr2=0x7f1ff4157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:48.677 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.675+0000 7f1febe00640 1 --2- 172.21.15.121:0/2237714228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fd8004800 0x7f1ff4157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:48.677 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.675+0000 7f1febe00640 1 -- 172.21.15.121:0/2237714228 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff4159140 con 0x7f1ff4155a80 2024-04-20T10:21:48.677 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.676+0000 7f1febe00640 1 --2- 172.21.15.121:0/2237714228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ff4155a80 0x7f1ff40b4200 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc02c4a0 tx=0x7f1fdc02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:48.677 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.676+0000 7f1feaa00640 1 -- 172.21.15.121:0/2237714228 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc004bf0 con 0x7f1ff4155a80 2024-04-20T10:21:48.677 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.676+0000 7f1feaa00640 1 -- 172.21.15.121:0/2237714228 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fdc000ed0 con 0x7f1ff4155a80 2024-04-20T10:21:48.677 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.676+0000 7f1feaa00640 1 -- 172.21.15.121:0/2237714228 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fdc007070 con 0x7f1ff4155a80 2024-04-20T10:21:48.678 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.676+0000 7f1ff3e00640 1 -- 172.21.15.121:0/2237714228 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ff4159420 con 0x7f1ff4155a80 2024-04-20T10:21:48.678 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.676+0000 7f1ff3e00640 1 -- 172.21.15.121:0/2237714228 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ff41599e0 con 0x7f1ff4155a80 2024-04-20T10:21:48.678 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.678+0000 7f1ff3e00640 1 -- 172.21.15.121:0/2237714228 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ff4075b60 con 0x7f1ff4155a80 2024-04-20T10:21:48.678 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.678+0000 7f1feaa00640 1 -- 172.21.15.121:0/2237714228 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1fdc048020 con 0x7f1ff4155a80 2024-04-20T10:21:48.679 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.679+0000 7f1feaa00640 1 --2- 172.21.15.121:0/2237714228 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f1fd006c8f0 0x7f1fd006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:48.679 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.679+0000 7f1feaa00640 1 -- 172.21.15.121:0/2237714228 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f1fdc0ad280 con 0x7f1ff4155a80 2024-04-20T10:21:48.679 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.680+0000 7f1ff2a00640 1 --2- 172.21.15.121:0/2237714228 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f1fd006c8f0 0x7f1fd006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:48.679 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.680+0000 7f1ff2a00640 1 --2- 172.21.15.121:0/2237714228 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f1fd006c8f0 0x7f1fd006edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4006e80 tx=0x7f1fe4006dd0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:48.683 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.684+0000 7f1feaa00640 1 -- 172.21.15.121:0/2237714228 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1fdc07b2b0 con 0x7f1ff4155a80 2024-04-20T10:21:48.923 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.924+0000 7f1ff3e00640 1 -- 172.21.15.121:0/2237714228 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1ff415a760 con 0x7f1ff4155a80 2024-04-20T10:21:48.924 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.924+0000 7f1feaa00640 1 -- 172.21.15.121:0/2237714228 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7f1fdc07abb0 con 0x7f1ff4155a80 2024-04-20T10:21:48.924 INFO:teuthology.orchestra.run.smithi121.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-20T10:21:48.927 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.927+0000 7f1ff3e00640 1 -- 172.21.15.121:0/2237714228 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f1fd006c8f0 msgr2=0x7f1fd006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:48.927 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.927+0000 7f1ff3e00640 1 --2- 172.21.15.121:0/2237714228 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f1fd006c8f0 0x7f1fd006edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1fe4006e80 tx=0x7f1fe4006dd0 comp rx=0 tx=0).stop 2024-04-20T10:21:48.927 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.927+0000 7f1ff3e00640 1 -- 172.21.15.121:0/2237714228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ff4155a80 msgr2=0x7f1ff40b4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:48.927 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.927+0000 7f1ff3e00640 1 --2- 172.21.15.121:0/2237714228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ff4155a80 0x7f1ff40b4200 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc02c4a0 tx=0x7f1fdc02e070 comp rx=0 tx=0).stop 2024-04-20T10:21:48.927 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.927+0000 7f1ff3e00640 1 -- 172.21.15.121:0/2237714228 shutdown_connections 2024-04-20T10:21:48.927 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.927+0000 7f1ff3e00640 1 --2- 172.21.15.121:0/2237714228 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1ff4155ee0 0x7f1ff40b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:48.927 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.927+0000 7f1ff3e00640 1 --2- 172.21.15.121:0/2237714228 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1ff4155a80 0x7f1ff40b4200 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:48.927 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.927+0000 7f1ff3e00640 1 --2- 172.21.15.121:0/2237714228 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f1fd006c8f0 0x7f1fd006edb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:48.927 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.927+0000 7f1ff3e00640 1 --2- 172.21.15.121:0/2237714228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1fd8004800 0x7f1ff4157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:48.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.927+0000 7f1ff3e00640 1 -- 172.21.15.121:0/2237714228 >> 172.21.15.121:0/2237714228 conn(0x7f1ff40d0040 msgr2=0x7f1ff40b0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:48.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.927+0000 7f1ff3e00640 1 -- 172.21.15.121:0/2237714228 shutdown_connections 2024-04-20T10:21:48.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:48.927+0000 7f1ff3e00640 1 -- 172.21.15.121:0/2237714228 wait complete. 2024-04-20T10:21:48.944 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T10:21:48.944 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T10:21:48.944 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2024-04-20T10:21:48.944 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-20T10:21:48.944 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-20T10:21:48.944 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T10:21:48.944 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T10:21:48.944 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T10:21:48.944 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-20T10:21:48.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:48.945 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2024-04-20T10:21:48.996 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T10:21:48.996 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T10:21:48.996 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T10:21:48.996 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:21:48.996 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2024-04-20T10:21:48.996 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:21:48.996 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:21:48.997 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:21:48.997 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2024-04-20T10:21:48.997 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:21:48.997 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:21:48.997 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2024-04-20T10:21:48.997 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:21:48.997 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:21:48.997 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T10:21:48.997 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:21:48.997 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T10:21:48.997 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 42600sec preferred_lft 42600sec 2024-04-20T10:21:48.997 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ec4:7aff:fe8f:cba3/64 scope link 2024-04-20T10:21:48.997 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:21:48.997 INFO:teuthology.orchestra.run.smithi121.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T10:21:48.998 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 2a:73:a9:6c:a9:c5 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:21:48.998 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T10:21:48.998 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:21:48.998 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::30ec:1cff:fe04:5f23/64 scope link 2024-04-20T10:21:48.998 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:21:48.998 INFO:teuthology.orchestra.run.smithi121.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T10:21:48.998 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 2a:73:a9:6c:a9:c5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:21:48.998 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::2873:a9ff:fe6c:a9c5/64 scope link 2024-04-20T10:21:48.998 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:21:49.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:49.000 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T10:21:49.072 INFO:teuthology.orchestra.run.smithi121.stdout:1 2024-04-20T10:21:49.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:49.076 DEBUG:teuthology.orchestra.run.smithi121:> route 2024-04-20T10:21:49.127 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2024-04-20T10:21:49.128 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T10:21:49.128 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T10:21:49.128 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T10:21:49.128 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T10:21:49.128 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:21:49.128 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-20T10:21:49.128 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T10:21:49.128 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T10:21:49.128 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T10:21:49.128 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-20T10:21:49.235 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:21:49.330 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:21:49.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:49.340 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2024-04-20T10:21:49.389 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T10:21:49.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:49.390 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list-id 2024-04-20T10:21:49.437 INFO:teuthology.orchestra.run.smithi121.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-20T10:21:49.438 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:21:49.438 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-20T10:21:49.438 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T10:21:49.438 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-20T10:21:49.438 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-20T10:21:49.543 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:21:49.609 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:21:49.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:49.618 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-20T10:21:49.694 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T10:21:49.694 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T10:21:49.694 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T10:21:49.695 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:21:49.695 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2024-04-20T10:21:49.695 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:21:49.695 INFO:teuthology.orchestra.run.smithi121.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T10:21:49.695 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 26:45:0a:b4:9e:25 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-20T10:21:49.695 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-20T10:21:49.695 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:21:49.695 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::2445:aff:feb4:9e25/64 scope link 2024-04-20T10:21:49.695 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:21:49.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:49.698 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-20T10:21:49.779 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T10:21:49.779 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T10:21:49.779 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T10:21:49.779 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:21:49.779 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2024-04-20T10:21:49.779 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:21:49.779 INFO:teuthology.orchestra.run.smithi121.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T10:21:49.779 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 26:45:0a:b4:9e:25 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-20T10:21:49.779 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-20T10:21:49.779 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:21:49.780 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::2445:aff:feb4:9e25/64 scope link 2024-04-20T10:21:49.780 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:21:49.782 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-20T10:21:49.783 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:21:49.783 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-20T10:21:49.783 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-20T10:21:49.783 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-20T10:21:49.783 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-20T10:21:49.783 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-20T10:21:49.783 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-20T10:21:49.783 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-20T10:21:49.886 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:21:50.055 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:21:50.065 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:21:50.065 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-20T10:21:50.065 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.1 up 2024-04-20T10:21:50.065 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set dev brx.1 master ceph-brx 2024-04-20T10:21:50.065 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-20T10:21:50.162 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:21:50.262 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:21:50.272 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-04-20T10:21:50.272 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T10:21:50.272 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-20T10:21:50.322 INFO:teuthology.orchestra.run.smithi121.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-20T10:21:50.322 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T10:21:50.322 DEBUG:teuthology.orchestra.run.smithi121:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-20T10:21:50.370 DEBUG:teuthology.orchestra.run.smithi121:> sudo modprobe fuse 2024-04-20T10:21:50.455 DEBUG:teuthology.orchestra.run.smithi121:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T10:21:50.510 INFO:teuthology.orchestra.run.smithi121.stdout:/proc 2024-04-20T10:21:50.510 INFO:teuthology.orchestra.run.smithi121.stdout:/sys 2024-04-20T10:21:50.510 INFO:teuthology.orchestra.run.smithi121.stdout:/dev 2024-04-20T10:21:50.510 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/security 2024-04-20T10:21:50.510 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/shm 2024-04-20T10:21:50.511 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/pts 2024-04-20T10:21:50.511 INFO:teuthology.orchestra.run.smithi121.stdout:/run 2024-04-20T10:21:50.511 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/cgroup 2024-04-20T10:21:50.511 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/pstore 2024-04-20T10:21:50.511 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/bpf 2024-04-20T10:21:50.511 INFO:teuthology.orchestra.run.smithi121.stdout:/ 2024-04-20T10:21:50.511 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/selinux 2024-04-20T10:21:50.511 INFO:teuthology.orchestra.run.smithi121.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T10:21:50.511 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/hugepages 2024-04-20T10:21:50.511 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/mqueue 2024-04-20T10:21:50.511 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/debug 2024-04-20T10:21:50.511 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/tracing 2024-04-20T10:21:50.511 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/fuse/connections 2024-04-20T10:21:50.511 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/config 2024-04-20T10:21:50.511 INFO:teuthology.orchestra.run.smithi121.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T10:21:50.512 INFO:teuthology.orchestra.run.smithi121.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T10:21:50.512 INFO:teuthology.orchestra.run.smithi121.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T10:21:50.512 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T10:21:50.512 INFO:teuthology.orchestra.run.smithi121.stdout:/run/user/1000 2024-04-20T10:21:50.512 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph 2024-04-20T10:21:50.512 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T10:21:50.512 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T10:21:50.512 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T10:21:50.512 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T10:21:50.512 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns 2024-04-20T10:21:50.512 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:21:50.512 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:21:50.512 INFO:teuthology.orchestra.run.smithi121.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-20T10:21:50.512 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T10:21:50.512 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T10:21:50.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:50.513 DEBUG:teuthology.orchestra.run.smithi121:> ls /sys/fs/fuse/connections 2024-04-20T10:21:50.563 INFO:teuthology.orchestra.run.smithi121.stdout:41 2024-04-20T10:21:50.563 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2024-04-20T10:21:50.563 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-04-20T10:21:50.568 DEBUG:teuthology.orchestra.run.smithi121:> sudo modprobe fuse 2024-04-20T10:21:50.639 DEBUG:teuthology.orchestra.run.smithi121:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T10:21:50.682 INFO:teuthology.orchestra.run.smithi121.stdout:/proc 2024-04-20T10:21:50.682 INFO:teuthology.orchestra.run.smithi121.stdout:/sys 2024-04-20T10:21:50.682 INFO:teuthology.orchestra.run.smithi121.stdout:/dev 2024-04-20T10:21:50.683 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/security 2024-04-20T10:21:50.683 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/shm 2024-04-20T10:21:50.683 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/pts 2024-04-20T10:21:50.683 INFO:teuthology.orchestra.run.smithi121.stdout:/run 2024-04-20T10:21:50.683 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/cgroup 2024-04-20T10:21:50.683 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/pstore 2024-04-20T10:21:50.683 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/bpf 2024-04-20T10:21:50.683 INFO:teuthology.orchestra.run.smithi121.stdout:/ 2024-04-20T10:21:50.683 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/selinux 2024-04-20T10:21:50.683 INFO:teuthology.orchestra.run.smithi121.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T10:21:50.683 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/hugepages 2024-04-20T10:21:50.683 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/mqueue 2024-04-20T10:21:50.683 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/debug 2024-04-20T10:21:50.683 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/tracing 2024-04-20T10:21:50.684 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/fuse/connections 2024-04-20T10:21:50.684 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/config 2024-04-20T10:21:50.684 INFO:teuthology.orchestra.run.smithi121.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T10:21:50.684 INFO:teuthology.orchestra.run.smithi121.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T10:21:50.684 INFO:teuthology.orchestra.run.smithi121.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T10:21:50.684 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T10:21:50.684 INFO:teuthology.orchestra.run.smithi121.stdout:/run/user/1000 2024-04-20T10:21:50.684 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph 2024-04-20T10:21:50.684 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T10:21:50.684 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T10:21:50.684 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T10:21:50.684 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T10:21:50.684 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns 2024-04-20T10:21:50.685 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:21:50.685 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:21:50.685 INFO:teuthology.orchestra.run.smithi121.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-20T10:21:50.685 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T10:21:50.685 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T10:21:50.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:50.686 DEBUG:teuthology.orchestra.run.smithi121:> ls /sys/fs/fuse/connections 2024-04-20T10:21:50.735 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi121.stderr:2024-04-20T10:21:50.736+0000 7f3d6bdb4480 -1 init, newargv = 0x55accd23bf50 newargc=15 2024-04-20T10:21:50.736 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi121.stderr:ceph-fuse[33688]: starting ceph client 2024-04-20T10:21:50.746 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi121.stderr:ceph-fuse[33688]: starting fuse 2024-04-20T10:21:50.768 INFO:teuthology.orchestra.run.smithi121.stdout:41 2024-04-20T10:21:50.768 INFO:teuthology.orchestra.run.smithi121.stdout:43 2024-04-20T10:21:50.768 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 43] 2024-04-20T10:21:50.769 DEBUG:teuthology.orchestra.run.smithi121:> sudo stdin-killer -- python3 -c ' 2024-04-20T10:21:50.769 DEBUG:teuthology.orchestra.run.smithi121:> import glob 2024-04-20T10:21:50.769 DEBUG:teuthology.orchestra.run.smithi121:> import re 2024-04-20T10:21:50.769 DEBUG:teuthology.orchestra.run.smithi121:> import os 2024-04-20T10:21:50.769 DEBUG:teuthology.orchestra.run.smithi121:> import subprocess 2024-04-20T10:21:50.769 DEBUG:teuthology.orchestra.run.smithi121:> 2024-04-20T10:21:50.769 DEBUG:teuthology.orchestra.run.smithi121:> def _find_admin_socket(client_name): 2024-04-20T10:21:50.769 DEBUG:teuthology.orchestra.run.smithi121:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-04-20T10:21:50.769 DEBUG:teuthology.orchestra.run.smithi121:> files = glob.glob(asok_path) 2024-04-20T10:21:50.769 DEBUG:teuthology.orchestra.run.smithi121:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-04-20T10:21:50.769 DEBUG:teuthology.orchestra.run.smithi121:> 2024-04-20T10:21:50.769 DEBUG:teuthology.orchestra.run.smithi121:> # Given a non-glob path, it better be there 2024-04-20T10:21:50.769 DEBUG:teuthology.orchestra.run.smithi121:> if "*" not in asok_path: 2024-04-20T10:21:50.769 DEBUG:teuthology.orchestra.run.smithi121:> assert(len(files) == 1) 2024-04-20T10:21:50.770 DEBUG:teuthology.orchestra.run.smithi121:> return files[0] 2024-04-20T10:21:50.770 DEBUG:teuthology.orchestra.run.smithi121:> 2024-04-20T10:21:50.770 DEBUG:teuthology.orchestra.run.smithi121:> for f in files: 2024-04-20T10:21:50.770 DEBUG:teuthology.orchestra.run.smithi121:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T10:21:50.770 DEBUG:teuthology.orchestra.run.smithi121:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T10:21:50.770 DEBUG:teuthology.orchestra.run.smithi121:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T10:21:50.770 DEBUG:teuthology.orchestra.run.smithi121:> contents = proc_f.read() 2024-04-20T10:21:50.770 DEBUG:teuthology.orchestra.run.smithi121:> if mountpoint in contents: 2024-04-20T10:21:50.770 DEBUG:teuthology.orchestra.run.smithi121:> return f 2024-04-20T10:21:50.770 DEBUG:teuthology.orchestra.run.smithi121:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T10:21:50.770 DEBUG:teuthology.orchestra.run.smithi121:> 2024-04-20T10:21:50.770 DEBUG:teuthology.orchestra.run.smithi121:> print(_find_admin_socket("client.2")) 2024-04-20T10:21:50.770 DEBUG:teuthology.orchestra.run.smithi121:> ' 2024-04-20T10:21:50.959 INFO:teuthology.orchestra.run.smithi121.stdout:/var/run/ceph/ceph-client.2.33688.asok 2024-04-20T10:21:50.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:21:50.975 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.33688.asok 2024-04-20T10:21:50.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:50.976 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.33688.asok status 2024-04-20T10:21:51.206 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T10:21:51.206 INFO:teuthology.orchestra.run.smithi121.stdout: "metadata": { 2024-04-20T10:21:51.206 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T10:21:51.206 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T10:21:51.206 INFO:teuthology.orchestra.run.smithi121.stdout: "entity_id": "2", 2024-04-20T10:21:51.206 INFO:teuthology.orchestra.run.smithi121.stdout: "hostname": "smithi121", 2024-04-20T10:21:51.206 INFO:teuthology.orchestra.run.smithi121.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-04-20T10:21:51.207 INFO:teuthology.orchestra.run.smithi121.stdout: "pid": "33688", 2024-04-20T10:21:51.207 INFO:teuthology.orchestra.run.smithi121.stdout: "root": "/" 2024-04-20T10:21:51.207 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T10:21:51.207 INFO:teuthology.orchestra.run.smithi121.stdout: "dentry_count": 0, 2024-04-20T10:21:51.207 INFO:teuthology.orchestra.run.smithi121.stdout: "dentry_pinned_count": 0, 2024-04-20T10:21:51.207 INFO:teuthology.orchestra.run.smithi121.stdout: "id": 4890, 2024-04-20T10:21:51.207 INFO:teuthology.orchestra.run.smithi121.stdout: "inst": { 2024-04-20T10:21:51.207 INFO:teuthology.orchestra.run.smithi121.stdout: "name": { 2024-04-20T10:21:51.207 INFO:teuthology.orchestra.run.smithi121.stdout: "type": "client", 2024-04-20T10:21:51.207 INFO:teuthology.orchestra.run.smithi121.stdout: "num": 4890 2024-04-20T10:21:51.207 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T10:21:51.207 INFO:teuthology.orchestra.run.smithi121.stdout: "addr": { 2024-04-20T10:21:51.207 INFO:teuthology.orchestra.run.smithi121.stdout: "type": "v1", 2024-04-20T10:21:51.207 INFO:teuthology.orchestra.run.smithi121.stdout: "addr": "192.168.0.2:0", 2024-04-20T10:21:51.207 INFO:teuthology.orchestra.run.smithi121.stdout: "nonce": 3075233946 2024-04-20T10:21:51.208 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T10:21:51.208 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T10:21:51.208 INFO:teuthology.orchestra.run.smithi121.stdout: "addr": { 2024-04-20T10:21:51.208 INFO:teuthology.orchestra.run.smithi121.stdout: "type": "v1", 2024-04-20T10:21:51.208 INFO:teuthology.orchestra.run.smithi121.stdout: "addr": "192.168.0.2:0", 2024-04-20T10:21:51.208 INFO:teuthology.orchestra.run.smithi121.stdout: "nonce": 3075233946 2024-04-20T10:21:51.208 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T10:21:51.208 INFO:teuthology.orchestra.run.smithi121.stdout: "inst_str": "client.4890 192.168.0.2:0/3075233946", 2024-04-20T10:21:51.208 INFO:teuthology.orchestra.run.smithi121.stdout: "addr_str": "192.168.0.2:0/3075233946", 2024-04-20T10:21:51.208 INFO:teuthology.orchestra.run.smithi121.stdout: "inode_count": 1, 2024-04-20T10:21:51.208 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_epoch": 8, 2024-04-20T10:21:51.208 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_epoch": 27, 2024-04-20T10:21:51.208 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_epoch_barrier": 0, 2024-04-20T10:21:51.208 INFO:teuthology.orchestra.run.smithi121.stdout: "blocklisted": false, 2024-04-20T10:21:51.209 INFO:teuthology.orchestra.run.smithi121.stdout: "fs_name": "cephfs" 2024-04-20T10:21:51.209 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T10:21:51.221 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T10:21:51.423 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.423+0000 7fbfcea00640 1 Processor -- start 2024-04-20T10:21:51.423 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.424+0000 7fbfcea00640 1 -- start start 2024-04-20T10:21:51.425 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.425+0000 7fbfcea00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fbfc806dcb0 0x7fbfc8071b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:51.425 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.425+0000 7fbfcea00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbfc8072090 0x7fbfc80d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:51.425 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.425+0000 7fbfcea00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbfc80ade40 con 0x7fbfc815b3b0 2024-04-20T10:21:51.425 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.425+0000 7fbfcea00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fbfc806f1b0 con 0x7fbfc806dcb0 2024-04-20T10:21:51.425 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.425+0000 7fbfcea00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fbfc80ae0e0 con 0x7fbfc8072090 2024-04-20T10:21:51.425 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.426+0000 7fbfc7400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fbfc806dcb0 0x7fbfc8071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:51.425 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.426+0000 7fbfc7400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fbfc806dcb0 0x7fbfc8071b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37458/0 (socket says 172.21.15.121:37458) 2024-04-20T10:21:51.426 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.426+0000 7fbfc7400640 1 -- 172.21.15.121:0/881156154 learned_addr learned my addr 172.21.15.121:0/881156154 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:51.426 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.426+0000 7fbfcca00640 1 --2- 172.21.15.121:0/881156154 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbfc8072090 0x7fbfc80d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:51.426 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.426+0000 7fbfc7400640 1 -- 172.21.15.121:0/881156154 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbfc8072090 msgr2=0x7fbfc80d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:51.426 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.426+0000 7fbfc7400640 1 --2- 172.21.15.121:0/881156154 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbfc8072090 0x7fbfc80d0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:51.426 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.426+0000 7fbfc7400640 1 -- 172.21.15.121:0/881156154 >> v1:172.21.15.123:6789/0 conn(0x7fbfc815b3b0 legacy=0x7fbfc806eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:51.426 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.426+0000 7fbfc7400640 1 -- 172.21.15.121:0/881156154 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc815b790 con 0x7fbfc806dcb0 2024-04-20T10:21:51.427 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.427+0000 7fbfc7400640 1 --2- 172.21.15.121:0/881156154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbfc806dcb0 0x7fbfc8071b50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0009870 tx=0x7fbfb002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c9a32f02cd980f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:51.427 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.428+0000 7fbfc6a00640 1 -- 172.21.15.121:0/881156154 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0035020 con 0x7fbfc806dcb0 2024-04-20T10:21:51.427 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.428+0000 7fbfc6a00640 1 -- 172.21.15.121:0/881156154 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfb0031640 con 0x7fbfc806dcb0 2024-04-20T10:21:51.427 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.428+0000 7fbfc6a00640 1 -- 172.21.15.121:0/881156154 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb002d040 con 0x7fbfc806dcb0 2024-04-20T10:21:51.428 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.428+0000 7fbfcea00640 1 -- 172.21.15.121:0/881156154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbfc806dcb0 msgr2=0x7fbfc8071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:51.428 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.428+0000 7fbfcea00640 1 --2- 172.21.15.121:0/881156154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbfc806dcb0 0x7fbfc8071b50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0009870 tx=0x7fbfb002b8f0 comp rx=0 tx=0).stop 2024-04-20T10:21:51.428 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.429+0000 7fbfcea00640 1 -- 172.21.15.121:0/881156154 shutdown_connections 2024-04-20T10:21:51.429 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.429+0000 7fbfcea00640 1 --2- 172.21.15.121:0/881156154 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbfc8072090 0x7fbfc80d0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:51.429 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.429+0000 7fbfcea00640 1 --2- 172.21.15.121:0/881156154 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbfc806dcb0 0x7fbfc8071b50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:51.429 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.429+0000 7fbfcea00640 1 -- 172.21.15.121:0/881156154 >> 172.21.15.121:0/881156154 conn(0x7fbfc80b3350 msgr2=0x7fbfc80b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:51.429 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.429+0000 7fbfcea00640 1 -- 172.21.15.121:0/881156154 shutdown_connections 2024-04-20T10:21:51.429 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.429+0000 7fbfcea00640 1 -- 172.21.15.121:0/881156154 wait complete. 2024-04-20T10:21:51.430 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.431+0000 7fbfcea00640 1 Processor -- start 2024-04-20T10:21:51.430 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.431+0000 7fbfcea00640 1 -- start start 2024-04-20T10:21:51.432 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.432+0000 7fbfcea00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbfc806dcb0 0x7fbfc80b0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:51.432 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.432+0000 7fbfcea00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbfc815b3b0 0x7fbfc809d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:51.432 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.432+0000 7fbfcea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfc809dcc0 0x7fbfc815b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:51.432 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.432+0000 7fbfcea00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc80ae940 con 0x7fbfc815b3b0 2024-04-20T10:21:51.432 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.432+0000 7fbfcea00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc80ae660 con 0x7fbfc809dcc0 2024-04-20T10:21:51.432 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.432+0000 7fbfcea00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fbfc80aec20 con 0x7fbfc806dcb0 2024-04-20T10:21:51.432 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.432+0000 7fbfc7400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbfc815b3b0 0x7fbfc809d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:51.432 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.432+0000 7fbfc7400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbfc815b3b0 0x7fbfc809d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37460/0 (socket says 172.21.15.121:37460) 2024-04-20T10:21:51.433 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.432+0000 7fbfc7400640 1 -- 172.21.15.121:0/2439746887 learned_addr learned my addr 172.21.15.121:0/2439746887 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:51.433 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.432+0000 7fbfc7e00640 1 --2- 172.21.15.121:0/2439746887 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbfc806dcb0 0x7fbfc80b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:51.433 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.432+0000 7fbfcca00640 1 --2- 172.21.15.121:0/2439746887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfc809dcc0 0x7fbfc815b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:51.433 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.432+0000 7fbfc7400640 1 -- 172.21.15.121:0/2439746887 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbfc806dcb0 msgr2=0x7fbfc80b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:51.433 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.432+0000 7fbfc7400640 1 --2- 172.21.15.121:0/2439746887 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbfc806dcb0 0x7fbfc80b0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:51.433 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.432+0000 7fbfc7400640 1 -- 172.21.15.121:0/2439746887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfc809dcc0 msgr2=0x7fbfc815b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:51.433 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.432+0000 7fbfc7400640 1 --2- 172.21.15.121:0/2439746887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfc809dcc0 0x7fbfc815b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:51.433 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.432+0000 7fbfc7400640 1 -- 172.21.15.121:0/2439746887 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc809e170 con 0x7fbfc815b3b0 2024-04-20T10:21:51.433 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.432+0000 7fbfc7e00640 1 --2- 172.21.15.121:0/2439746887 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbfc806dcb0 0x7fbfc80b0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:51.433 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.432+0000 7fbfcca00640 1 --2- 172.21.15.121:0/2439746887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfc809dcc0 0x7fbfc815b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:51.433 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.433+0000 7fbfc7400640 1 --2- 172.21.15.121:0/2439746887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbfc815b3b0 0x7fbfc809d780 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0000c00 tx=0x7fbfb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:51.434 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.433+0000 7fbfbfe00640 1 -- 172.21.15.121:0/2439746887 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0040070 con 0x7fbfc815b3b0 2024-04-20T10:21:51.435 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.433+0000 7fbfbfe00640 1 -- 172.21.15.121:0/2439746887 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfb00399f0 con 0x7fbfc815b3b0 2024-04-20T10:21:51.435 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.433+0000 7fbfcea00640 1 -- 172.21.15.121:0/2439746887 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfc8151f40 con 0x7fbfc815b3b0 2024-04-20T10:21:51.435 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.433+0000 7fbfbfe00640 1 -- 172.21.15.121:0/2439746887 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb0035040 con 0x7fbfc815b3b0 2024-04-20T10:21:51.435 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.433+0000 7fbfcea00640 1 -- 172.21.15.121:0/2439746887 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfc815c6e0 con 0x7fbfc815b3b0 2024-04-20T10:21:51.435 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.434+0000 7fbfcea00640 1 -- 172.21.15.121:0/2439746887 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfc80742b0 con 0x7fbfc815b3b0 2024-04-20T10:21:51.435 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.435+0000 7fbfbfe00640 1 -- 172.21.15.121:0/2439746887 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbfb004b060 con 0x7fbfc815b3b0 2024-04-20T10:21:51.436 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.437+0000 7fbfbfe00640 1 --2- 172.21.15.121:0/2439746887 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fbfa406c8f0 0x7fbfa406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:51.436 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.437+0000 7fbfbfe00640 1 -- 172.21.15.121:0/2439746887 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fbfb00ad9b0 con 0x7fbfc815b3b0 2024-04-20T10:21:51.436 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.437+0000 7fbfc7e00640 1 --2- 172.21.15.121:0/2439746887 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fbfa406c8f0 0x7fbfa406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:51.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.438+0000 7fbfc7e00640 1 --2- 172.21.15.121:0/2439746887 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fbfa406c8f0 0x7fbfa406edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8007e90 tx=0x7fbfb80073a0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:51.441 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.441+0000 7fbfbfe00640 1 -- 172.21.15.121:0/2439746887 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbfc80742b0 con 0x7fbfc815b3b0 2024-04-20T10:21:51.682 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.682+0000 7fbfcea00640 1 -- 172.21.15.121:0/2439746887 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbfc80d0a30 con 0x7fbfc815b3b0 2024-04-20T10:21:51.682 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.683+0000 7fbfbfe00640 1 -- 172.21.15.121:0/2439746887 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7fbfb007b940 con 0x7fbfc815b3b0 2024-04-20T10:21:51.683 INFO:teuthology.orchestra.run.smithi121.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-20T10:21:51.686 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.686+0000 7fbfcea00640 1 -- 172.21.15.121:0/2439746887 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fbfa406c8f0 msgr2=0x7fbfa406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:51.686 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.686+0000 7fbfcea00640 1 --2- 172.21.15.121:0/2439746887 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fbfa406c8f0 0x7fbfa406edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8007e90 tx=0x7fbfb80073a0 comp rx=0 tx=0).stop 2024-04-20T10:21:51.686 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.686+0000 7fbfcea00640 1 -- 172.21.15.121:0/2439746887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbfc815b3b0 msgr2=0x7fbfc809d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:51.686 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.686+0000 7fbfcea00640 1 --2- 172.21.15.121:0/2439746887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbfc815b3b0 0x7fbfc809d780 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbfb0000c00 tx=0x7fbfb00046c0 comp rx=0 tx=0).stop 2024-04-20T10:21:51.686 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.686+0000 7fbfcea00640 1 -- 172.21.15.121:0/2439746887 shutdown_connections 2024-04-20T10:21:51.686 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.686+0000 7fbfcea00640 1 --2- 172.21.15.121:0/2439746887 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fbfa406c8f0 0x7fbfa406edb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:51.686 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.686+0000 7fbfcea00640 1 --2- 172.21.15.121:0/2439746887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfc809dcc0 0x7fbfc815b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:51.686 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.686+0000 7fbfcea00640 1 --2- 172.21.15.121:0/2439746887 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbfc815b3b0 0x7fbfc809d780 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:51.686 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.686+0000 7fbfcea00640 1 --2- 172.21.15.121:0/2439746887 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbfc806dcb0 0x7fbfc80b0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:51.686 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.686+0000 7fbfcea00640 1 -- 172.21.15.121:0/2439746887 >> 172.21.15.121:0/2439746887 conn(0x7fbfc80b3350 msgr2=0x7fbfc8073580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:51.687 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.686+0000 7fbfcea00640 1 -- 172.21.15.121:0/2439746887 shutdown_connections 2024-04-20T10:21:51.687 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:51.686+0000 7fbfcea00640 1 -- 172.21.15.121:0/2439746887 wait complete. 2024-04-20T10:21:51.703 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T10:21:51.703 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T10:21:51.703 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi123.front.sepia.ceph.com 2024-04-20T10:21:51.703 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-20T10:21:51.703 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-20T10:21:51.703 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T10:21:51.703 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T10:21:51.703 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T10:21:51.704 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-20T10:21:51.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:51.704 DEBUG:teuthology.orchestra.run.smithi123:> ip addr 2024-04-20T10:21:51.763 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T10:21:51.763 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T10:21:51.763 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T10:21:51.763 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:21:51.763 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2024-04-20T10:21:51.764 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:21:51.764 INFO:teuthology.orchestra.run.smithi123.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:21:51.764 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6c brd ff:ff:ff:ff:ff:ff 2024-04-20T10:21:51.764 INFO:teuthology.orchestra.run.smithi123.stdout: altname enp7s0f0 2024-04-20T10:21:51.764 INFO:teuthology.orchestra.run.smithi123.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:21:51.764 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6d brd ff:ff:ff:ff:ff:ff 2024-04-20T10:21:51.764 INFO:teuthology.orchestra.run.smithi123.stdout: altname enp7s0f1 2024-04-20T10:21:51.764 INFO:teuthology.orchestra.run.smithi123.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:21:51.764 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f0 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:21:51.764 INFO:teuthology.orchestra.run.smithi123.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T10:21:51.764 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f1 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:21:51.764 INFO:teuthology.orchestra.run.smithi123.stdout: inet 172.21.15.123/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T10:21:51.764 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft 42626sec preferred_lft 42626sec 2024-04-20T10:21:51.765 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::ec4:7aff:fe8f:d3f1/64 scope link 2024-04-20T10:21:51.765 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:21:51.765 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-20T10:21:51.766 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:21:51.766 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-20T10:21:51.766 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link add name ceph-brx type bridge 2024-04-20T10:21:51.766 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip addr flush dev ceph-brx 2024-04-20T10:21:51.766 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set ceph-brx up 2024-04-20T10:21:51.766 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-20T10:21:51.766 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-20T10:21:51.875 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:21:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:21:52.226 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:21:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:21:52.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:52.236 DEBUG:teuthology.orchestra.run.smithi123:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T10:21:52.299 INFO:teuthology.orchestra.run.smithi123.stdout:1 2024-04-20T10:21:52.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:52.303 DEBUG:teuthology.orchestra.run.smithi123:> route 2024-04-20T10:21:52.354 INFO:teuthology.orchestra.run.smithi123.stdout:Kernel IP routing table 2024-04-20T10:21:52.354 INFO:teuthology.orchestra.run.smithi123.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T10:21:52.355 INFO:teuthology.orchestra.run.smithi123.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T10:21:52.355 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T10:21:52.355 INFO:teuthology.orchestra.run.smithi123.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T10:21:52.355 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:21:52.355 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-20T10:21:52.355 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T10:21:52.355 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T10:21:52.355 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T10:21:52.355 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-20T10:21:52.452 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:21:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:21:53.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:21:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:21:53.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:53.118 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2024-04-20T10:21:53.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:53.164 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list-id 2024-04-20T10:21:53.205 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:21:53.205 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-20T10:21:53.205 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T10:21:53.205 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-20T10:21:53.205 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-20T10:21:53.341 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:21:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:21:53.404 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:21:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:21:53.412 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-20T10:21:53.413 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:21:53.413 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-20T10:21:53.413 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-20T10:21:53.413 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-20T10:21:53.413 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-20T10:21:53.413 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-20T10:21:53.413 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-20T10:21:53.413 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-20T10:21:53.496 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:21:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:21:53.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:21:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:21:53.693 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:21:53.693 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-20T10:21:53.693 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set brx.0 up 2024-04-20T10:21:53.693 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set dev brx.0 master ceph-brx 2024-04-20T10:21:53.694 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-20T10:21:53.778 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:21:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:21:53.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:21:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:21:53.882 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-20T10:21:53.882 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T10:21:53.882 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-20T10:21:53.927 INFO:teuthology.orchestra.run.smithi123.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-20T10:21:53.928 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T10:21:53.928 DEBUG:teuthology.orchestra.run.smithi123:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-20T10:21:54.019 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2024-04-20T10:21:54.091 DEBUG:teuthology.orchestra.run.smithi123:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T10:21:54.138 INFO:teuthology.orchestra.run.smithi123.stdout:/proc 2024-04-20T10:21:54.138 INFO:teuthology.orchestra.run.smithi123.stdout:/sys 2024-04-20T10:21:54.138 INFO:teuthology.orchestra.run.smithi123.stdout:/dev 2024-04-20T10:21:54.138 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/security 2024-04-20T10:21:54.138 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/shm 2024-04-20T10:21:54.138 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/pts 2024-04-20T10:21:54.138 INFO:teuthology.orchestra.run.smithi123.stdout:/run 2024-04-20T10:21:54.139 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/cgroup 2024-04-20T10:21:54.139 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/pstore 2024-04-20T10:21:54.139 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/bpf 2024-04-20T10:21:54.139 INFO:teuthology.orchestra.run.smithi123.stdout:/ 2024-04-20T10:21:54.139 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/selinux 2024-04-20T10:21:54.139 INFO:teuthology.orchestra.run.smithi123.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T10:21:54.139 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/hugepages 2024-04-20T10:21:54.139 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/mqueue 2024-04-20T10:21:54.139 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/debug 2024-04-20T10:21:54.139 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/tracing 2024-04-20T10:21:54.139 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/fuse/connections 2024-04-20T10:21:54.139 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/config 2024-04-20T10:21:54.139 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T10:21:54.139 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T10:21:54.139 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T10:21:54.140 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T10:21:54.140 INFO:teuthology.orchestra.run.smithi123.stdout:/run/user/1000 2024-04-20T10:21:54.140 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph 2024-04-20T10:21:54.140 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T10:21:54.140 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T10:21:54.140 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T10:21:54.140 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T10:21:54.140 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns 2024-04-20T10:21:54.140 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T10:21:54.140 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T10:21:54.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:54.141 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2024-04-20T10:21:54.186 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T10:21:54.186 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-20T10:21:54.233 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2024-04-20T10:21:54.301 DEBUG:teuthology.orchestra.run.smithi123:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T10:21:54.341 INFO:teuthology.orchestra.run.smithi123.stdout:/proc 2024-04-20T10:21:54.417 INFO:teuthology.orchestra.run.smithi123.stdout:/sys 2024-04-20T10:21:54.417 INFO:teuthology.orchestra.run.smithi123.stdout:/dev 2024-04-20T10:21:54.417 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/security 2024-04-20T10:21:54.417 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/shm 2024-04-20T10:21:54.417 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/pts 2024-04-20T10:21:54.417 INFO:teuthology.orchestra.run.smithi123.stdout:/run 2024-04-20T10:21:54.417 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/cgroup 2024-04-20T10:21:54.417 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/pstore 2024-04-20T10:21:54.417 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/bpf 2024-04-20T10:21:54.417 INFO:teuthology.orchestra.run.smithi123.stdout:/ 2024-04-20T10:21:54.417 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/selinux 2024-04-20T10:21:54.418 INFO:teuthology.orchestra.run.smithi123.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T10:21:54.418 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/hugepages 2024-04-20T10:21:54.418 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/mqueue 2024-04-20T10:21:54.418 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/debug 2024-04-20T10:21:54.418 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/tracing 2024-04-20T10:21:54.418 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/fuse/connections 2024-04-20T10:21:54.418 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/config 2024-04-20T10:21:54.418 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T10:21:54.418 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T10:21:54.418 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T10:21:54.418 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T10:21:54.418 INFO:teuthology.orchestra.run.smithi123.stdout:/run/user/1000 2024-04-20T10:21:54.418 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph 2024-04-20T10:21:54.419 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T10:21:54.419 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T10:21:54.419 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T10:21:54.419 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T10:21:54.419 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns 2024-04-20T10:21:54.419 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T10:21:54.419 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T10:21:54.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:54.420 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2024-04-20T10:21:54.443 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi123.stderr:2024-04-20T10:21:54.443+0000 7f6d40d98480 -1 init, newargv = 0x562a3276cf00 newargc=15 2024-04-20T10:21:54.443 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi123.stderr:ceph-fuse[28122]: starting ceph client 2024-04-20T10:21:54.455 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi123.stderr:ceph-fuse[28122]: starting fuse 2024-04-20T10:21:54.483 INFO:teuthology.orchestra.run.smithi123.stdout:40 2024-04-20T10:21:54.484 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2024-04-20T10:21:54.484 DEBUG:teuthology.orchestra.run.smithi123:> sudo stdin-killer -- python3 -c ' 2024-04-20T10:21:54.484 DEBUG:teuthology.orchestra.run.smithi123:> import glob 2024-04-20T10:21:54.484 DEBUG:teuthology.orchestra.run.smithi123:> import re 2024-04-20T10:21:54.484 DEBUG:teuthology.orchestra.run.smithi123:> import os 2024-04-20T10:21:54.484 DEBUG:teuthology.orchestra.run.smithi123:> import subprocess 2024-04-20T10:21:54.484 DEBUG:teuthology.orchestra.run.smithi123:> 2024-04-20T10:21:54.484 DEBUG:teuthology.orchestra.run.smithi123:> def _find_admin_socket(client_name): 2024-04-20T10:21:54.484 DEBUG:teuthology.orchestra.run.smithi123:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-20T10:21:54.485 DEBUG:teuthology.orchestra.run.smithi123:> files = glob.glob(asok_path) 2024-04-20T10:21:54.485 DEBUG:teuthology.orchestra.run.smithi123:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-20T10:21:54.485 DEBUG:teuthology.orchestra.run.smithi123:> 2024-04-20T10:21:54.485 DEBUG:teuthology.orchestra.run.smithi123:> # Given a non-glob path, it better be there 2024-04-20T10:21:54.485 DEBUG:teuthology.orchestra.run.smithi123:> if "*" not in asok_path: 2024-04-20T10:21:54.485 DEBUG:teuthology.orchestra.run.smithi123:> assert(len(files) == 1) 2024-04-20T10:21:54.485 DEBUG:teuthology.orchestra.run.smithi123:> return files[0] 2024-04-20T10:21:54.485 DEBUG:teuthology.orchestra.run.smithi123:> 2024-04-20T10:21:54.485 DEBUG:teuthology.orchestra.run.smithi123:> for f in files: 2024-04-20T10:21:54.485 DEBUG:teuthology.orchestra.run.smithi123:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T10:21:54.485 DEBUG:teuthology.orchestra.run.smithi123:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T10:21:54.485 DEBUG:teuthology.orchestra.run.smithi123:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T10:21:54.485 DEBUG:teuthology.orchestra.run.smithi123:> contents = proc_f.read() 2024-04-20T10:21:54.485 DEBUG:teuthology.orchestra.run.smithi123:> if mountpoint in contents: 2024-04-20T10:21:54.485 DEBUG:teuthology.orchestra.run.smithi123:> return f 2024-04-20T10:21:54.485 DEBUG:teuthology.orchestra.run.smithi123:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T10:21:54.486 DEBUG:teuthology.orchestra.run.smithi123:> 2024-04-20T10:21:54.486 DEBUG:teuthology.orchestra.run.smithi123:> print(_find_admin_socket("client.1")) 2024-04-20T10:21:54.486 DEBUG:teuthology.orchestra.run.smithi123:> ' 2024-04-20T10:21:54.684 INFO:teuthology.orchestra.run.smithi123.stdout:/var/run/ceph/ceph-client.1.28122.asok 2024-04-20T10:21:54.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:21:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:21:54.701 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.28122.asok 2024-04-20T10:21:54.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:54.701 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.28122.asok status 2024-04-20T10:21:54.932 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T10:21:54.933 INFO:teuthology.orchestra.run.smithi123.stdout: "metadata": { 2024-04-20T10:21:54.933 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T10:21:54.933 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T10:21:54.933 INFO:teuthology.orchestra.run.smithi123.stdout: "entity_id": "1", 2024-04-20T10:21:54.933 INFO:teuthology.orchestra.run.smithi123.stdout: "hostname": "smithi123", 2024-04-20T10:21:54.933 INFO:teuthology.orchestra.run.smithi123.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-20T10:21:54.933 INFO:teuthology.orchestra.run.smithi123.stdout: "pid": "28122", 2024-04-20T10:21:54.933 INFO:teuthology.orchestra.run.smithi123.stdout: "root": "/" 2024-04-20T10:21:54.933 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T10:21:54.933 INFO:teuthology.orchestra.run.smithi123.stdout: "dentry_count": 0, 2024-04-20T10:21:54.933 INFO:teuthology.orchestra.run.smithi123.stdout: "dentry_pinned_count": 0, 2024-04-20T10:21:54.934 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 4514, 2024-04-20T10:21:54.934 INFO:teuthology.orchestra.run.smithi123.stdout: "inst": { 2024-04-20T10:21:54.934 INFO:teuthology.orchestra.run.smithi123.stdout: "name": { 2024-04-20T10:21:54.934 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "client", 2024-04-20T10:21:54.934 INFO:teuthology.orchestra.run.smithi123.stdout: "num": 4514 2024-04-20T10:21:54.934 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T10:21:54.934 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": { 2024-04-20T10:21:54.934 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-20T10:21:54.934 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "192.168.0.1:0", 2024-04-20T10:21:54.934 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3545496530 2024-04-20T10:21:54.934 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T10:21:54.934 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T10:21:54.934 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": { 2024-04-20T10:21:54.934 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-20T10:21:54.935 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "192.168.0.1:0", 2024-04-20T10:21:54.935 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 3545496530 2024-04-20T10:21:54.935 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T10:21:54.935 INFO:teuthology.orchestra.run.smithi123.stdout: "inst_str": "client.4514 192.168.0.1:0/3545496530", 2024-04-20T10:21:54.935 INFO:teuthology.orchestra.run.smithi123.stdout: "addr_str": "192.168.0.1:0/3545496530", 2024-04-20T10:21:54.935 INFO:teuthology.orchestra.run.smithi123.stdout: "inode_count": 1, 2024-04-20T10:21:54.935 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_epoch": 8, 2024-04-20T10:21:54.935 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_epoch": 27, 2024-04-20T10:21:54.935 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_epoch_barrier": 0, 2024-04-20T10:21:54.935 INFO:teuthology.orchestra.run.smithi123.stdout: "blocklisted": false, 2024-04-20T10:21:54.935 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs" 2024-04-20T10:21:54.935 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T10:21:54.948 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T10:21:55.155 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.155+0000 7f10d3000640 1 Processor -- start 2024-04-20T10:21:55.155 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.156+0000 7f10d3000640 1 -- start start 2024-04-20T10:21:55.157 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.157+0000 7f10d3000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f10cc15e5d0 0x7f10cc0ce160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:55.157 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.158+0000 7f10d3000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10cc0b5b10 con 0x7f10cc06c010 2024-04-20T10:21:55.157 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.158+0000 7f10d3000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10cc0b48b0 con 0x7f10cc15e9b0 2024-04-20T10:21:55.158 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.158+0000 7f10d3000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f10cc0b5670 con 0x7f10cc15e5d0 2024-04-20T10:21:55.158 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.158+0000 7f10cbe00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f10cc15e5d0 0x7f10cc0ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:55.158 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.158+0000 7f10d1000640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f10cc06c010 0x7f10cc0ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:38310/0 (socket says 172.21.15.121:38310) 2024-04-20T10:21:55.158 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.158+0000 7f10d1000640 1 -- 172.21.15.121:0/2009455241 learned_addr learned my addr 172.21.15.121:0/2009455241 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:55.158 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.158+0000 7f10cbe00640 1 -- 172.21.15.121:0/2009455241 >> v1:172.21.15.123:6789/0 conn(0x7f10cc15e9b0 legacy=0x7f10cc06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:55.158 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.158+0000 7f10cbe00640 1 -- 172.21.15.121:0/2009455241 >> v1:172.21.15.121:6789/0 conn(0x7f10cc06c010 legacy=0x7f10cc0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:55.158 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.158+0000 7f10cb400640 1 -- 172.21.15.121:0/2009455241 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1143391363 0 0) 0x7f10c4001110 con 0x7f10cc06c010 2024-04-20T10:21:55.158 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.158+0000 7f10cbe00640 1 -- 172.21.15.121:0/2009455241 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10cc0b4b90 con 0x7f10cc15e5d0 2024-04-20T10:21:55.159 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.160+0000 7f10cbe00640 1 --2- 172.21.15.121:0/2009455241 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10cc15e5d0 0x7f10cc0ce160 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f10bc009d90 tx=0x7f10bc02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa59625c1b43148 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:55.159 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.160+0000 7f10cb400640 1 -- 172.21.15.121:0/2009455241 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc039070 con 0x7f10cc15e5d0 2024-04-20T10:21:55.159 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.160+0000 7f10cb400640 1 -- 172.21.15.121:0/2009455241 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10bc02cbd0 con 0x7f10cc15e5d0 2024-04-20T10:21:55.159 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.160+0000 7f10cb400640 1 -- 172.21.15.121:0/2009455241 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc034440 con 0x7f10cc15e5d0 2024-04-20T10:21:55.160 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.161+0000 7f10d3000640 1 -- 172.21.15.121:0/2009455241 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10cc15e5d0 msgr2=0x7f10cc0ce160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:55.160 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.161+0000 7f10d3000640 1 --2- 172.21.15.121:0/2009455241 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10cc15e5d0 0x7f10cc0ce160 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f10bc009d90 tx=0x7f10bc02be20 comp rx=0 tx=0).stop 2024-04-20T10:21:55.160 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.161+0000 7f10d3000640 1 -- 172.21.15.121:0/2009455241 shutdown_connections 2024-04-20T10:21:55.160 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.161+0000 7f10d3000640 1 --2- 172.21.15.121:0/2009455241 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10cc15e5d0 0x7f10cc0ce160 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:55.161 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.161+0000 7f10d3000640 1 -- 172.21.15.121:0/2009455241 >> 172.21.15.121:0/2009455241 conn(0x7f10cc0cd4b0 msgr2=0x7f10cc0cd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:55.161 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.161+0000 7f10d3000640 1 -- 172.21.15.121:0/2009455241 shutdown_connections 2024-04-20T10:21:55.161 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.161+0000 7f10d3000640 1 -- 172.21.15.121:0/2009455241 wait complete. 2024-04-20T10:21:55.163 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.163+0000 7f10d3000640 1 Processor -- start 2024-04-20T10:21:55.163 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.163+0000 7f10d3000640 1 -- start start 2024-04-20T10:21:55.164 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.165+0000 7f10d3000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10cc06c010 0x7f10cc0b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:55.164 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.165+0000 7f10d3000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10cc15e5d0 0x7f10cc159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:55.165 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.165+0000 7f10d3000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10cc15e9b0 0x7f10cc1ea180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:55.165 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.165+0000 7f10d3000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc0b5150 con 0x7f10cc15e5d0 2024-04-20T10:21:55.165 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.165+0000 7f10d3000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc0cd1c0 con 0x7f10cc06c010 2024-04-20T10:21:55.165 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.165+0000 7f10d3000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f10cc0ccc50 con 0x7f10cc15e9b0 2024-04-20T10:21:55.165 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.165+0000 7f10c3400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10cc15e5d0 0x7f10cc159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:55.165 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.165+0000 7f10c3400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10cc15e5d0 0x7f10cc159ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:37480/0 (socket says 172.21.15.121:37480) 2024-04-20T10:21:55.165 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.165+0000 7f10c3400640 1 -- 172.21.15.121:0/2693741850 learned_addr learned my addr 172.21.15.121:0/2693741850 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:55.166 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.165+0000 7f10d1000640 1 --2- 172.21.15.121:0/2693741850 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10cc15e9b0 0x7f10cc1ea180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:55.166 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.165+0000 7f10c3400640 1 -- 172.21.15.121:0/2693741850 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10cc15e9b0 msgr2=0x7f10cc1ea180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:55.166 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.166+0000 7f10c3400640 1 --2- 172.21.15.121:0/2693741850 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10cc15e9b0 0x7f10cc1ea180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:55.166 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.166+0000 7f10c3400640 1 -- 172.21.15.121:0/2693741850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10cc06c010 msgr2=0x7f10cc0b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:55.167 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.166+0000 7f10cbe00640 1 --2- 172.21.15.121:0/2693741850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10cc06c010 0x7f10cc0b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:55.167 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.166+0000 7f10c3400640 1 --2- 172.21.15.121:0/2693741850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10cc06c010 0x7f10cc0b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:55.167 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.166+0000 7f10c3400640 1 -- 172.21.15.121:0/2693741850 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10cc161370 con 0x7f10cc15e5d0 2024-04-20T10:21:55.167 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.166+0000 7f10cbe00640 1 --2- 172.21.15.121:0/2693741850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10cc06c010 0x7f10cc0b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:55.167 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.166+0000 7f10c3400640 1 --2- 172.21.15.121:0/2693741850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10cc15e5d0 0x7f10cc159ea0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f10b400b3c0 tx=0x7f10b400b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:55.167 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.166+0000 7f10c8c00640 1 -- 172.21.15.121:0/2693741850 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b4011830 con 0x7f10cc15e5d0 2024-04-20T10:21:55.167 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.166+0000 7f10c8c00640 1 -- 172.21.15.121:0/2693741850 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10b4016070 con 0x7f10cc15e5d0 2024-04-20T10:21:55.167 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.166+0000 7f10c8c00640 1 -- 172.21.15.121:0/2693741850 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b4011c50 con 0x7f10cc15e5d0 2024-04-20T10:21:55.167 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.166+0000 7f10d3000640 1 -- 172.21.15.121:0/2693741850 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10cc161650 con 0x7f10cc15e5d0 2024-04-20T10:21:55.167 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.166+0000 7f10d3000640 1 -- 172.21.15.121:0/2693741850 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10cc161c10 con 0x7f10cc15e5d0 2024-04-20T10:21:55.168 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.168+0000 7f10d3000640 1 -- 172.21.15.121:0/2693741850 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10cc069090 con 0x7f10cc15e5d0 2024-04-20T10:21:55.168 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.169+0000 7f10c8c00640 1 -- 172.21.15.121:0/2693741850 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10b401b460 con 0x7f10cc15e5d0 2024-04-20T10:21:55.169 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.170+0000 7f10c8c00640 1 --2- 172.21.15.121:0/2693741850 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f10a806c8f0 0x7f10a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:55.169 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.170+0000 7f10c8c00640 1 -- 172.21.15.121:0/2693741850 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f10b408ba70 con 0x7f10cc15e5d0 2024-04-20T10:21:55.169 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.170+0000 7f10cbe00640 1 --2- 172.21.15.121:0/2693741850 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f10a806c8f0 0x7f10a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:55.170 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.171+0000 7f10cbe00640 1 --2- 172.21.15.121:0/2693741850 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f10a806c8f0 0x7f10a806edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f10bc036000 tx=0x7f10bc030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:55.174 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.175+0000 7f10c8c00640 1 -- 172.21.15.121:0/2693741850 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f10b4020020 con 0x7f10cc15e5d0 2024-04-20T10:21:55.420 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.420+0000 7f10d3000640 1 -- 172.21.15.121:0/2693741850 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f10cc0cfc80 con 0x7f10cc15e5d0 2024-04-20T10:21:55.421 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.421+0000 7f10c8c00640 1 -- 172.21.15.121:0/2693741850 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7f10b4059aa0 con 0x7f10cc15e5d0 2024-04-20T10:21:55.421 INFO:teuthology.orchestra.run.smithi121.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-20T10:21:55.424 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.424+0000 7f10d3000640 1 -- 172.21.15.121:0/2693741850 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f10a806c8f0 msgr2=0x7f10a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:55.424 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.424+0000 7f10d3000640 1 --2- 172.21.15.121:0/2693741850 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f10a806c8f0 0x7f10a806edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f10bc036000 tx=0x7f10bc030000 comp rx=0 tx=0).stop 2024-04-20T10:21:55.424 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.424+0000 7f10d3000640 1 -- 172.21.15.121:0/2693741850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10cc15e5d0 msgr2=0x7f10cc159ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:55.424 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.424+0000 7f10d3000640 1 --2- 172.21.15.121:0/2693741850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10cc15e5d0 0x7f10cc159ea0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f10b400b3c0 tx=0x7f10b400b880 comp rx=0 tx=0).stop 2024-04-20T10:21:55.424 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.424+0000 7f10d3000640 1 -- 172.21.15.121:0/2693741850 shutdown_connections 2024-04-20T10:21:55.424 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.424+0000 7f10d3000640 1 --2- 172.21.15.121:0/2693741850 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10cc15e9b0 0x7f10cc1ea180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:55.425 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.424+0000 7f10d3000640 1 --2- 172.21.15.121:0/2693741850 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10cc15e5d0 0x7f10cc159ea0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:55.425 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.424+0000 7f10d3000640 1 --2- 172.21.15.121:0/2693741850 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f10a806c8f0 0x7f10a806edb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:55.425 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.424+0000 7f10d3000640 1 --2- 172.21.15.121:0/2693741850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10cc06c010 0x7f10cc0b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:55.425 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.424+0000 7f10d3000640 1 -- 172.21.15.121:0/2693741850 >> 172.21.15.121:0/2693741850 conn(0x7f10cc0cd4b0 msgr2=0x7f10cc06f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:55.425 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.424+0000 7f10d3000640 1 -- 172.21.15.121:0/2693741850 shutdown_connections 2024-04-20T10:21:55.425 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:55.424+0000 7f10d3000640 1 -- 172.21.15.121:0/2693741850 wait complete. 2024-04-20T10:21:55.429 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:21:55.437 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:21:55.442 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T10:21:55.442 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T10:21:55.442 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi123.front.sepia.ceph.com 2024-04-20T10:21:55.442 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-20T10:21:55.442 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-20T10:21:55.442 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T10:21:55.442 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T10:21:55.442 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T10:21:55.442 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-20T10:21:55.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:55.443 DEBUG:teuthology.orchestra.run.smithi123:> ip addr 2024-04-20T10:21:55.534 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T10:21:55.534 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T10:21:55.534 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T10:21:55.534 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:21:55.535 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2024-04-20T10:21:55.535 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:21:55.535 INFO:teuthology.orchestra.run.smithi123.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:21:55.535 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6c brd ff:ff:ff:ff:ff:ff 2024-04-20T10:21:55.535 INFO:teuthology.orchestra.run.smithi123.stdout: altname enp7s0f0 2024-04-20T10:21:55.535 INFO:teuthology.orchestra.run.smithi123.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:21:55.535 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6d brd ff:ff:ff:ff:ff:ff 2024-04-20T10:21:55.535 INFO:teuthology.orchestra.run.smithi123.stdout: altname enp7s0f1 2024-04-20T10:21:55.535 INFO:teuthology.orchestra.run.smithi123.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:21:55.535 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f0 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:21:55.535 INFO:teuthology.orchestra.run.smithi123.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T10:21:55.535 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f1 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:21:55.535 INFO:teuthology.orchestra.run.smithi123.stdout: inet 172.21.15.123/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T10:21:55.535 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft 42622sec preferred_lft 42622sec 2024-04-20T10:21:55.536 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::ec4:7aff:fe8f:d3f1/64 scope link 2024-04-20T10:21:55.536 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:21:55.536 INFO:teuthology.orchestra.run.smithi123.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T10:21:55.536 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 7a:50:1d:20:61:1e brd ff:ff:ff:ff:ff:ff 2024-04-20T10:21:55.536 INFO:teuthology.orchestra.run.smithi123.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T10:21:55.536 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:21:55.536 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::f8f4:21ff:fe26:8986/64 scope link 2024-04-20T10:21:55.536 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:21:55.536 INFO:teuthology.orchestra.run.smithi123.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T10:21:55.536 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 7a:50:1d:20:61:1e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T10:21:55.536 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::7850:1dff:fe20:611e/64 scope link tentative 2024-04-20T10:21:55.536 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:21:55.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:55.537 DEBUG:teuthology.orchestra.run.smithi123:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T10:21:55.692 INFO:teuthology.orchestra.run.smithi123.stdout:1 2024-04-20T10:21:55.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:55.696 DEBUG:teuthology.orchestra.run.smithi123:> route 2024-04-20T10:21:55.751 INFO:teuthology.orchestra.run.smithi123.stdout:Kernel IP routing table 2024-04-20T10:21:55.751 INFO:teuthology.orchestra.run.smithi123.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T10:21:55.751 INFO:teuthology.orchestra.run.smithi123.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T10:21:55.751 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T10:21:55.751 INFO:teuthology.orchestra.run.smithi123.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T10:21:55.752 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:21:55.752 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-20T10:21:55.752 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T10:21:55.753 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T10:21:55.753 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T10:21:55.753 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-20T10:21:55.859 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:21:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:21:55.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:21:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:21:55.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:55.963 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2024-04-20T10:21:56.008 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-20T10:21:56.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:56.008 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list-id 2024-04-20T10:21:56.103 INFO:teuthology.orchestra.run.smithi123.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-20T10:21:56.103 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:21:56.104 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-20T10:21:56.104 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T10:21:56.104 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-20T10:21:56.104 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-20T10:21:56.198 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:21:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:21:56.265 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:21:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:21:56.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:56.275 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-20T10:21:56.349 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T10:21:56.349 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T10:21:56.349 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T10:21:56.349 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:21:56.349 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2024-04-20T10:21:56.349 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:21:56.349 INFO:teuthology.orchestra.run.smithi123.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T10:21:56.349 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 5e:ae:ae:6e:02:0d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-20T10:21:56.349 INFO:teuthology.orchestra.run.smithi123.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-20T10:21:56.350 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:21:56.350 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::5cae:aeff:fe6e:20d/64 scope link 2024-04-20T10:21:56.350 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:21:56.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:56.353 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-20T10:21:56.432 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T10:21:56.432 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T10:21:56.432 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T10:21:56.432 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:21:56.432 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2024-04-20T10:21:56.433 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:21:56.433 INFO:teuthology.orchestra.run.smithi123.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T10:21:56.433 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 5e:ae:ae:6e:02:0d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-20T10:21:56.433 INFO:teuthology.orchestra.run.smithi123.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-20T10:21:56.433 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:21:56.433 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::5cae:aeff:fe6e:20d/64 scope link 2024-04-20T10:21:56.433 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:21:56.437 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-20T10:21:56.437 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:21:56.437 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-20T10:21:56.437 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-20T10:21:56.437 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-20T10:21:56.437 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-20T10:21:56.437 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-20T10:21:56.437 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-20T10:21:56.437 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-20T10:21:56.538 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:21:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:21:56.696 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:21:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:21:56.705 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:21:56.705 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2024-04-20T10:21:56.705 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set brx.1 up 2024-04-20T10:21:56.705 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set dev brx.1 master ceph-brx 2024-04-20T10:21:56.705 DEBUG:teuthology.orchestra.run.smithi123:> ') 2024-04-20T10:21:56.802 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:21:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:21:56.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:21:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:21:56.909 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-04-20T10:21:56.909 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T10:21:56.909 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-20T10:21:56.953 INFO:teuthology.orchestra.run.smithi123.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-20T10:21:56.953 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T10:21:56.953 DEBUG:teuthology.orchestra.run.smithi123:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-20T10:21:57.047 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2024-04-20T10:21:57.122 DEBUG:teuthology.orchestra.run.smithi123:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T10:21:57.176 INFO:teuthology.orchestra.run.smithi123.stdout:/proc 2024-04-20T10:21:57.176 INFO:teuthology.orchestra.run.smithi123.stdout:/sys 2024-04-20T10:21:57.176 INFO:teuthology.orchestra.run.smithi123.stdout:/dev 2024-04-20T10:21:57.176 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/security 2024-04-20T10:21:57.176 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/shm 2024-04-20T10:21:57.176 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/pts 2024-04-20T10:21:57.176 INFO:teuthology.orchestra.run.smithi123.stdout:/run 2024-04-20T10:21:57.176 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/cgroup 2024-04-20T10:21:57.177 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/pstore 2024-04-20T10:21:57.177 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/bpf 2024-04-20T10:21:57.177 INFO:teuthology.orchestra.run.smithi123.stdout:/ 2024-04-20T10:21:57.177 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/selinux 2024-04-20T10:21:57.177 INFO:teuthology.orchestra.run.smithi123.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T10:21:57.177 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/hugepages 2024-04-20T10:21:57.177 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/mqueue 2024-04-20T10:21:57.177 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/debug 2024-04-20T10:21:57.177 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/tracing 2024-04-20T10:21:57.177 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/fuse/connections 2024-04-20T10:21:57.177 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/config 2024-04-20T10:21:57.178 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T10:21:57.178 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T10:21:57.178 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T10:21:57.178 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T10:21:57.178 INFO:teuthology.orchestra.run.smithi123.stdout:/run/user/1000 2024-04-20T10:21:57.178 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph 2024-04-20T10:21:57.178 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T10:21:57.178 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T10:21:57.178 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T10:21:57.179 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T10:21:57.179 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns 2024-04-20T10:21:57.179 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T10:21:57.179 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T10:21:57.179 INFO:teuthology.orchestra.run.smithi123.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-20T10:21:57.179 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T10:21:57.180 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T10:21:57.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:57.181 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2024-04-20T10:21:57.232 INFO:teuthology.orchestra.run.smithi123.stdout:40 2024-04-20T10:21:57.232 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [40] 2024-04-20T10:21:57.233 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-04-20T10:21:57.236 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2024-04-20T10:21:57.305 DEBUG:teuthology.orchestra.run.smithi123:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T10:21:57.349 INFO:teuthology.orchestra.run.smithi123.stdout:/proc 2024-04-20T10:21:57.349 INFO:teuthology.orchestra.run.smithi123.stdout:/sys 2024-04-20T10:21:57.349 INFO:teuthology.orchestra.run.smithi123.stdout:/dev 2024-04-20T10:21:57.349 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/security 2024-04-20T10:21:57.349 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/shm 2024-04-20T10:21:57.349 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/pts 2024-04-20T10:21:57.349 INFO:teuthology.orchestra.run.smithi123.stdout:/run 2024-04-20T10:21:57.349 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/cgroup 2024-04-20T10:21:57.349 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/pstore 2024-04-20T10:21:57.350 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/bpf 2024-04-20T10:21:57.350 INFO:teuthology.orchestra.run.smithi123.stdout:/ 2024-04-20T10:21:57.350 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/selinux 2024-04-20T10:21:57.350 INFO:teuthology.orchestra.run.smithi123.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T10:21:57.350 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/hugepages 2024-04-20T10:21:57.350 INFO:teuthology.orchestra.run.smithi123.stdout:/dev/mqueue 2024-04-20T10:21:57.350 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/debug 2024-04-20T10:21:57.350 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/tracing 2024-04-20T10:21:57.350 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/fs/fuse/connections 2024-04-20T10:21:57.350 INFO:teuthology.orchestra.run.smithi123.stdout:/sys/kernel/config 2024-04-20T10:21:57.350 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T10:21:57.350 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T10:21:57.351 INFO:teuthology.orchestra.run.smithi123.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T10:21:57.351 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T10:21:57.351 INFO:teuthology.orchestra.run.smithi123.stdout:/run/user/1000 2024-04-20T10:21:57.351 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph 2024-04-20T10:21:57.351 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T10:21:57.351 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T10:21:57.351 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T10:21:57.351 INFO:teuthology.orchestra.run.smithi123.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T10:21:57.351 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns 2024-04-20T10:21:57.351 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T10:21:57.351 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T10:21:57.352 INFO:teuthology.orchestra.run.smithi123.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-20T10:21:57.352 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T10:21:57.352 INFO:teuthology.orchestra.run.smithi123.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T10:21:57.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:57.353 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2024-04-20T10:21:57.405 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi123.stderr:2024-04-20T10:21:57.405+0000 7fd96b073480 -1 init, newargv = 0x55eef3631840 newargc=15 2024-04-20T10:21:57.405 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi123.stderr:ceph-fuse[28596]: starting ceph client 2024-04-20T10:21:57.414 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi123.stderr:ceph-fuse[28596]: starting fuse 2024-04-20T10:21:57.436 INFO:teuthology.orchestra.run.smithi123.stdout:40 2024-04-20T10:21:57.436 INFO:teuthology.orchestra.run.smithi123.stdout:43 2024-04-20T10:21:57.436 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40, 43] 2024-04-20T10:21:57.437 DEBUG:teuthology.orchestra.run.smithi123:> sudo stdin-killer -- python3 -c ' 2024-04-20T10:21:57.437 DEBUG:teuthology.orchestra.run.smithi123:> import glob 2024-04-20T10:21:57.437 DEBUG:teuthology.orchestra.run.smithi123:> import re 2024-04-20T10:21:57.437 DEBUG:teuthology.orchestra.run.smithi123:> import os 2024-04-20T10:21:57.437 DEBUG:teuthology.orchestra.run.smithi123:> import subprocess 2024-04-20T10:21:57.437 DEBUG:teuthology.orchestra.run.smithi123:> 2024-04-20T10:21:57.437 DEBUG:teuthology.orchestra.run.smithi123:> def _find_admin_socket(client_name): 2024-04-20T10:21:57.437 DEBUG:teuthology.orchestra.run.smithi123:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-04-20T10:21:57.437 DEBUG:teuthology.orchestra.run.smithi123:> files = glob.glob(asok_path) 2024-04-20T10:21:57.437 DEBUG:teuthology.orchestra.run.smithi123:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-04-20T10:21:57.437 DEBUG:teuthology.orchestra.run.smithi123:> 2024-04-20T10:21:57.437 DEBUG:teuthology.orchestra.run.smithi123:> # Given a non-glob path, it better be there 2024-04-20T10:21:57.437 DEBUG:teuthology.orchestra.run.smithi123:> if "*" not in asok_path: 2024-04-20T10:21:57.437 DEBUG:teuthology.orchestra.run.smithi123:> assert(len(files) == 1) 2024-04-20T10:21:57.437 DEBUG:teuthology.orchestra.run.smithi123:> return files[0] 2024-04-20T10:21:57.438 DEBUG:teuthology.orchestra.run.smithi123:> 2024-04-20T10:21:57.438 DEBUG:teuthology.orchestra.run.smithi123:> for f in files: 2024-04-20T10:21:57.438 DEBUG:teuthology.orchestra.run.smithi123:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T10:21:57.438 DEBUG:teuthology.orchestra.run.smithi123:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T10:21:57.438 DEBUG:teuthology.orchestra.run.smithi123:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T10:21:57.438 DEBUG:teuthology.orchestra.run.smithi123:> contents = proc_f.read() 2024-04-20T10:21:57.438 DEBUG:teuthology.orchestra.run.smithi123:> if mountpoint in contents: 2024-04-20T10:21:57.438 DEBUG:teuthology.orchestra.run.smithi123:> return f 2024-04-20T10:21:57.438 DEBUG:teuthology.orchestra.run.smithi123:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T10:21:57.438 DEBUG:teuthology.orchestra.run.smithi123:> 2024-04-20T10:21:57.438 DEBUG:teuthology.orchestra.run.smithi123:> print(_find_admin_socket("client.3")) 2024-04-20T10:21:57.438 DEBUG:teuthology.orchestra.run.smithi123:> ' 2024-04-20T10:21:57.631 INFO:teuthology.orchestra.run.smithi123.stdout:/var/run/ceph/ceph-client.3.28596.asok 2024-04-20T10:21:57.636 INFO:teuthology.orchestra.run.smithi123.stderr:2024-04-20T10:21:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:21:57.648 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.28596.asok 2024-04-20T10:21:57.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:57.648 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.28596.asok status 2024-04-20T10:21:57.877 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-04-20T10:21:57.878 INFO:teuthology.orchestra.run.smithi123.stdout: "metadata": { 2024-04-20T10:21:57.878 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T10:21:57.878 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T10:21:57.878 INFO:teuthology.orchestra.run.smithi123.stdout: "entity_id": "3", 2024-04-20T10:21:57.878 INFO:teuthology.orchestra.run.smithi123.stdout: "hostname": "smithi123", 2024-04-20T10:21:57.878 INFO:teuthology.orchestra.run.smithi123.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-04-20T10:21:57.878 INFO:teuthology.orchestra.run.smithi123.stdout: "pid": "28596", 2024-04-20T10:21:57.878 INFO:teuthology.orchestra.run.smithi123.stdout: "root": "/" 2024-04-20T10:21:57.878 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T10:21:57.878 INFO:teuthology.orchestra.run.smithi123.stdout: "dentry_count": 0, 2024-04-20T10:21:57.878 INFO:teuthology.orchestra.run.smithi123.stdout: "dentry_pinned_count": 0, 2024-04-20T10:21:57.878 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 4681, 2024-04-20T10:21:57.878 INFO:teuthology.orchestra.run.smithi123.stdout: "inst": { 2024-04-20T10:21:57.879 INFO:teuthology.orchestra.run.smithi123.stdout: "name": { 2024-04-20T10:21:57.879 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "client", 2024-04-20T10:21:57.879 INFO:teuthology.orchestra.run.smithi123.stdout: "num": 4681 2024-04-20T10:21:57.879 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T10:21:57.879 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": { 2024-04-20T10:21:57.879 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-20T10:21:57.879 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "192.168.0.2:0", 2024-04-20T10:21:57.879 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1679723541 2024-04-20T10:21:57.879 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-04-20T10:21:57.879 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T10:21:57.879 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": { 2024-04-20T10:21:57.880 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2024-04-20T10:21:57.880 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "192.168.0.2:0", 2024-04-20T10:21:57.880 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1679723541 2024-04-20T10:21:57.880 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-04-20T10:21:57.880 INFO:teuthology.orchestra.run.smithi123.stdout: "inst_str": "client.4681 192.168.0.2:0/1679723541", 2024-04-20T10:21:57.880 INFO:teuthology.orchestra.run.smithi123.stdout: "addr_str": "192.168.0.2:0/1679723541", 2024-04-20T10:21:57.880 INFO:teuthology.orchestra.run.smithi123.stdout: "inode_count": 1, 2024-04-20T10:21:57.880 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_epoch": 8, 2024-04-20T10:21:57.880 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_epoch": 27, 2024-04-20T10:21:57.880 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_epoch_barrier": 0, 2024-04-20T10:21:57.880 INFO:teuthology.orchestra.run.smithi123.stdout: "blocklisted": false, 2024-04-20T10:21:57.880 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs" 2024-04-20T10:21:57.880 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-04-20T10:21:57.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:57.892 DEBUG:teuthology.orchestra.run.smithi121:> stat --file-system '--printf=%T 2024-04-20T10:21:57.892 DEBUG:teuthology.orchestra.run.smithi121:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-20T10:21:57.951 INFO:teuthology.orchestra.run.smithi121.stdout:fuseblk 2024-04-20T10:21:57.951 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-20T10:21:57.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:57.952 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T10:21:58.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:58.037 DEBUG:teuthology.orchestra.run.smithi121:> stat --file-system '--printf=%T 2024-04-20T10:21:58.037 DEBUG:teuthology.orchestra.run.smithi121:> ' -- /home/ubuntu/cephtest/mnt.2 2024-04-20T10:21:58.092 INFO:teuthology.orchestra.run.smithi121.stdout:fuseblk 2024-04-20T10:21:58.093 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-04-20T10:21:58.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:58.093 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-20T10:21:58.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:58.170 DEBUG:teuthology.orchestra.run.smithi123:> stat --file-system '--printf=%T 2024-04-20T10:21:58.170 DEBUG:teuthology.orchestra.run.smithi123:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-20T10:21:58.229 INFO:teuthology.orchestra.run.smithi123.stdout:fuseblk 2024-04-20T10:21:58.230 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-20T10:21:58.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:58.230 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-20T10:21:58.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:58.315 DEBUG:teuthology.orchestra.run.smithi123:> stat --file-system '--printf=%T 2024-04-20T10:21:58.315 DEBUG:teuthology.orchestra.run.smithi123:> ' -- /home/ubuntu/cephtest/mnt.3 2024-04-20T10:21:58.367 INFO:teuthology.orchestra.run.smithi123.stdout:fuseblk 2024-04-20T10:21:58.367 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-04-20T10:21:58.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:21:58.367 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-20T10:21:58.451 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-20T10:21:58.467 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:21:58.679 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.678+0000 7f0510200640 1 Processor -- start 2024-04-20T10:21:58.679 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.679+0000 7f0510200640 1 -- start start 2024-04-20T10:21:58.680 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.680+0000 7f0510200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f05080b11c0 0x7f05080d1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:58.680 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.680+0000 7f0510200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05080d23b0 con 0x7f05080d4930 2024-04-20T10:21:58.680 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.680+0000 7f0510200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05080d2690 con 0x7f050815e9c0 2024-04-20T10:21:58.681 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.680+0000 7f0510200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f05080d0f20 con 0x7f05080b11c0 2024-04-20T10:21:58.683 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.680+0000 7f050e200640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f05080d4930 0x7f05080d3170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:40442/0 (socket says 172.21.15.121:40442) 2024-04-20T10:21:58.683 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.680+0000 7f050e200640 1 -- 172.21.15.121:0/1868870092 learned_addr learned my addr 172.21.15.121:0/1868870092 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:58.683 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.681+0000 7f050ce00640 1 --2- 172.21.15.121:0/1868870092 >> v2:172.21.15.123:3300/0 conn(0x7f05080b11c0 0x7f05080d1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:58.683 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.681+0000 7f0507e00640 1 -- 172.21.15.121:0/1868870092 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2864778661 0 0) 0x7f05080d23b0 con 0x7f05080d4930 2024-04-20T10:21:58.683 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.681+0000 7f0507e00640 1 -- 172.21.15.121:0/1868870092 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f04f0003520 con 0x7f05080d4930 2024-04-20T10:21:58.683 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.681+0000 7f0507e00640 1 -- 172.21.15.121:0/1868870092 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3042045433 0 0) 0x7f04f0003520 con 0x7f05080d4930 2024-04-20T10:21:58.683 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.681+0000 7f0507e00640 1 -- 172.21.15.121:0/1868870092 >> v2:172.21.15.123:3300/0 conn(0x7f05080b11c0 msgr2=0x7f05080d1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:58.683 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.681+0000 7f0507e00640 1 --2- 172.21.15.121:0/1868870092 >> v2:172.21.15.123:3300/0 conn(0x7f05080b11c0 0x7f05080d1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:58.683 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.681+0000 7f0507e00640 1 -- 172.21.15.121:0/1868870092 >> v1:172.21.15.123:6789/0 conn(0x7f050815e9c0 legacy=0x7f05080d0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:58.683 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.681+0000 7f0507e00640 1 -- 172.21.15.121:0/1868870092 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05080d2970 con 0x7f05080d4930 2024-04-20T10:21:58.683 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.681+0000 7f0507e00640 1 -- 172.21.15.121:0/1868870092 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f05000019f0 con 0x7f05080d4930 2024-04-20T10:21:58.684 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.681+0000 7f0507e00640 1 -- 172.21.15.121:0/1868870092 >> v1:172.21.15.121:6789/0 conn(0x7f05080d4930 legacy=0x7f05080d3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:58.684 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.681+0000 7f0507e00640 1 --2- 172.21.15.121:0/1868870092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04f0003a00 0x7f04f0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:58.684 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.681+0000 7f0507e00640 1 --2- 172.21.15.121:0/1868870092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04f0004800 0x7f04f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:58.684 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.682+0000 7f0507e00640 1 --2- 172.21.15.121:0/1868870092 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f04f0005210 0x7f04f000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:58.684 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.682+0000 7f0507e00640 1 -- 172.21.15.121:0/1868870092 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f05080d0f20 con 0x7f04f0004800 2024-04-20T10:21:58.684 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.682+0000 7f0507e00640 1 -- 172.21.15.121:0/1868870092 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f04f000c160 con 0x7f04f0003a00 2024-04-20T10:21:58.684 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.682+0000 7f0507e00640 1 -- 172.21.15.121:0/1868870092 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f04f000c360 con 0x7f04f0005210 2024-04-20T10:21:58.684 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.682+0000 7f050ce00640 1 --2- 172.21.15.121:0/1868870092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04f0004800 0x7f04f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:58.684 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.682+0000 7f050d800640 1 --2- 172.21.15.121:0/1868870092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04f0003a00 0x7f04f0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:58.684 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.682+0000 7f050ce00640 1 -- 172.21.15.121:0/1868870092 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f04f0005210 msgr2=0x7f04f000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:58.684 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.682+0000 7f050ce00640 1 --2- 172.21.15.121:0/1868870092 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f04f0005210 0x7f04f000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:58.684 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.682+0000 7f050ce00640 1 -- 172.21.15.121:0/1868870092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04f0003a00 msgr2=0x7f04f0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:58.684 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.682+0000 7f050ce00640 1 --2- 172.21.15.121:0/1868870092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04f0003a00 0x7f04f0003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:58.684 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.682+0000 7f050ce00640 1 -- 172.21.15.121:0/1868870092 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f0003520 con 0x7f04f0004800 2024-04-20T10:21:58.685 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.682+0000 7f050e200640 1 -- 172.21.15.121:0/1868870092 reap_dead start 2024-04-20T10:21:58.686 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.683+0000 7f050ce00640 1 --2- 172.21.15.121:0/1868870092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04f0004800 0x7f04f0004c80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f05080b15e0 tx=0x7f04f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3e4701ae56a051d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:58.686 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.684+0000 7f0507e00640 1 -- 172.21.15.121:0/1868870092 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f4039070 con 0x7f04f0004800 2024-04-20T10:21:58.686 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.684+0000 7f0507e00640 1 -- 172.21.15.121:0/1868870092 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04f402c660 con 0x7f04f0004800 2024-04-20T10:21:58.686 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.684+0000 7f0507e00640 1 -- 172.21.15.121:0/1868870092 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f4009690 con 0x7f04f0004800 2024-04-20T10:21:58.686 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.684+0000 7f0510200640 1 -- 172.21.15.121:0/1868870092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04f0004800 msgr2=0x7f04f0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:58.686 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.684+0000 7f0510200640 1 --2- 172.21.15.121:0/1868870092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04f0004800 0x7f04f0004c80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f05080b15e0 tx=0x7f04f0005830 comp rx=0 tx=0).stop 2024-04-20T10:21:58.686 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.684+0000 7f0510200640 1 -- 172.21.15.121:0/1868870092 shutdown_connections 2024-04-20T10:21:58.686 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.684+0000 7f0510200640 1 --2- 172.21.15.121:0/1868870092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f04f0004800 0x7f04f0004c80 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:58.686 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.684+0000 7f0510200640 1 -- 172.21.15.121:0/1868870092 >> 172.21.15.121:0/1868870092 conn(0x7f05080b9600 msgr2=0x7f05080b9a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:58.686 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.684+0000 7f0510200640 1 -- 172.21.15.121:0/1868870092 shutdown_connections 2024-04-20T10:21:58.686 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.684+0000 7f0510200640 1 -- 172.21.15.121:0/1868870092 wait complete. 2024-04-20T10:21:58.687 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.686+0000 7f0510200640 1 Processor -- start 2024-04-20T10:21:58.687 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.686+0000 7f0510200640 1 -- start start 2024-04-20T10:21:58.689 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.687+0000 7f0510200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04f0004800 0x7f05080ce110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:58.689 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.687+0000 7f0510200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05081588f0 0x7f05081f8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:58.689 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.687+0000 7f0510200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0508158d70 0x7f05080b7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:58.689 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.687+0000 7f0510200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f05080d1f30 con 0x7f05081588f0 2024-04-20T10:21:58.689 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.687+0000 7f0510200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f05080d1db0 con 0x7f04f0004800 2024-04-20T10:21:58.689 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.687+0000 7f0510200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f05080d2ce0 con 0x7f0508158d70 2024-04-20T10:21:58.689 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.688+0000 7f050ce00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05081588f0 0x7f05081f8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:58.689 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.688+0000 7f050ce00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05081588f0 0x7f05081f8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42974/0 (socket says 172.21.15.121:42974) 2024-04-20T10:21:58.689 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.688+0000 7f050ce00640 1 -- 172.21.15.121:0/918794177 learned_addr learned my addr 172.21.15.121:0/918794177 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:58.689 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.688+0000 7f050d800640 1 --2- 172.21.15.121:0/918794177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04f0004800 0x7f05080ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:58.689 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.688+0000 7f050e200640 1 --2- 172.21.15.121:0/918794177 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0508158d70 0x7f05080b7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:58.689 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.688+0000 7f050ce00640 1 -- 172.21.15.121:0/918794177 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0508158d70 msgr2=0x7f05080b7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:58.690 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.688+0000 7f050ce00640 1 --2- 172.21.15.121:0/918794177 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0508158d70 0x7f05080b7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:58.690 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.688+0000 7f050ce00640 1 -- 172.21.15.121:0/918794177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04f0004800 msgr2=0x7f05080ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:58.690 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.688+0000 7f050ce00640 1 --2- 172.21.15.121:0/918794177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04f0004800 0x7f05080ce110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:58.690 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.688+0000 7f050ce00640 1 -- 172.21.15.121:0/918794177 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05081f8af0 con 0x7f05081588f0 2024-04-20T10:21:58.690 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.688+0000 7f050d800640 1 --2- 172.21.15.121:0/918794177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04f0004800 0x7f05080ce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:58.690 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.688+0000 7f050ce00640 1 --2- 172.21.15.121:0/918794177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05081588f0 0x7f05081f8030 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f04f4000c00 tx=0x7f04f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:58.691 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.689+0000 7f0505600640 1 -- 172.21.15.121:0/918794177 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f4039050 con 0x7f05081588f0 2024-04-20T10:21:58.691 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.689+0000 7f0505600640 1 -- 172.21.15.121:0/918794177 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04f4045070 con 0x7f05081588f0 2024-04-20T10:21:58.691 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.689+0000 7f0505600640 1 -- 172.21.15.121:0/918794177 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f40044e0 con 0x7f05081588f0 2024-04-20T10:21:58.691 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.689+0000 7f0510200640 1 -- 172.21.15.121:0/918794177 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05081f8dd0 con 0x7f05081588f0 2024-04-20T10:21:58.691 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.689+0000 7f0510200640 1 -- 172.21.15.121:0/918794177 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05080cd910 con 0x7f05081588f0 2024-04-20T10:21:58.691 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.690+0000 7f0510200640 1 -- 172.21.15.121:0/918794177 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0508068980 con 0x7f05081588f0 2024-04-20T10:21:58.692 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.691+0000 7f0505600640 1 -- 172.21.15.121:0/918794177 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f04f4003a10 con 0x7f05081588f0 2024-04-20T10:21:58.692 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.692+0000 7f0505600640 1 --2- 172.21.15.121:0/918794177 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f04e806c8f0 0x7f04e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:58.692 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.692+0000 7f0505600640 1 -- 172.21.15.121:0/918794177 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f04f40ada60 con 0x7f05081588f0 2024-04-20T10:21:58.693 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.692+0000 7f050d800640 1 --2- 172.21.15.121:0/918794177 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f04e806c8f0 0x7f04e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:58.694 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.693+0000 7f050d800640 1 --2- 172.21.15.121:0/918794177 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f04e806c8f0 0x7f04e806edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f050806a810 tx=0x7f04fc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:58.697 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.697+0000 7f0505600640 1 -- 172.21.15.121:0/918794177 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f04f404e020 con 0x7f05081588f0 2024-04-20T10:21:58.898 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.898+0000 7f0510200640 1 -- 172.21.15.121:0/918794177 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f050806efb0 con 0x7f05081588f0 2024-04-20T10:21:58.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.899+0000 7f0505600640 1 -- 172.21.15.121:0/918794177 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18940 (secure 0 0 0) 0x7f04f407ba80 con 0x7f05081588f0 2024-04-20T10:21:58.900 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:21:58.900 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":27,"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","created":"2024-04-20T10:20:38.076076+0000","modified":"2024-04-20T10:21:44.715898+0000","last_up_change":"2024-04-20T10:20:50.162700+0000","last_in_change":"2024-04-20T10:20:41.510382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:20:51.141365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:21:00.436937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:21:01.026957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:21:04.862693+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"629a7b8c-a40a-4a2a-a3cc-d9f357774310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6809","nonce":1044588835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6811","nonce":1044588835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6815","nonce":1044588835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6813","nonce":1044588835}]},"public_addr":"172.21.15.121:6809/1044588835","cluster_addr":"172.21.15.121:6811/1044588835","heartbeat_back_addr":"172.21.15.121:6815/1044588835","heartbeat_front_addr":"172.21.15.121:6813/1044588835","state":["exists","up"]},{"osd":1,"uuid":"f2570399-aa50-4ff3-bf5d-3d09207290db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6801","nonce":3505199624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6803","nonce":3505199624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6807","nonce":3505199624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6805","nonce":3505199624}]},"public_addr":"172.21.15.121:6801/3505199624","cluster_addr":"172.21.15.121:6803/3505199624","heartbeat_back_addr":"172.21.15.121:6807/3505199624","heartbeat_front_addr":"172.21.15.121:6805/3505199624","state":["exists","up"]},{"osd":2,"uuid":"50893807-dc30-49f4-b41f-739366626658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6817","nonce":3656724207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6819","nonce":3656724207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6827","nonce":3656724207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6822","nonce":3656724207}]},"public_addr":"172.21.15.121:6817/3656724207","cluster_addr":"172.21.15.121:6819/3656724207","heartbeat_back_addr":"172.21.15.121:6827/3656724207","heartbeat_front_addr":"172.21.15.121:6822/3656724207","state":["exists","up"]},{"osd":3,"uuid":"dadb01bc-5491-4ffc-8824-d48a057240b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6823","nonce":2232342832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6826","nonce":2232342832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6831","nonce":2232342832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6829","nonce":2232342832}]},"public_addr":"172.21.15.121:6823/2232342832","cluster_addr":"172.21.15.121:6826/2232342832","heartbeat_back_addr":"172.21.15.121:6831/2232342832","heartbeat_front_addr":"172.21.15.121:6829/2232342832","state":["exists","up"]},{"osd":4,"uuid":"b3f3b4a1-b375-4526-b69e-344ab24c2da1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6801","nonce":99353111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6803","nonce":99353111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6807","nonce":99353111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6805","nonce":99353111}]},"public_addr":"172.21.15.123:6801/99353111","cluster_addr":"172.21.15.123:6803/99353111","heartbeat_back_addr":"172.21.15.123:6807/99353111","heartbeat_front_addr":"172.21.15.123:6805/99353111","state":["exists","up"]},{"osd":5,"uuid":"2ee24a08-1624-4961-9642-d7b510282484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6818","nonce":2504374688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6822","nonce":2504374688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6830","nonce":2504374688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6826","nonce":2504374688}]},"public_addr":"172.21.15.123:6818/2504374688","cluster_addr":"172.21.15.123:6822/2504374688","heartbeat_back_addr":"172.21.15.123:6830/2504374688","heartbeat_front_addr":"172.21.15.123:6826/2504374688","state":["exists","up"]},{"osd":6,"uuid":"186118cc-6128-49de-acca-55260945c1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6819","nonce":4139299599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6823","nonce":4139299599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6829","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6831","nonce":4139299599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6827","nonce":4139299599}]},"public_addr":"172.21.15.123:6819/4139299599","cluster_addr":"172.21.15.123:6823/4139299599","heartbeat_back_addr":"172.21.15.123:6831/4139299599","heartbeat_front_addr":"172.21.15.123:6827/4139299599","state":["exists","up"]},{"osd":7,"uuid":"a00aac6e-4bd9-415a-a93d-ae85210865ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6809","nonce":4043981069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6811","nonce":4043981069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6815","nonce":4043981069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6813","nonce":4043981069}]},"public_addr":"172.21.15.123:6809/4043981069","cluster_addr":"172.21.15.123:6811/4043981069","heartbeat_back_addr":"172.21.15.123:6815/4043981069","heartbeat_front_addr":"172.21.15.123:6813/4043981069","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.904540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.958585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.616367+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.885244+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.103127+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.167248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.884348+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.862356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.23","mappings":[{"from":3,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:21:58.903 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.902+0000 7f0510200640 1 -- 172.21.15.121:0/918794177 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f04e806c8f0 msgr2=0x7f04e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:58.903 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.902+0000 7f0510200640 1 --2- 172.21.15.121:0/918794177 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f04e806c8f0 0x7f04e806edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f050806a810 tx=0x7f04fc009040 comp rx=0 tx=0).stop 2024-04-20T10:21:58.903 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.902+0000 7f0510200640 1 -- 172.21.15.121:0/918794177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05081588f0 msgr2=0x7f05081f8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:58.903 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.902+0000 7f0510200640 1 --2- 172.21.15.121:0/918794177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05081588f0 0x7f05081f8030 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f04f4000c00 tx=0x7f04f4004730 comp rx=0 tx=0).stop 2024-04-20T10:21:58.903 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.902+0000 7f0510200640 1 -- 172.21.15.121:0/918794177 shutdown_connections 2024-04-20T10:21:58.903 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.902+0000 7f0510200640 1 --2- 172.21.15.121:0/918794177 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0508158d70 0x7f05080b7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:58.903 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.902+0000 7f0510200640 1 --2- 172.21.15.121:0/918794177 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05081588f0 0x7f05081f8030 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:58.903 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.902+0000 7f0510200640 1 --2- 172.21.15.121:0/918794177 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f04e806c8f0 0x7f04e806edb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:58.903 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.902+0000 7f0510200640 1 --2- 172.21.15.121:0/918794177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f04f0004800 0x7f05080ce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:58.903 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.902+0000 7f0510200640 1 -- 172.21.15.121:0/918794177 >> 172.21.15.121:0/918794177 conn(0x7f05080b9600 msgr2=0x7f05080b15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:58.903 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.902+0000 7f0510200640 1 -- 172.21.15.121:0/918794177 shutdown_connections 2024-04-20T10:21:58.903 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:58.902+0000 7f0510200640 1 -- 172.21.15.121:0/918794177 wait complete. 2024-04-20T10:21:58.923 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:20:51.141365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:21:00.436937+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:21:01.026957+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T10:21:04.862693+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T10:21:58.924 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:21:59.130 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.130+0000 7fba1b200640 1 Processor -- start 2024-04-20T10:21:59.131 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.130+0000 7fba1b200640 1 -- start start 2024-04-20T10:21:59.131 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.131+0000 7fba1b200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fba1c162210 0x7fba1c162670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:59.132 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.131+0000 7fba1b200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fba1c0b6400 0x7fba1c0b6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:59.132 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.131+0000 7fba1b200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba1c0b3ba0 con 0x7fba1c15e770 2024-04-20T10:21:59.132 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.131+0000 7fba1b200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fba1c15eb50 con 0x7fba1c162210 2024-04-20T10:21:59.132 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.131+0000 7fba1b200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fba1c0d3590 con 0x7fba1c0b6400 2024-04-20T10:21:59.133 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.132+0000 7fba11400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fba1c162210 0x7fba1c162670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:59.133 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.132+0000 7fba19e00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fba1c15e770 0x7fba1c0b3490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:40458/0 (socket says 172.21.15.121:40458) 2024-04-20T10:21:59.133 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.132+0000 7fba19e00640 1 -- 172.21.15.121:0/3494713981 learned_addr learned my addr 172.21.15.121:0/3494713981 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:59.133 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.132+0000 7fba1a800640 1 --2- 172.21.15.121:0/3494713981 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fba1c0b6400 0x7fba1c0b6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:59.133 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.132+0000 7fba11400640 1 -- 172.21.15.121:0/3494713981 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fba1c0b6400 msgr2=0x7fba1c0b6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:59.133 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.132+0000 7fba11400640 1 --2- 172.21.15.121:0/3494713981 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fba1c0b6400 0x7fba1c0b6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:59.133 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.132+0000 7fba11400640 1 -- 172.21.15.121:0/3494713981 >> v1:172.21.15.121:6789/0 conn(0x7fba1c15e770 legacy=0x7fba1c0b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:59.133 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.132+0000 7fba11400640 1 -- 172.21.15.121:0/3494713981 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba1c0d3870 con 0x7fba1c162210 2024-04-20T10:21:59.133 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.132+0000 7fba19400640 1 -- 172.21.15.121:0/3494713981 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3480521988 0 0) 0x7fba1c0b3ba0 con 0x7fba1c15e770 2024-04-20T10:21:59.134 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.134+0000 7fba11400640 1 --2- 172.21.15.121:0/3494713981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba1c162210 0x7fba1c162670 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fba040097a0 tx=0x7fba0402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7677f698a1df1358 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:59.134 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.134+0000 7fba19400640 1 -- 172.21.15.121:0/3494713981 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba040315a0 con 0x7fba1c162210 2024-04-20T10:21:59.134 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.134+0000 7fba19400640 1 -- 172.21.15.121:0/3494713981 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba04037070 con 0x7fba1c162210 2024-04-20T10:21:59.134 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.134+0000 7fba19400640 1 -- 172.21.15.121:0/3494713981 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba04031960 con 0x7fba1c162210 2024-04-20T10:21:59.135 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.134+0000 7fba1b200640 1 -- 172.21.15.121:0/3494713981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba1c162210 msgr2=0x7fba1c162670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:59.135 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.135+0000 7fba1b200640 1 --2- 172.21.15.121:0/3494713981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba1c162210 0x7fba1c162670 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fba040097a0 tx=0x7fba0402b840 comp rx=0 tx=0).stop 2024-04-20T10:21:59.135 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.135+0000 7fba1b200640 1 -- 172.21.15.121:0/3494713981 shutdown_connections 2024-04-20T10:21:59.135 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.135+0000 7fba1b200640 1 --2- 172.21.15.121:0/3494713981 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fba1c0b6400 0x7fba1c0b6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:59.135 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.135+0000 7fba1b200640 1 --2- 172.21.15.121:0/3494713981 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba1c162210 0x7fba1c162670 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:59.136 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.135+0000 7fba1b200640 1 -- 172.21.15.121:0/3494713981 >> 172.21.15.121:0/3494713981 conn(0x7fba1c09d6d0 msgr2=0x7fba1c0b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:59.136 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.135+0000 7fba1b200640 1 -- 172.21.15.121:0/3494713981 shutdown_connections 2024-04-20T10:21:59.136 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.135+0000 7fba1b200640 1 -- 172.21.15.121:0/3494713981 wait complete. 2024-04-20T10:21:59.137 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.136+0000 7fba1b200640 1 Processor -- start 2024-04-20T10:21:59.137 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.137+0000 7fba1b200640 1 -- start start 2024-04-20T10:21:59.138 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.137+0000 7fba1b200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fba1c0b6400 0x7fba1c150790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:59.138 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.137+0000 7fba1b200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba1c15e770 0x7fba1c06e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:59.138 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.137+0000 7fba1b200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba1c162210 0x7fba1c160b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:59.138 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.137+0000 7fba1b200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c0d3e30 con 0x7fba1c162210 2024-04-20T10:21:59.138 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.138+0000 7fba1b200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c0b5ee0 con 0x7fba1c15e770 2024-04-20T10:21:59.138 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.138+0000 7fba1b200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fba1c0d3120 con 0x7fba1c0b6400 2024-04-20T10:21:59.139 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.138+0000 7fba1a800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba1c162210 0x7fba1c160b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:59.139 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.138+0000 7fba1a800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba1c162210 0x7fba1c160b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42982/0 (socket says 172.21.15.121:42982) 2024-04-20T10:21:59.139 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.138+0000 7fba1a800640 1 -- 172.21.15.121:0/3324626060 learned_addr learned my addr 172.21.15.121:0/3324626060 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:59.140 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.138+0000 7fba19e00640 1 --2- 172.21.15.121:0/3324626060 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fba1c0b6400 0x7fba1c150790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:59.140 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.138+0000 7fba11400640 1 --2- 172.21.15.121:0/3324626060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba1c15e770 0x7fba1c06e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:59.140 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.138+0000 7fba1a800640 1 -- 172.21.15.121:0/3324626060 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fba1c0b6400 msgr2=0x7fba1c150790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:59.140 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.138+0000 7fba1a800640 1 --2- 172.21.15.121:0/3324626060 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fba1c0b6400 0x7fba1c150790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:59.140 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.138+0000 7fba1a800640 1 -- 172.21.15.121:0/3324626060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba1c15e770 msgr2=0x7fba1c06e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:59.140 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.138+0000 7fba1a800640 1 --2- 172.21.15.121:0/3324626060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba1c15e770 0x7fba1c06e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:59.140 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.138+0000 7fba1a800640 1 -- 172.21.15.121:0/3324626060 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba1c15f7c0 con 0x7fba1c162210 2024-04-20T10:21:59.140 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.139+0000 7fba1a800640 1 --2- 172.21.15.121:0/3324626060 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba1c162210 0x7fba1c160b10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fba1400d830 tx=0x7fba1400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:59.141 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.139+0000 7fba12a00640 1 -- 172.21.15.121:0/3324626060 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1400b6c0 con 0x7fba1c162210 2024-04-20T10:21:59.141 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.139+0000 7fba12a00640 1 -- 172.21.15.121:0/3324626060 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba14019070 con 0x7fba1c162210 2024-04-20T10:21:59.141 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.139+0000 7fba12a00640 1 -- 172.21.15.121:0/3324626060 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1400bac0 con 0x7fba1c162210 2024-04-20T10:21:59.141 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.139+0000 7fba1b200640 1 -- 172.21.15.121:0/3324626060 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba1c15faa0 con 0x7fba1c162210 2024-04-20T10:21:59.141 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.139+0000 7fba1b200640 1 -- 172.21.15.121:0/3324626060 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba1c160060 con 0x7fba1c162210 2024-04-20T10:21:59.141 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.140+0000 7fba1b200640 1 -- 172.21.15.121:0/3324626060 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba1c075b30 con 0x7fba1c162210 2024-04-20T10:21:59.141 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.141+0000 7fba12a00640 1 -- 172.21.15.121:0/3324626060 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fba1401e460 con 0x7fba1c162210 2024-04-20T10:21:59.142 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.142+0000 7fba12a00640 1 --2- 172.21.15.121:0/3324626060 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fb9f806c8f0 0x7fb9f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:59.142 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.142+0000 7fba12a00640 1 -- 172.21.15.121:0/3324626060 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fba1408c570 con 0x7fba1c162210 2024-04-20T10:21:59.143 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.142+0000 7fba19e00640 1 --2- 172.21.15.121:0/3324626060 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fb9f806c8f0 0x7fb9f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:59.143 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.143+0000 7fba19e00640 1 --2- 172.21.15.121:0/3324626060 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fb9f806c8f0 0x7fb9f806edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fba0c0012e0 tx=0x7fba0c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:59.147 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.146+0000 7fba12a00640 1 -- 172.21.15.121:0/3324626060 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fba1405a630 con 0x7fba1c162210 2024-04-20T10:21:59.347 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.346+0000 7fba1b200640 1 -- 172.21.15.121:0/3324626060 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fba1c0cb170 con 0x7fba1c162210 2024-04-20T10:21:59.347 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.347+0000 7fba12a00640 1 -- 172.21.15.121:0/3324626060 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fba14059f30 con 0x7fba1c162210 2024-04-20T10:21:59.347 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2024-04-20T10:21:59.350 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.349+0000 7fba1b200640 1 -- 172.21.15.121:0/3324626060 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fb9f806c8f0 msgr2=0x7fb9f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:59.350 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.349+0000 7fba1b200640 1 --2- 172.21.15.121:0/3324626060 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fb9f806c8f0 0x7fb9f806edb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fba0c0012e0 tx=0x7fba0c009040 comp rx=0 tx=0).stop 2024-04-20T10:21:59.350 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.349+0000 7fba1b200640 1 -- 172.21.15.121:0/3324626060 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba1c162210 msgr2=0x7fba1c160b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:59.350 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.349+0000 7fba1b200640 1 --2- 172.21.15.121:0/3324626060 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba1c162210 0x7fba1c160b10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fba1400d830 tx=0x7fba1400dcf0 comp rx=0 tx=0).stop 2024-04-20T10:21:59.351 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.350+0000 7fba1b200640 1 -- 172.21.15.121:0/3324626060 shutdown_connections 2024-04-20T10:21:59.351 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.350+0000 7fba1b200640 1 --2- 172.21.15.121:0/3324626060 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fba1c162210 0x7fba1c160b10 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:59.351 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.350+0000 7fba1b200640 1 --2- 172.21.15.121:0/3324626060 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fb9f806c8f0 0x7fb9f806edb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:59.351 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.350+0000 7fba1b200640 1 --2- 172.21.15.121:0/3324626060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba1c15e770 0x7fba1c06e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:59.351 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.350+0000 7fba1b200640 1 --2- 172.21.15.121:0/3324626060 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fba1c0b6400 0x7fba1c150790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:59.351 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.350+0000 7fba1b200640 1 -- 172.21.15.121:0/3324626060 >> 172.21.15.121:0/3324626060 conn(0x7fba1c09d6d0 msgr2=0x7fba1c0b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:59.351 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.350+0000 7fba1b200640 1 -- 172.21.15.121:0/3324626060 shutdown_connections 2024-04-20T10:21:59.351 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.350+0000 7fba1b200640 1 -- 172.21.15.121:0/3324626060 wait complete. 2024-04-20T10:21:59.370 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:21:59.573 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.572+0000 7feaba800640 1 Processor -- start 2024-04-20T10:21:59.573 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.572+0000 7feaba800640 1 -- start start 2024-04-20T10:21:59.575 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.574+0000 7feaba800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7feab406ca10 0x7feab406ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:59.575 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.574+0000 7feaba800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feab415b5d0 con 0x7feab40d0010 2024-04-20T10:21:59.576 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.574+0000 7feaba800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feab406e730 con 0x7feab415b1f0 2024-04-20T10:21:59.576 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.574+0000 7feaba800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7feab406d3a0 con 0x7feab406ca10 2024-04-20T10:21:59.576 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.574+0000 7feab8a00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7feab40d0010 0x7feab40adc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:40464/0 (socket says 172.21.15.121:40464) 2024-04-20T10:21:59.576 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.574+0000 7feab8a00640 1 -- 172.21.15.121:0/1263861783 learned_addr learned my addr 172.21.15.121:0/1263861783 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:59.576 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.575+0000 7feab9e00640 1 --2- 172.21.15.121:0/1263861783 >> v2:172.21.15.123:3300/0 conn(0x7feab406ca10 0x7feab406ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:59.576 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.575+0000 7feab3e00640 1 -- 172.21.15.121:0/1263861783 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1285855527 0 0) 0x7feab415b5d0 con 0x7feab40d0010 2024-04-20T10:21:59.576 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.575+0000 7feab3e00640 1 -- 172.21.15.121:0/1263861783 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea9c005730 con 0x7feab40d0010 2024-04-20T10:21:59.580 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.575+0000 7feab3e00640 1 -- 172.21.15.121:0/1263861783 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2178545221 0 0) 0x7fea9c005730 con 0x7feab40d0010 2024-04-20T10:21:59.580 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.575+0000 7feab3e00640 1 -- 172.21.15.121:0/1263861783 >> v2:172.21.15.123:3300/0 conn(0x7feab406ca10 msgr2=0x7feab406ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:59.580 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.575+0000 7feab3e00640 1 --2- 172.21.15.121:0/1263861783 >> v2:172.21.15.123:3300/0 conn(0x7feab406ca10 0x7feab406ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:59.580 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.575+0000 7feab3e00640 1 -- 172.21.15.121:0/1263861783 >> v1:172.21.15.123:6789/0 conn(0x7feab415b1f0 legacy=0x7feab415f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:59.580 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.575+0000 7feab3e00640 1 -- 172.21.15.121:0/1263861783 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feab40ae310 con 0x7feab40d0010 2024-04-20T10:21:59.580 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.575+0000 7feab3e00640 1 -- 172.21.15.121:0/1263861783 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7feaa0001670 con 0x7feab40d0010 2024-04-20T10:21:59.581 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.575+0000 7feab3e00640 1 -- 172.21.15.121:0/1263861783 >> v1:172.21.15.121:6789/0 conn(0x7feab40d0010 legacy=0x7feab40adc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:59.581 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.575+0000 7feab3e00640 1 --2- 172.21.15.121:0/1263861783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea9c005bc0 0x7fea9c005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:59.581 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.575+0000 7feab3e00640 1 --2- 172.21.15.121:0/1263861783 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea9c006a10 0x7fea9c006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:59.581 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.575+0000 7feab3e00640 1 --2- 172.21.15.121:0/1263861783 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea9c0073d0 0x7fea9c00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:59.581 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.575+0000 7feab3e00640 1 -- 172.21.15.121:0/1263861783 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7feab406d3a0 con 0x7fea9c0073d0 2024-04-20T10:21:59.581 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.575+0000 7feab3e00640 1 -- 172.21.15.121:0/1263861783 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fea9c00e370 con 0x7fea9c005bc0 2024-04-20T10:21:59.581 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.576+0000 7feab3e00640 1 -- 172.21.15.121:0/1263861783 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fea9c00e570 con 0x7fea9c006a10 2024-04-20T10:21:59.581 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.576+0000 7feab9e00640 1 --2- 172.21.15.121:0/1263861783 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea9c0073d0 0x7fea9c00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:59.581 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.576+0000 7feab8a00640 1 --2- 172.21.15.121:0/1263861783 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea9c006a10 0x7fea9c006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:59.582 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.576+0000 7feab9400640 1 --2- 172.21.15.121:0/1263861783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea9c005bc0 0x7fea9c005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:59.582 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.576+0000 7feab9e00640 1 -- 172.21.15.121:0/1263861783 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea9c006a10 msgr2=0x7fea9c006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:59.582 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.576+0000 7feab9e00640 1 --2- 172.21.15.121:0/1263861783 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea9c006a10 0x7fea9c006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:59.582 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.576+0000 7feab9e00640 1 -- 172.21.15.121:0/1263861783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea9c005bc0 msgr2=0x7fea9c005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:59.582 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.576+0000 7feab9e00640 1 --2- 172.21.15.121:0/1263861783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea9c005bc0 0x7fea9c005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:59.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.576+0000 7feab9e00640 1 -- 172.21.15.121:0/1263861783 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea9c005730 con 0x7fea9c0073d0 2024-04-20T10:21:59.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.576+0000 7feab9400640 1 --2- 172.21.15.121:0/1263861783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea9c005bc0 0x7fea9c005fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:21:59.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.577+0000 7feab9e00640 1 -- 172.21.15.121:0/1263861783 reap_dead start 2024-04-20T10:21:59.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.577+0000 7feab9e00640 1 --2- 172.21.15.121:0/1263861783 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea9c0073d0 0x7fea9c00dbd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7feaac009fc0 tx=0x7fea9c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=934d46969f77cda4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:59.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.578+0000 7feab3e00640 1 -- 172.21.15.121:0/1263861783 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaac009c10 con 0x7fea9c0073d0 2024-04-20T10:21:59.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.578+0000 7feab3e00640 1 -- 172.21.15.121:0/1263861783 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaac02cca0 con 0x7fea9c0073d0 2024-04-20T10:21:59.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.578+0000 7feab3e00640 1 -- 172.21.15.121:0/1263861783 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaac003bd0 con 0x7fea9c0073d0 2024-04-20T10:21:59.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.578+0000 7feaba800640 1 -- 172.21.15.121:0/1263861783 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea9c0073d0 msgr2=0x7fea9c00dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:59.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.578+0000 7feaba800640 1 --2- 172.21.15.121:0/1263861783 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea9c0073d0 0x7fea9c00dbd0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7feaac009fc0 tx=0x7fea9c007a40 comp rx=0 tx=0).stop 2024-04-20T10:21:59.584 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.578+0000 7feaba800640 1 -- 172.21.15.121:0/1263861783 shutdown_connections 2024-04-20T10:21:59.584 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.578+0000 7feaba800640 1 --2- 172.21.15.121:0/1263861783 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea9c0073d0 0x7fea9c00dbd0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:59.584 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.578+0000 7feaba800640 1 -- 172.21.15.121:0/1263861783 >> 172.21.15.121:0/1263861783 conn(0x7feab40b5270 msgr2=0x7feab40b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:59.584 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.578+0000 7feaba800640 1 -- 172.21.15.121:0/1263861783 shutdown_connections 2024-04-20T10:21:59.584 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.578+0000 7feaba800640 1 -- 172.21.15.121:0/1263861783 wait complete. 2024-04-20T10:21:59.587 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.580+0000 7feaba800640 1 Processor -- start 2024-04-20T10:21:59.587 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.580+0000 7feaba800640 1 -- start start 2024-04-20T10:21:59.588 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.581+0000 7feaba800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea9c0073d0 0x7feab4150bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:59.588 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.581+0000 7feaba800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feab4151130 0x7feab4158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:59.588 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.581+0000 7feaba800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feab41594b0 0x7feab4159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:59.588 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.581+0000 7feaba800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7feab40d0b90 con 0x7fea9c0073d0 2024-04-20T10:21:59.588 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.581+0000 7feaba800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feab40d08b0 con 0x7feab41594b0 2024-04-20T10:21:59.588 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.581+0000 7feaba800640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7feab40d0e70 con 0x7feab4151130 2024-04-20T10:21:59.588 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.582+0000 7feab9400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea9c0073d0 0x7feab4150bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:59.588 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.582+0000 7feab9400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea9c0073d0 0x7feab4150bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43006/0 (socket says 172.21.15.121:43006) 2024-04-20T10:21:59.588 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.582+0000 7feab9400640 1 -- 172.21.15.121:0/2620085176 learned_addr learned my addr 172.21.15.121:0/2620085176 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:21:59.588 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.582+0000 7feab8a00640 1 --2- 172.21.15.121:0/2620085176 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feab4151130 0x7feab4158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:59.588 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.582+0000 7feab9400640 1 -- 172.21.15.121:0/2620085176 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feab4151130 msgr2=0x7feab4158f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:59.588 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.582+0000 7feab9400640 1 --2- 172.21.15.121:0/2620085176 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feab4151130 0x7feab4158f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:59.589 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.582+0000 7feab9400640 1 -- 172.21.15.121:0/2620085176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feab41594b0 msgr2=0x7feab4159940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:59.589 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.582+0000 7feab9400640 1 --2- 172.21.15.121:0/2620085176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feab41594b0 0x7feab4159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:59.589 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.582+0000 7feab9400640 1 -- 172.21.15.121:0/2620085176 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feab4159e80 con 0x7fea9c0073d0 2024-04-20T10:21:59.589 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.582+0000 7feab9400640 1 --2- 172.21.15.121:0/2620085176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea9c0073d0 0x7feab4150bf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7feaa800d560 tx=0x7feaa800da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:59.589 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.582+0000 7feab1600640 1 -- 172.21.15.121:0/2620085176 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa8005480 con 0x7fea9c0073d0 2024-04-20T10:21:59.589 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.583+0000 7feab1600640 1 -- 172.21.15.121:0/2620085176 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaa8017070 con 0x7fea9c0073d0 2024-04-20T10:21:59.589 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.583+0000 7feab1600640 1 -- 172.21.15.121:0/2620085176 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa8003810 con 0x7fea9c0073d0 2024-04-20T10:21:59.589 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.583+0000 7feaba800640 1 -- 172.21.15.121:0/2620085176 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feab4149a20 con 0x7fea9c0073d0 2024-04-20T10:21:59.589 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.583+0000 7feaba800640 1 -- 172.21.15.121:0/2620085176 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feab41518e0 con 0x7fea9c0073d0 2024-04-20T10:21:59.589 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.584+0000 7feaba800640 1 -- 172.21.15.121:0/2620085176 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feab4075b30 con 0x7fea9c0073d0 2024-04-20T10:21:59.589 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.585+0000 7feab1600640 1 -- 172.21.15.121:0/2620085176 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feaa80039b0 con 0x7fea9c0073d0 2024-04-20T10:21:59.589 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.586+0000 7feab1600640 1 --2- 172.21.15.121:0/2620085176 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fea9406c8f0 0x7fea9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:59.590 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.586+0000 7feab1600640 1 -- 172.21.15.121:0/2620085176 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7feaa808be10 con 0x7fea9c0073d0 2024-04-20T10:21:59.590 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.586+0000 7feab8a00640 1 --2- 172.21.15.121:0/2620085176 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fea9406c8f0 0x7fea9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:59.590 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.586+0000 7feab8a00640 1 --2- 172.21.15.121:0/2620085176 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fea9406c8f0 0x7fea9406edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7feab4151530 tx=0x7feaa0000f10 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:21:59.590 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.590+0000 7feab1600640 1 -- 172.21.15.121:0/2620085176 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7feaa808f030 con 0x7fea9c0073d0 2024-04-20T10:21:59.778 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.777+0000 7feaba800640 1 -- 172.21.15.121:0/2620085176 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7feab4073a80 con 0x7fea9c0073d0 2024-04-20T10:21:59.779 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.778+0000 7feab1600640 1 -- 172.21.15.121:0/2620085176 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7feaa8019310 con 0x7fea9c0073d0 2024-04-20T10:21:59.779 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 32 2024-04-20T10:21:59.782 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.781+0000 7feaba800640 1 -- 172.21.15.121:0/2620085176 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fea9406c8f0 msgr2=0x7fea9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:59.782 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.781+0000 7feaba800640 1 --2- 172.21.15.121:0/2620085176 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fea9406c8f0 0x7fea9406edb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7feab4151530 tx=0x7feaa0000f10 comp rx=0 tx=0).stop 2024-04-20T10:21:59.782 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.781+0000 7feaba800640 1 -- 172.21.15.121:0/2620085176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea9c0073d0 msgr2=0x7feab4150bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:59.782 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.781+0000 7feaba800640 1 --2- 172.21.15.121:0/2620085176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea9c0073d0 0x7feab4150bf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7feaa800d560 tx=0x7feaa800da20 comp rx=0 tx=0).stop 2024-04-20T10:21:59.782 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.781+0000 7feaba800640 1 -- 172.21.15.121:0/2620085176 shutdown_connections 2024-04-20T10:21:59.782 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.781+0000 7feaba800640 1 --2- 172.21.15.121:0/2620085176 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fea9406c8f0 0x7fea9406edb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:59.782 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.781+0000 7feaba800640 1 --2- 172.21.15.121:0/2620085176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feab41594b0 0x7feab4159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:59.783 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.781+0000 7feaba800640 1 --2- 172.21.15.121:0/2620085176 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feab4151130 0x7feab4158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:59.783 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.781+0000 7feaba800640 1 --2- 172.21.15.121:0/2620085176 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea9c0073d0 0x7feab4150bf0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:59.783 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.781+0000 7feaba800640 1 -- 172.21.15.121:0/2620085176 >> 172.21.15.121:0/2620085176 conn(0x7feab40b5270 msgr2=0x7feab4156ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:21:59.783 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.781+0000 7feaba800640 1 -- 172.21.15.121:0/2620085176 shutdown_connections 2024-04-20T10:21:59.783 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.781+0000 7feaba800640 1 -- 172.21.15.121:0/2620085176 wait complete. 2024-04-20T10:21:59.799 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:21:59.994 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.994+0000 7f767fc00640 1 Processor -- start 2024-04-20T10:21:59.995 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.994+0000 7f767fc00640 1 -- start start 2024-04-20T10:21:59.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.995+0000 7f767fc00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f767815b3b0 0x7f767806eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:59.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.995+0000 7f767fc00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76780ade40 con 0x7f76780d0360 2024-04-20T10:21:59.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.995+0000 7f767fc00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76780ae0e0 con 0x7f767806dcb0 2024-04-20T10:21:59.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.995+0000 7f767fc00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f767815b790 con 0x7f767815b3b0 2024-04-20T10:21:59.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.996+0000 7f767dc00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f76780d0360 0x7f76780d0770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:40478/0 (socket says 172.21.15.121:40478) 2024-04-20T10:21:59.996 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.996+0000 7f767dc00640 1 -- 172.21.15.121:0/4273910537 learned_addr learned my addr 172.21.15.121:0/4273910537 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:21:59.998 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.996+0000 7f7677400640 1 -- 172.21.15.121:0/4273910537 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1838479433 0 0) 0x7f76780ade40 con 0x7f76780d0360 2024-04-20T10:21:59.998 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.996+0000 7f767d200640 1 --2- 172.21.15.121:0/4273910537 >> v2:172.21.15.123:3300/0 conn(0x7f767815b3b0 0x7f767806eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:59.998 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.996+0000 7f7677400640 1 -- 172.21.15.121:0/4273910537 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7660005730 con 0x7f76780d0360 2024-04-20T10:21:59.998 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.996+0000 7f7677400640 1 -- 172.21.15.121:0/4273910537 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2191985419 0 0) 0x7f7660005730 con 0x7f76780d0360 2024-04-20T10:21:59.998 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.996+0000 7f7677400640 1 -- 172.21.15.121:0/4273910537 >> v2:172.21.15.123:3300/0 conn(0x7f767815b3b0 msgr2=0x7f767806eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:59.998 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.996+0000 7f7677400640 1 --2- 172.21.15.121:0/4273910537 >> v2:172.21.15.123:3300/0 conn(0x7f767815b3b0 0x7f767806eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:59.998 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.996+0000 7f7677400640 1 -- 172.21.15.121:0/4273910537 >> v1:172.21.15.123:6789/0 conn(0x7f767806dcb0 legacy=0x7f7678071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:59.999 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.996+0000 7f7677400640 1 -- 172.21.15.121:0/4273910537 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7678072260 con 0x7f76780d0360 2024-04-20T10:21:59.999 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.997+0000 7f7677400640 1 -- 172.21.15.121:0/4273910537 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f76700019f0 con 0x7f76780d0360 2024-04-20T10:21:59.999 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.997+0000 7f7677400640 1 -- 172.21.15.121:0/4273910537 >> v1:172.21.15.121:6789/0 conn(0x7f76780d0360 legacy=0x7f76780d0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:21:59.999 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.997+0000 7f7677400640 1 --2- 172.21.15.121:0/4273910537 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7660005c10 0x7f7660006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:59.999 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.997+0000 7f7677400640 1 --2- 172.21.15.121:0/4273910537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7660006a10 0x7f7660006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:59.999 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.997+0000 7f7677400640 1 --2- 172.21.15.121:0/4273910537 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7660007420 0x7f766000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:21:59.999 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.997+0000 7f7677400640 1 -- 172.21.15.121:0/4273910537 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f767815b790 con 0x7f7660007420 2024-04-20T10:21:59.999 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.997+0000 7f7677400640 1 -- 172.21.15.121:0/4273910537 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f766000e370 con 0x7f7660006a10 2024-04-20T10:21:59.999 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.997+0000 7f7677400640 1 -- 172.21.15.121:0/4273910537 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f766000e570 con 0x7f7660005c10 2024-04-20T10:21:59.999 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.997+0000 7f767dc00640 1 --2- 172.21.15.121:0/4273910537 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7660007420 0x7f766000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:59.999 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.997+0000 7f767d200640 1 --2- 172.21.15.121:0/4273910537 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7660005c10 0x7f7660006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:21:59.999 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.997+0000 7f767dc00640 1 -- 172.21.15.121:0/4273910537 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7660005c10 msgr2=0x7f7660006030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:21:59.999 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.997+0000 7f767dc00640 1 --2- 172.21.15.121:0/4273910537 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7660005c10 0x7f7660006030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:21:59.999 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.997+0000 7f767dc00640 1 -- 172.21.15.121:0/4273910537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7660006a10 msgr2=0x7f7660006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:00.000 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.997+0000 7f767dc00640 1 --2- 172.21.15.121:0/4273910537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7660006a10 0x7f7660006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:00.000 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.997+0000 7f767dc00640 1 -- 172.21.15.121:0/4273910537 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7660005730 con 0x7f7660007420 2024-04-20T10:22:00.001 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.998+0000 7f767dc00640 1 -- 172.21.15.121:0/4273910537 reap_dead start 2024-04-20T10:22:00.001 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.999+0000 7f767dc00640 1 --2- 172.21.15.121:0/4273910537 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7660007420 0x7f766000dbd0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f767000a7b0 tx=0x7f7660007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6099dd26a8d0dda server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:00.001 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.999+0000 7f7677400640 1 -- 172.21.15.121:0/4273910537 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767000a3d0 con 0x7f7660007420 2024-04-20T10:22:00.001 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.999+0000 7f7677400640 1 -- 172.21.15.121:0/4273910537 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7670034650 con 0x7f7660007420 2024-04-20T10:22:00.001 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:21:59.999+0000 7f7677400640 1 -- 172.21.15.121:0/4273910537 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7670034a40 con 0x7f7660007420 2024-04-20T10:22:00.001 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.000+0000 7f767fc00640 1 -- 172.21.15.121:0/4273910537 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7660007420 msgr2=0x7f766000dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:00.001 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.000+0000 7f767fc00640 1 --2- 172.21.15.121:0/4273910537 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7660007420 0x7f766000dbd0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f767000a7b0 tx=0x7f7660007a40 comp rx=0 tx=0).stop 2024-04-20T10:22:00.001 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.000+0000 7f767fc00640 1 -- 172.21.15.121:0/4273910537 shutdown_connections 2024-04-20T10:22:00.002 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.000+0000 7f767fc00640 1 --2- 172.21.15.121:0/4273910537 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7660007420 0x7f766000dbd0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:00.002 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.000+0000 7f767fc00640 1 -- 172.21.15.121:0/4273910537 >> 172.21.15.121:0/4273910537 conn(0x7f76780b3350 msgr2=0x7f76780b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:00.002 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.000+0000 7f767fc00640 1 -- 172.21.15.121:0/4273910537 shutdown_connections 2024-04-20T10:22:00.002 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.000+0000 7f767fc00640 1 -- 172.21.15.121:0/4273910537 wait complete. 2024-04-20T10:22:00.002 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.001+0000 7f767fc00640 1 Processor -- start 2024-04-20T10:22:00.002 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.002+0000 7f767fc00640 1 -- start start 2024-04-20T10:22:00.005 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.002+0000 7f767fc00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7660007420 0x7f767814b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:00.005 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.002+0000 7f767fc00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7678154f00 0x7f76780b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:00.005 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.002+0000 7f767fc00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7678155380 0x7f76780b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:00.005 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.002+0000 7f767fc00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f76780ae940 con 0x7f7678155380 2024-04-20T10:22:00.005 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.002+0000 7f767fc00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f76780ae660 con 0x7f7660007420 2024-04-20T10:22:00.005 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.002+0000 7f767fc00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f76780aec20 con 0x7f7678154f00 2024-04-20T10:22:00.005 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.003+0000 7f767dc00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7678155380 0x7f76780b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:00.005 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.003+0000 7f767dc00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7678155380 0x7f76780b1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43012/0 (socket says 172.21.15.121:43012) 2024-04-20T10:22:00.005 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.003+0000 7f767dc00640 1 -- 172.21.15.121:0/2262149019 learned_addr learned my addr 172.21.15.121:0/2262149019 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:00.005 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.003+0000 7f767d200640 1 --2- 172.21.15.121:0/2262149019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7660007420 0x7f767814b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:00.005 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.003+0000 7f7677e00640 1 --2- 172.21.15.121:0/2262149019 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7678154f00 0x7f76780b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:00.005 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.003+0000 7f767dc00640 1 -- 172.21.15.121:0/2262149019 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7678154f00 msgr2=0x7f76780b1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:00.005 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.003+0000 7f767dc00640 1 --2- 172.21.15.121:0/2262149019 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7678154f00 0x7f76780b1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:00.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.003+0000 7f767dc00640 1 -- 172.21.15.121:0/2262149019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7660007420 msgr2=0x7f767814b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:00.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.003+0000 7f767dc00640 1 --2- 172.21.15.121:0/2262149019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7660007420 0x7f767814b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:00.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.003+0000 7f767dc00640 1 -- 172.21.15.121:0/2262149019 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76780af040 con 0x7f7678155380 2024-04-20T10:22:00.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.004+0000 7f767dc00640 1 --2- 172.21.15.121:0/2262149019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7678155380 0x7f76780b1ad0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f767002cbd0 tx=0x7f7670001cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:00.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.004+0000 7f7674c00640 1 -- 172.21.15.121:0/2262149019 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7670044070 con 0x7f7678155380 2024-04-20T10:22:00.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.004+0000 7f7674c00640 1 -- 172.21.15.121:0/2262149019 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7670001730 con 0x7f7678155380 2024-04-20T10:22:00.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.004+0000 7f7674c00640 1 -- 172.21.15.121:0/2262149019 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7670039040 con 0x7f7678155380 2024-04-20T10:22:00.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.004+0000 7f767fc00640 1 -- 172.21.15.121:0/2262149019 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7678155870 con 0x7f7678155380 2024-04-20T10:22:00.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.004+0000 7f767fc00640 1 -- 172.21.15.121:0/2262149019 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76780af5c0 con 0x7f7678155380 2024-04-20T10:22:00.007 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.005+0000 7f767fc00640 1 -- 172.21.15.121:0/2262149019 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7678068980 con 0x7f7678155380 2024-04-20T10:22:00.007 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.006+0000 7f7674c00640 1 -- 172.21.15.121:0/2262149019 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7670047070 con 0x7f7678155380 2024-04-20T10:22:00.007 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.007+0000 7f7674c00640 1 --2- 172.21.15.121:0/2262149019 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f765c06c920 0x7f765c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:00.007 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.007+0000 7f7674c00640 1 -- 172.21.15.121:0/2262149019 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f76700ad810 con 0x7f7678155380 2024-04-20T10:22:00.007 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.007+0000 7f767d200640 1 --2- 172.21.15.121:0/2262149019 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f765c06c920 0x7f765c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:00.008 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.008+0000 7f767d200640 1 --2- 172.21.15.121:0/2262149019 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f765c06c920 0x7f765c06ede0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f766c006fd0 tx=0x7f766c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:00.011 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.011+0000 7f7674c00640 1 -- 172.21.15.121:0/2262149019 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f767007b150 con 0x7f7678155380 2024-04-20T10:22:00.211 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.211+0000 7f767fc00640 1 -- 172.21.15.121:0/2262149019 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f767815fb40 con 0x7f7678155380 2024-04-20T10:22:00.212 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.212+0000 7f7674c00640 1 -- 172.21.15.121:0/2262149019 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f767007f450 con 0x7f7678155380 2024-04-20T10:22:00.212 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2024-04-20T10:22:00.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.214+0000 7f767fc00640 1 -- 172.21.15.121:0/2262149019 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f765c06c920 msgr2=0x7f765c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:00.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.214+0000 7f767fc00640 1 --2- 172.21.15.121:0/2262149019 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f765c06c920 0x7f765c06ede0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f766c006fd0 tx=0x7f766c008040 comp rx=0 tx=0).stop 2024-04-20T10:22:00.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.214+0000 7f767fc00640 1 -- 172.21.15.121:0/2262149019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7678155380 msgr2=0x7f76780b1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:00.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.214+0000 7f767fc00640 1 --2- 172.21.15.121:0/2262149019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7678155380 0x7f76780b1ad0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f767002cbd0 tx=0x7f7670001cf0 comp rx=0 tx=0).stop 2024-04-20T10:22:00.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.214+0000 7f767fc00640 1 -- 172.21.15.121:0/2262149019 shutdown_connections 2024-04-20T10:22:00.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.214+0000 7f767fc00640 1 --2- 172.21.15.121:0/2262149019 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7678155380 0x7f76780b1ad0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:00.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.214+0000 7f767fc00640 1 --2- 172.21.15.121:0/2262149019 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7678154f00 0x7f76780b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:00.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.214+0000 7f767fc00640 1 --2- 172.21.15.121:0/2262149019 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f765c06c920 0x7f765c06ede0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:00.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.214+0000 7f767fc00640 1 --2- 172.21.15.121:0/2262149019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7660007420 0x7f767814b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:00.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.214+0000 7f767fc00640 1 -- 172.21.15.121:0/2262149019 >> 172.21.15.121:0/2262149019 conn(0x7f76780b3350 msgr2=0x7f7678159ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:00.216 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.214+0000 7f767fc00640 1 -- 172.21.15.121:0/2262149019 shutdown_connections 2024-04-20T10:22:00.216 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.214+0000 7f767fc00640 1 -- 172.21.15.121:0/2262149019 wait complete. 2024-04-20T10:22:00.233 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T10:22:00.425 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.424+0000 7f7c97200640 1 Processor -- start 2024-04-20T10:22:00.425 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.424+0000 7f7c97200640 1 -- start start 2024-04-20T10:22:00.427 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.426+0000 7f7c97200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7c9815f2b0 0x7f7c98169420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:00.427 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.426+0000 7f7c97200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c9815f6e0 con 0x7f7c98158ca0 2024-04-20T10:22:00.427 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.426+0000 7f7c97200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c9815f960 con 0x7f7c9806e2a0 2024-04-20T10:22:00.427 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.426+0000 7f7c97200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f7c98147100 con 0x7f7c9815f2b0 2024-04-20T10:22:00.427 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.426+0000 7f7c96800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7c9815f2b0 0x7f7c98169420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:00.427 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.426+0000 7f7c96800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7c9815f2b0 0x7f7c98169420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43026/0 (socket says 172.21.15.121:43026) 2024-04-20T10:22:00.427 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.426+0000 7f7c96800640 1 -- 172.21.15.121:0/1948848629 learned_addr learned my addr 172.21.15.121:0/1948848629 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:00.427 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.426+0000 7f7c95e00640 1 --1- 172.21.15.121:0/1948848629 >> v1:172.21.15.121:6789/0 conn(0x7f7c98158ca0 0x7f7c9806db90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:40492/0 (socket says 172.21.15.121:40492) 2024-04-20T10:22:00.428 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.427+0000 7f7c96800640 1 -- 172.21.15.121:0/1948848629 >> v1:172.21.15.123:6789/0 conn(0x7f7c9806e2a0 legacy=0x7f7c9815eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:00.428 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.427+0000 7f7c96800640 1 -- 172.21.15.121:0/1948848629 >> v1:172.21.15.121:6789/0 conn(0x7f7c98158ca0 legacy=0x7f7c9806db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:00.428 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.427+0000 7f7c96800640 1 -- 172.21.15.121:0/1948848629 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c9806b9a0 con 0x7f7c9815f2b0 2024-04-20T10:22:00.428 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.428+0000 7f7c96800640 1 --2- 172.21.15.121:0/1948848629 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c9815f2b0 0x7f7c98169420 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7c90009e50 tx=0x7f7c9002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f20be7460ee6569 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:00.429 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.428+0000 7f7c95400640 1 -- 172.21.15.121:0/1948848629 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c9002cb20 con 0x7f7c9815f2b0 2024-04-20T10:22:00.429 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.428+0000 7f7c95400640 1 -- 172.21.15.121:0/1948848629 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c9002ccc0 con 0x7f7c9815f2b0 2024-04-20T10:22:00.429 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.428+0000 7f7c95400640 1 -- 172.21.15.121:0/1948848629 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c90034610 con 0x7f7c9815f2b0 2024-04-20T10:22:00.430 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.429+0000 7f7c97200640 1 -- 172.21.15.121:0/1948848629 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c9815f2b0 msgr2=0x7f7c98169420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:00.430 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.429+0000 7f7c97200640 1 --2- 172.21.15.121:0/1948848629 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c9815f2b0 0x7f7c98169420 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7c90009e50 tx=0x7f7c9002bf80 comp rx=0 tx=0).stop 2024-04-20T10:22:00.430 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.429+0000 7f7c97200640 1 -- 172.21.15.121:0/1948848629 shutdown_connections 2024-04-20T10:22:00.430 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.429+0000 7f7c97200640 1 --2- 172.21.15.121:0/1948848629 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c9815f2b0 0x7f7c98169420 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:00.430 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.429+0000 7f7c97200640 1 -- 172.21.15.121:0/1948848629 >> 172.21.15.121:0/1948848629 conn(0x7f7c980b5270 msgr2=0x7f7c980b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:00.430 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.429+0000 7f7c97200640 1 -- 172.21.15.121:0/1948848629 shutdown_connections 2024-04-20T10:22:00.430 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.429+0000 7f7c97200640 1 -- 172.21.15.121:0/1948848629 wait complete. 2024-04-20T10:22:00.431 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.431+0000 7f7c97200640 1 Processor -- start 2024-04-20T10:22:00.431 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.431+0000 7f7c97200640 1 -- start start 2024-04-20T10:22:00.434 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.432+0000 7f7c97200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7c9806e2a0 0x7f7c98072530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:00.434 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.432+0000 7f7c97200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c98158ca0 0x7f7c981565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:00.434 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.432+0000 7f7c97200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c9815f2b0 0x7f7c981f7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:00.434 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.432+0000 7f7c97200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9806c240 con 0x7f7c98158ca0 2024-04-20T10:22:00.434 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.432+0000 7f7c97200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7c9806bf60 con 0x7f7c9815f2b0 2024-04-20T10:22:00.434 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.432+0000 7f7c97200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f7c9806c760 con 0x7f7c9806e2a0 2024-04-20T10:22:00.434 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.432+0000 7f7c8fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c98158ca0 0x7f7c981565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:00.434 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.432+0000 7f7c8fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c98158ca0 0x7f7c981565a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43038/0 (socket says 172.21.15.121:43038) 2024-04-20T10:22:00.434 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.432+0000 7f7c8fe00640 1 -- 172.21.15.121:0/3583807813 learned_addr learned my addr 172.21.15.121:0/3583807813 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:00.434 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.432+0000 7f7c96800640 1 --2- 172.21.15.121:0/3583807813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c9815f2b0 0x7f7c981f7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:00.434 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.432+0000 7f7c95e00640 1 --2- 172.21.15.121:0/3583807813 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7c9806e2a0 0x7f7c98072530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:00.435 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.432+0000 7f7c8fe00640 1 -- 172.21.15.121:0/3583807813 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7c9806e2a0 msgr2=0x7f7c98072530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:00.435 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.433+0000 7f7c8fe00640 1 --2- 172.21.15.121:0/3583807813 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7c9806e2a0 0x7f7c98072530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:00.435 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.433+0000 7f7c8fe00640 1 -- 172.21.15.121:0/3583807813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c9815f2b0 msgr2=0x7f7c981f7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:00.435 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.433+0000 7f7c8fe00640 1 --2- 172.21.15.121:0/3583807813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c9815f2b0 0x7f7c981f7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:00.435 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.433+0000 7f7c8fe00640 1 -- 172.21.15.121:0/3583807813 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c9809e200 con 0x7f7c98158ca0 2024-04-20T10:22:00.435 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.433+0000 7f7c96800640 1 --2- 172.21.15.121:0/3583807813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c9815f2b0 0x7f7c981f7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:00.435 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.433+0000 7f7c8fe00640 1 --2- 172.21.15.121:0/3583807813 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c98158ca0 0x7f7c981565a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7c8000c430 tx=0x7f7c8000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:00.435 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.433+0000 7f7c8e000640 1 -- 172.21.15.121:0/3583807813 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c80011790 con 0x7f7c98158ca0 2024-04-20T10:22:00.435 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.433+0000 7f7c8e000640 1 -- 172.21.15.121:0/3583807813 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c80017070 con 0x7f7c98158ca0 2024-04-20T10:22:00.435 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.433+0000 7f7c8e000640 1 -- 172.21.15.121:0/3583807813 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c80011bb0 con 0x7f7c98158ca0 2024-04-20T10:22:00.435 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.433+0000 7f7c97200640 1 -- 172.21.15.121:0/3583807813 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c981f6980 con 0x7f7c98158ca0 2024-04-20T10:22:00.435 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.433+0000 7f7c97200640 1 -- 172.21.15.121:0/3583807813 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c981f6f40 con 0x7f7c98158ca0 2024-04-20T10:22:00.436 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.434+0000 7f7c97200640 1 -- 172.21.15.121:0/3583807813 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c98075b30 con 0x7f7c98158ca0 2024-04-20T10:22:00.436 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.435+0000 7f7c8e000640 1 -- 172.21.15.121:0/3583807813 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7c8001c460 con 0x7f7c98158ca0 2024-04-20T10:22:00.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.436+0000 7f7c8e000640 1 --2- 172.21.15.121:0/3583807813 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7c7406c8f0 0x7f7c7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:00.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.436+0000 7f7c8e000640 1 -- 172.21.15.121:0/3583807813 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f7c8008be50 con 0x7f7c98158ca0 2024-04-20T10:22:00.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.436+0000 7f7c95e00640 1 --2- 172.21.15.121:0/3583807813 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7c7406c8f0 0x7f7c7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:00.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.437+0000 7f7c95e00640 1 --2- 172.21.15.121:0/3583807813 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7c7406c8f0 0x7f7c7406edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7c88000f60 tx=0x7f7c88009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:00.441 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.441+0000 7f7c8e000640 1 -- 172.21.15.121:0/3583807813 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7c80059e80 con 0x7f7c98158ca0 2024-04-20T10:22:00.628 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.628+0000 7f7c97200640 1 -- 172.21.15.121:0/3583807813 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f7c98073d60 con 0x7f7c98158ca0 2024-04-20T10:22:00.629 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.629+0000 7f7c8e000640 1 -- 172.21.15.121:0/3583807813 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f7c80059780 con 0x7f7c98158ca0 2024-04-20T10:22:00.629 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 32 2024-04-20T10:22:00.632 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.631+0000 7f7c97200640 1 -- 172.21.15.121:0/3583807813 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7c7406c8f0 msgr2=0x7f7c7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:00.632 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.631+0000 7f7c97200640 1 --2- 172.21.15.121:0/3583807813 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7c7406c8f0 0x7f7c7406edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7c88000f60 tx=0x7f7c88009040 comp rx=0 tx=0).stop 2024-04-20T10:22:00.632 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.631+0000 7f7c97200640 1 -- 172.21.15.121:0/3583807813 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c98158ca0 msgr2=0x7f7c981565a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:00.632 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.631+0000 7f7c97200640 1 --2- 172.21.15.121:0/3583807813 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c98158ca0 0x7f7c981565a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7c8000c430 tx=0x7f7c8000c8f0 comp rx=0 tx=0).stop 2024-04-20T10:22:00.632 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.631+0000 7f7c97200640 1 -- 172.21.15.121:0/3583807813 shutdown_connections 2024-04-20T10:22:00.632 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.631+0000 7f7c97200640 1 --2- 172.21.15.121:0/3583807813 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7c7406c8f0 0x7f7c7406edb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:00.633 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.631+0000 7f7c97200640 1 --2- 172.21.15.121:0/3583807813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7c9815f2b0 0x7f7c981f7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:00.633 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.631+0000 7f7c97200640 1 --2- 172.21.15.121:0/3583807813 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7c98158ca0 0x7f7c981565a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:00.633 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.631+0000 7f7c97200640 1 --2- 172.21.15.121:0/3583807813 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7c9806e2a0 0x7f7c98072530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:00.633 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.631+0000 7f7c97200640 1 -- 172.21.15.121:0/3583807813 >> 172.21.15.121:0/3583807813 conn(0x7f7c980b5270 msgr2=0x7f7c980ade90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:00.633 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.631+0000 7f7c97200640 1 -- 172.21.15.121:0/3583807813 shutdown_connections 2024-04-20T10:22:00.633 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.632+0000 7f7c97200640 1 -- 172.21.15.121:0/3583807813 wait complete. 2024-04-20T10:22:00.650 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:22:00.841 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.840+0000 7f16c9200640 1 Processor -- start 2024-04-20T10:22:00.841 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.840+0000 7f16c9200640 1 -- start start 2024-04-20T10:22:00.842 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.842+0000 7f16c9200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16c40b2f70 0x7f16c40b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:00.843 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.842+0000 7f16c9200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f16c40adf60 0x7f16c415f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:00.843 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.842+0000 7f16c9200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f16c415faf0 0x7f16c406b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:00.843 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.842+0000 7f16c9200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f16c4071a80 con 0x7f16c40adf60 2024-04-20T10:22:00.843 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.842+0000 7f16c9200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f16c40ae3c0 con 0x7f16c415faf0 2024-04-20T10:22:00.843 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.842+0000 7f16c9200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f16c4071cf0 con 0x7f16c40b2f70 2024-04-20T10:22:00.844 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.842+0000 7f16c2a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f16c40adf60 0x7f16c415f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:00.844 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.843+0000 7f16c2a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f16c40adf60 0x7f16c415f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43046/0 (socket says 172.21.15.121:43046) 2024-04-20T10:22:00.844 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.843+0000 7f16c2a00640 1 -- 172.21.15.121:0/555603823 learned_addr learned my addr 172.21.15.121:0/555603823 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:00.844 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.843+0000 7f16c3e00640 1 --2- 172.21.15.121:0/555603823 >> v2:172.21.15.123:3300/0 conn(0x7f16c415faf0 0x7f16c406b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:00.844 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.843+0000 7f16c3400640 1 --2- 172.21.15.121:0/555603823 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16c40b2f70 0x7f16c40b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:00.844 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.843+0000 7f16c2a00640 1 -- 172.21.15.121:0/555603823 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16c40b2f70 msgr2=0x7f16c40b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:00.844 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.843+0000 7f16c2a00640 1 --2- 172.21.15.121:0/555603823 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16c40b2f70 0x7f16c40b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:00.844 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.843+0000 7f16c2a00640 1 -- 172.21.15.121:0/555603823 >> v2:172.21.15.123:3300/0 conn(0x7f16c415faf0 msgr2=0x7f16c406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:00.844 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.843+0000 7f16c2a00640 1 --2- 172.21.15.121:0/555603823 >> v2:172.21.15.123:3300/0 conn(0x7f16c415faf0 0x7f16c406b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:00.844 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.843+0000 7f16c2a00640 1 -- 172.21.15.121:0/555603823 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16c4071fd0 con 0x7f16c40adf60 2024-04-20T10:22:00.845 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.844+0000 7f16c2a00640 1 --2- 172.21.15.121:0/555603823 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16c40adf60 0x7f16c415f5b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f16c40d1120 tx=0x7f16b002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=630bffba4555c79b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:00.845 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.845+0000 7f16c2000640 1 -- 172.21.15.121:0/555603823 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b0035020 con 0x7f16c40adf60 2024-04-20T10:22:00.845 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.845+0000 7f16c2000640 1 -- 172.21.15.121:0/555603823 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16b0031870 con 0x7f16c40adf60 2024-04-20T10:22:00.845 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.845+0000 7f16c2000640 1 -- 172.21.15.121:0/555603823 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b0031c40 con 0x7f16c40adf60 2024-04-20T10:22:00.846 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.845+0000 7f16c9200640 1 -- 172.21.15.121:0/555603823 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16c40adf60 msgr2=0x7f16c415f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:00.846 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.845+0000 7f16c9200640 1 --2- 172.21.15.121:0/555603823 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16c40adf60 0x7f16c415f5b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f16c40d1120 tx=0x7f16b002b910 comp rx=0 tx=0).stop 2024-04-20T10:22:00.846 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.845+0000 7f16c9200640 1 -- 172.21.15.121:0/555603823 shutdown_connections 2024-04-20T10:22:00.846 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.845+0000 7f16c9200640 1 --2- 172.21.15.121:0/555603823 >> v2:172.21.15.123:3300/0 conn(0x7f16c415faf0 0x7f16c406b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:00.846 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.845+0000 7f16c9200640 1 --2- 172.21.15.121:0/555603823 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16c40adf60 0x7f16c415f5b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:00.846 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.845+0000 7f16c9200640 1 --2- 172.21.15.121:0/555603823 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16c40b2f70 0x7f16c40b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:00.847 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.845+0000 7f16c9200640 1 -- 172.21.15.121:0/555603823 >> 172.21.15.121:0/555603823 conn(0x7f16c40d0040 msgr2=0x7f16c40d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:00.847 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.845+0000 7f16c9200640 1 -- 172.21.15.121:0/555603823 shutdown_connections 2024-04-20T10:22:00.847 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.845+0000 7f16c9200640 1 -- 172.21.15.121:0/555603823 wait complete. 2024-04-20T10:22:00.847 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.847+0000 7f16c9200640 1 Processor -- start 2024-04-20T10:22:00.847 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.847+0000 7f16c9200640 1 -- start start 2024-04-20T10:22:00.849 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.848+0000 7f16c9200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16c40adf60 0x7f16c4150c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:00.849 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.848+0000 7f16c9200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16c40b2f70 0x7f16c40c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:00.849 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.848+0000 7f16c9200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16c415faf0 0x7f16c40ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:00.849 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.848+0000 7f16c9200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f16c406be40 con 0x7f16c40adf60 2024-04-20T10:22:00.849 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.848+0000 7f16c9200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f16c40736c0 con 0x7f16c40b2f70 2024-04-20T10:22:00.849 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.848+0000 7f16c9200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f16c406bfc0 con 0x7f16c415faf0 2024-04-20T10:22:00.849 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.848+0000 7f16c3400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16c40adf60 0x7f16c4150c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:00.849 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.848+0000 7f16c3400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16c40adf60 0x7f16c4150c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43056/0 (socket says 172.21.15.121:43056) 2024-04-20T10:22:00.849 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.848+0000 7f16c3400640 1 -- 172.21.15.121:0/241017906 learned_addr learned my addr 172.21.15.121:0/241017906 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:00.851 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.849+0000 7f16c3e00640 1 --2- 172.21.15.121:0/241017906 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16c415faf0 0x7f16c40ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:00.851 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.849+0000 7f16c2a00640 1 --2- 172.21.15.121:0/241017906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16c40b2f70 0x7f16c40c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:00.851 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.849+0000 7f16c3400640 1 -- 172.21.15.121:0/241017906 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16c415faf0 msgr2=0x7f16c40ca2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:00.851 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.849+0000 7f16c3400640 1 --2- 172.21.15.121:0/241017906 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16c415faf0 0x7f16c40ca2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:00.851 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.849+0000 7f16c3400640 1 -- 172.21.15.121:0/241017906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16c40b2f70 msgr2=0x7f16c40c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:00.851 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.849+0000 7f16c3400640 1 --2- 172.21.15.121:0/241017906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16c40b2f70 0x7f16c40c9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:00.851 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.849+0000 7f16c3400640 1 -- 172.21.15.121:0/241017906 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16c415b900 con 0x7f16c40adf60 2024-04-20T10:22:00.851 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.849+0000 7f16c3400640 1 --2- 172.21.15.121:0/241017906 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16c40adf60 0x7f16c4150c40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f16b4007e60 tx=0x7f16b400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:00.851 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.849+0000 7f16bb400640 1 -- 172.21.15.121:0/241017906 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b4014620 con 0x7f16c40adf60 2024-04-20T10:22:00.851 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.849+0000 7f16bb400640 1 -- 172.21.15.121:0/241017906 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16b4019070 con 0x7f16c40adf60 2024-04-20T10:22:00.851 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.849+0000 7f16bb400640 1 -- 172.21.15.121:0/241017906 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b4014a00 con 0x7f16c40adf60 2024-04-20T10:22:00.851 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.850+0000 7f16c9200640 1 -- 172.21.15.121:0/241017906 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16c4151180 con 0x7f16c40adf60 2024-04-20T10:22:00.851 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.850+0000 7f16c9200640 1 -- 172.21.15.121:0/241017906 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16c40cd4d0 con 0x7f16c40adf60 2024-04-20T10:22:00.852 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.851+0000 7f16c9200640 1 -- 172.21.15.121:0/241017906 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16c4075b60 con 0x7f16c40adf60 2024-04-20T10:22:00.852 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.852+0000 7f16bb400640 1 -- 172.21.15.121:0/241017906 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f16b4014ba0 con 0x7f16c40adf60 2024-04-20T10:22:00.853 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.852+0000 7f16bb400640 1 --2- 172.21.15.121:0/241017906 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f16a406c920 0x7f16a406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:00.853 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.853+0000 7f16bb400640 1 -- 172.21.15.121:0/241017906 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f16b408c7a0 con 0x7f16c40adf60 2024-04-20T10:22:00.853 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.853+0000 7f16c2a00640 1 --2- 172.21.15.121:0/241017906 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f16a406c920 0x7f16a406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:00.854 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.853+0000 7f16c2a00640 1 --2- 172.21.15.121:0/241017906 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f16a406c920 0x7f16a406ede0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f16b0036000 tx=0x7f16b003c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:00.857 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:00.857+0000 7f16bb400640 1 -- 172.21.15.121:0/241017906 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f16b405a800 con 0x7f16c40adf60 2024-04-20T10:22:01.061 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.060+0000 7f16c9200640 1 -- 172.21.15.121:0/241017906 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f16c406d220 con 0x7f16c40adf60 2024-04-20T10:22:01.062 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.062+0000 7f16bb400640 1 -- 172.21.15.121:0/241017906 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18940 (secure 0 0 0) 0x7f16b405a100 con 0x7f16c40adf60 2024-04-20T10:22:01.063 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:22:01.063 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":27,"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","created":"2024-04-20T10:20:38.076076+0000","modified":"2024-04-20T10:21:44.715898+0000","last_up_change":"2024-04-20T10:20:50.162700+0000","last_in_change":"2024-04-20T10:20:41.510382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:20:51.141365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:21:00.436937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:21:01.026957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:21:04.862693+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"629a7b8c-a40a-4a2a-a3cc-d9f357774310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6809","nonce":1044588835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6811","nonce":1044588835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6815","nonce":1044588835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6813","nonce":1044588835}]},"public_addr":"172.21.15.121:6809/1044588835","cluster_addr":"172.21.15.121:6811/1044588835","heartbeat_back_addr":"172.21.15.121:6815/1044588835","heartbeat_front_addr":"172.21.15.121:6813/1044588835","state":["exists","up"]},{"osd":1,"uuid":"f2570399-aa50-4ff3-bf5d-3d09207290db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6801","nonce":3505199624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6803","nonce":3505199624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6807","nonce":3505199624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6805","nonce":3505199624}]},"public_addr":"172.21.15.121:6801/3505199624","cluster_addr":"172.21.15.121:6803/3505199624","heartbeat_back_addr":"172.21.15.121:6807/3505199624","heartbeat_front_addr":"172.21.15.121:6805/3505199624","state":["exists","up"]},{"osd":2,"uuid":"50893807-dc30-49f4-b41f-739366626658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6817","nonce":3656724207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6819","nonce":3656724207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6827","nonce":3656724207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6822","nonce":3656724207}]},"public_addr":"172.21.15.121:6817/3656724207","cluster_addr":"172.21.15.121:6819/3656724207","heartbeat_back_addr":"172.21.15.121:6827/3656724207","heartbeat_front_addr":"172.21.15.121:6822/3656724207","state":["exists","up"]},{"osd":3,"uuid":"dadb01bc-5491-4ffc-8824-d48a057240b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6823","nonce":2232342832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6826","nonce":2232342832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6831","nonce":2232342832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6829","nonce":2232342832}]},"public_addr":"172.21.15.121:6823/2232342832","cluster_addr":"172.21.15.121:6826/2232342832","heartbeat_back_addr":"172.21.15.121:6831/2232342832","heartbeat_front_addr":"172.21.15.121:6829/2232342832","state":["exists","up"]},{"osd":4,"uuid":"b3f3b4a1-b375-4526-b69e-344ab24c2da1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6801","nonce":99353111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6803","nonce":99353111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6807","nonce":99353111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6805","nonce":99353111}]},"public_addr":"172.21.15.123:6801/99353111","cluster_addr":"172.21.15.123:6803/99353111","heartbeat_back_addr":"172.21.15.123:6807/99353111","heartbeat_front_addr":"172.21.15.123:6805/99353111","state":["exists","up"]},{"osd":5,"uuid":"2ee24a08-1624-4961-9642-d7b510282484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6818","nonce":2504374688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6822","nonce":2504374688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6830","nonce":2504374688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6826","nonce":2504374688}]},"public_addr":"172.21.15.123:6818/2504374688","cluster_addr":"172.21.15.123:6822/2504374688","heartbeat_back_addr":"172.21.15.123:6830/2504374688","heartbeat_front_addr":"172.21.15.123:6826/2504374688","state":["exists","up"]},{"osd":6,"uuid":"186118cc-6128-49de-acca-55260945c1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6819","nonce":4139299599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6823","nonce":4139299599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6829","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6831","nonce":4139299599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6827","nonce":4139299599}]},"public_addr":"172.21.15.123:6819/4139299599","cluster_addr":"172.21.15.123:6823/4139299599","heartbeat_back_addr":"172.21.15.123:6831/4139299599","heartbeat_front_addr":"172.21.15.123:6827/4139299599","state":["exists","up"]},{"osd":7,"uuid":"a00aac6e-4bd9-415a-a93d-ae85210865ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6809","nonce":4043981069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6811","nonce":4043981069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6815","nonce":4043981069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6813","nonce":4043981069}]},"public_addr":"172.21.15.123:6809/4043981069","cluster_addr":"172.21.15.123:6811/4043981069","heartbeat_back_addr":"172.21.15.123:6815/4043981069","heartbeat_front_addr":"172.21.15.123:6813/4043981069","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.904540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.958585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.616367+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.885244+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.103127+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.167248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.884348+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.862356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.23","mappings":[{"from":3,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:22:01.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.065+0000 7f16c9200640 1 -- 172.21.15.121:0/241017906 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f16a406c920 msgr2=0x7f16a406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:01.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.065+0000 7f16c9200640 1 --2- 172.21.15.121:0/241017906 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f16a406c920 0x7f16a406ede0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f16b0036000 tx=0x7f16b003c040 comp rx=0 tx=0).stop 2024-04-20T10:22:01.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.065+0000 7f16c9200640 1 -- 172.21.15.121:0/241017906 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16c40adf60 msgr2=0x7f16c4150c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:01.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.065+0000 7f16c9200640 1 --2- 172.21.15.121:0/241017906 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16c40adf60 0x7f16c4150c40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f16b4007e60 tx=0x7f16b400e4b0 comp rx=0 tx=0).stop 2024-04-20T10:22:01.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.065+0000 7f16c9200640 1 -- 172.21.15.121:0/241017906 shutdown_connections 2024-04-20T10:22:01.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.065+0000 7f16c9200640 1 --2- 172.21.15.121:0/241017906 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f16c415faf0 0x7f16c40ca2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:01.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.065+0000 7f16c9200640 1 --2- 172.21.15.121:0/241017906 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f16a406c920 0x7f16a406ede0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:01.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.065+0000 7f16c9200640 1 --2- 172.21.15.121:0/241017906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16c40b2f70 0x7f16c40c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:01.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.065+0000 7f16c9200640 1 --2- 172.21.15.121:0/241017906 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f16c40adf60 0x7f16c4150c40 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:01.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.065+0000 7f16c9200640 1 -- 172.21.15.121:0/241017906 >> 172.21.15.121:0/241017906 conn(0x7f16c40d0040 msgr2=0x7f16c409d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:01.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.065+0000 7f16c9200640 1 -- 172.21.15.121:0/241017906 shutdown_connections 2024-04-20T10:22:01.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.065+0000 7f16c9200640 1 -- 172.21.15.121:0/241017906 wait complete. 2024-04-20T10:22:01.086 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:20:51.141365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:21:00.436937+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:21:01.026957+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T10:21:04.862693+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T10:22:01.087 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:22:01.276 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.275+0000 7fe53fa00640 1 Processor -- start 2024-04-20T10:22:01.276 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.276+0000 7fe53fa00640 1 -- start start 2024-04-20T10:22:01.278 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.277+0000 7fe53fa00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe53815b3b0 0x7fe53806eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:01.278 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.277+0000 7fe53fa00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe53806dcb0 0x7fe538071b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:01.278 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.277+0000 7fe53fa00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe538072090 0x7fe5380d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:01.278 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.277+0000 7fe53fa00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fe5380ade40 con 0x7fe538072090 2024-04-20T10:22:01.278 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.277+0000 7fe53fa00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fe5380ae0e0 con 0x7fe53815b3b0 2024-04-20T10:22:01.278 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.277+0000 7fe53fa00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe53815b790 con 0x7fe53806dcb0 2024-04-20T10:22:01.278 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.277+0000 7fe53da00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe538072090 0x7fe5380d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:01.278 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.278+0000 7fe53da00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe538072090 0x7fe5380d0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43068/0 (socket says 172.21.15.121:43068) 2024-04-20T10:22:01.278 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.278+0000 7fe53da00640 1 -- 172.21.15.121:0/1355043700 learned_addr learned my addr 172.21.15.121:0/1355043700 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:01.279 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.278+0000 7fe53d000640 1 --2- 172.21.15.121:0/1355043700 >> v2:172.21.15.123:3300/0 conn(0x7fe53815b3b0 0x7fe53806eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:01.279 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.278+0000 7fe537e00640 1 --2- 172.21.15.121:0/1355043700 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe53806dcb0 0x7fe538071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:01.279 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.278+0000 7fe53da00640 1 -- 172.21.15.121:0/1355043700 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe53806dcb0 msgr2=0x7fe538071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:01.280 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.278+0000 7fe53da00640 1 --2- 172.21.15.121:0/1355043700 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe53806dcb0 0x7fe538071b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:01.280 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.278+0000 7fe53da00640 1 -- 172.21.15.121:0/1355043700 >> v2:172.21.15.123:3300/0 conn(0x7fe53815b3b0 msgr2=0x7fe53806eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:01.280 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.278+0000 7fe53da00640 1 --2- 172.21.15.121:0/1355043700 >> v2:172.21.15.123:3300/0 conn(0x7fe53815b3b0 0x7fe53806eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:01.280 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.278+0000 7fe53da00640 1 -- 172.21.15.121:0/1355043700 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5380ae380 con 0x7fe538072090 2024-04-20T10:22:01.280 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.278+0000 7fe537e00640 1 --2- 172.21.15.121:0/1355043700 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe53806dcb0 0x7fe538071b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:01.281 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.279+0000 7fe53da00640 1 --2- 172.21.15.121:0/1355043700 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe538072090 0x7fe5380d0360 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe53806cfa0 tx=0x7fe53002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4402035a304d525f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:01.281 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.280+0000 7fe537400640 1 -- 172.21.15.121:0/1355043700 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53002ca70 con 0x7fe538072090 2024-04-20T10:22:01.281 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.280+0000 7fe537400640 1 -- 172.21.15.121:0/1355043700 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe53002cc10 con 0x7fe538072090 2024-04-20T10:22:01.281 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.280+0000 7fe537400640 1 -- 172.21.15.121:0/1355043700 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5300343f0 con 0x7fe538072090 2024-04-20T10:22:01.281 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.280+0000 7fe53fa00640 1 -- 172.21.15.121:0/1355043700 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe538072090 msgr2=0x7fe5380d0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:01.281 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.280+0000 7fe53fa00640 1 --2- 172.21.15.121:0/1355043700 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe538072090 0x7fe5380d0360 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe53806cfa0 tx=0x7fe53002bed0 comp rx=0 tx=0).stop 2024-04-20T10:22:01.282 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.280+0000 7fe53fa00640 1 -- 172.21.15.121:0/1355043700 shutdown_connections 2024-04-20T10:22:01.282 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.281+0000 7fe53fa00640 1 --2- 172.21.15.121:0/1355043700 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe538072090 0x7fe5380d0360 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:01.282 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.281+0000 7fe53fa00640 1 --2- 172.21.15.121:0/1355043700 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe53806dcb0 0x7fe538071b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:01.282 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.281+0000 7fe53fa00640 1 --2- 172.21.15.121:0/1355043700 >> v2:172.21.15.123:3300/0 conn(0x7fe53815b3b0 0x7fe53806eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:01.282 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.281+0000 7fe53fa00640 1 -- 172.21.15.121:0/1355043700 >> 172.21.15.121:0/1355043700 conn(0x7fe5380b3350 msgr2=0x7fe5380b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:01.282 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.281+0000 7fe53fa00640 1 -- 172.21.15.121:0/1355043700 shutdown_connections 2024-04-20T10:22:01.282 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.281+0000 7fe53fa00640 1 -- 172.21.15.121:0/1355043700 wait complete. 2024-04-20T10:22:01.283 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.283+0000 7fe53fa00640 1 Processor -- start 2024-04-20T10:22:01.283 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.283+0000 7fe53fa00640 1 -- start start 2024-04-20T10:22:01.286 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.284+0000 7fe53fa00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe53806dcb0 0x7fe5381f4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:01.287 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.284+0000 7fe53fa00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe53815b3b0 0x7fe5380af550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:01.287 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.284+0000 7fe53fa00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe538153600 0x7fe538159e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:01.287 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.284+0000 7fe53fa00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe5380aec20 con 0x7fe538153600 2024-04-20T10:22:01.287 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.284+0000 7fe53fa00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe5380ae940 con 0x7fe53815b3b0 2024-04-20T10:22:01.287 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.284+0000 7fe53fa00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe53806efe0 con 0x7fe53806dcb0 2024-04-20T10:22:01.287 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.285+0000 7fe53da00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe538153600 0x7fe538159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:01.287 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.285+0000 7fe53da00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe538153600 0x7fe538159e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43082/0 (socket says 172.21.15.121:43082) 2024-04-20T10:22:01.287 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.285+0000 7fe53da00640 1 -- 172.21.15.121:0/2497572630 learned_addr learned my addr 172.21.15.121:0/2497572630 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:01.287 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.285+0000 7fe53da00640 1 -- 172.21.15.121:0/2497572630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe53806dcb0 msgr2=0x7fe5381f4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:01.287 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.285+0000 7fe53da00640 1 --2- 172.21.15.121:0/2497572630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe53806dcb0 0x7fe5381f4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:01.287 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.285+0000 7fe53da00640 1 -- 172.21.15.121:0/2497572630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe53815b3b0 msgr2=0x7fe5380af550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:01.287 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.285+0000 7fe537e00640 1 --2- 172.21.15.121:0/2497572630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe53815b3b0 0x7fe5380af550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:01.287 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.285+0000 7fe53da00640 1 --2- 172.21.15.121:0/2497572630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe53815b3b0 0x7fe5380af550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:01.287 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.285+0000 7fe53da00640 1 -- 172.21.15.121:0/2497572630 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5380afa90 con 0x7fe538153600 2024-04-20T10:22:01.288 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.285+0000 7fe53da00640 1 --2- 172.21.15.121:0/2497572630 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe538153600 0x7fe538159e50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe530009f40 tx=0x7fe530004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:01.288 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.286+0000 7fe534c00640 1 -- 172.21.15.121:0/2497572630 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530043020 con 0x7fe538153600 2024-04-20T10:22:01.288 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.286+0000 7fe534c00640 1 -- 172.21.15.121:0/2497572630 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe53003e450 con 0x7fe538153600 2024-04-20T10:22:01.288 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.286+0000 7fe53fa00640 1 -- 172.21.15.121:0/2497572630 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe53815e0b0 con 0x7fe538153600 2024-04-20T10:22:01.288 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.286+0000 7fe534c00640 1 -- 172.21.15.121:0/2497572630 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530009510 con 0x7fe538153600 2024-04-20T10:22:01.288 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.286+0000 7fe53fa00640 1 -- 172.21.15.121:0/2497572630 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe53815e670 con 0x7fe538153600 2024-04-20T10:22:01.288 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.287+0000 7fe53fa00640 1 -- 172.21.15.121:0/2497572630 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe538068980 con 0x7fe538153600 2024-04-20T10:22:01.288 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.288+0000 7fe534c00640 1 -- 172.21.15.121:0/2497572630 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe530046020 con 0x7fe538153600 2024-04-20T10:22:01.289 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.289+0000 7fe534c00640 1 --2- 172.21.15.121:0/2497572630 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe51806c8f0 0x7fe51806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:01.289 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.289+0000 7fe534c00640 1 -- 172.21.15.121:0/2497572630 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fe5300ae710 con 0x7fe538153600 2024-04-20T10:22:01.290 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.289+0000 7fe53d000640 1 --2- 172.21.15.121:0/2497572630 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe51806c8f0 0x7fe51806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:01.290 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.290+0000 7fe53d000640 1 --2- 172.21.15.121:0/2497572630 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe51806c8f0 0x7fe51806edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe52c0098f0 tx=0x7fe52c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:01.294 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.293+0000 7fe534c00640 1 -- 172.21.15.121:0/2497572630 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe53004d030 con 0x7fe538153600 2024-04-20T10:22:01.493 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.492+0000 7fe53fa00640 1 -- 172.21.15.121:0/2497572630 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe53806e340 con 0x7fe538153600 2024-04-20T10:22:01.494 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.493+0000 7fe534c00640 1 -- 172.21.15.121:0/2497572630 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fe53007c730 con 0x7fe538153600 2024-04-20T10:22:01.494 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2024-04-20T10:22:01.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.496+0000 7fe53fa00640 1 -- 172.21.15.121:0/2497572630 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe51806c8f0 msgr2=0x7fe51806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:01.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.496+0000 7fe53fa00640 1 --2- 172.21.15.121:0/2497572630 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe51806c8f0 0x7fe51806edb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe52c0098f0 tx=0x7fe52c008040 comp rx=0 tx=0).stop 2024-04-20T10:22:01.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.496+0000 7fe53fa00640 1 -- 172.21.15.121:0/2497572630 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe538153600 msgr2=0x7fe538159e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:01.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.496+0000 7fe53fa00640 1 --2- 172.21.15.121:0/2497572630 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe538153600 0x7fe538159e50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe530009f40 tx=0x7fe530004a40 comp rx=0 tx=0).stop 2024-04-20T10:22:01.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.496+0000 7fe53fa00640 1 -- 172.21.15.121:0/2497572630 shutdown_connections 2024-04-20T10:22:01.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.496+0000 7fe53fa00640 1 --2- 172.21.15.121:0/2497572630 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe538153600 0x7fe538159e50 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:01.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.496+0000 7fe53fa00640 1 --2- 172.21.15.121:0/2497572630 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe51806c8f0 0x7fe51806edb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:01.498 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.496+0000 7fe53fa00640 1 --2- 172.21.15.121:0/2497572630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe53815b3b0 0x7fe5380af550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:01.498 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.496+0000 7fe53fa00640 1 --2- 172.21.15.121:0/2497572630 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe53806dcb0 0x7fe5381f4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:01.498 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.496+0000 7fe53fa00640 1 -- 172.21.15.121:0/2497572630 >> 172.21.15.121:0/2497572630 conn(0x7fe5380b3350 msgr2=0x7fe53815ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:01.498 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.496+0000 7fe53fa00640 1 -- 172.21.15.121:0/2497572630 shutdown_connections 2024-04-20T10:22:01.498 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.497+0000 7fe53fa00640 1 -- 172.21.15.121:0/2497572630 wait complete. 2024-04-20T10:22:01.516 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:22:01.719 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.718+0000 7facf6200640 1 Processor -- start 2024-04-20T10:22:01.719 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.719+0000 7facf6200640 1 -- start start 2024-04-20T10:22:01.721 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.720+0000 7facf6200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7facf0156c20 0x7facf006b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:01.721 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.720+0000 7facf6200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facf015ca40 0x7facf00ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:01.722 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.720+0000 7facf6200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facf0141c60 con 0x7facf00ad8e0 2024-04-20T10:22:01.722 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.720+0000 7facf6200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7facf015cf90 con 0x7facf0156c20 2024-04-20T10:22:01.722 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.720+0000 7facf6200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7facf015d270 con 0x7facf015ca40 2024-04-20T10:22:01.722 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.720+0000 7facef400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7facf0156c20 0x7facf006b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:01.722 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.721+0000 7facefe00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7facf00ad8e0 0x7facf00d0360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:40504/0 (socket says 172.21.15.121:40504) 2024-04-20T10:22:01.723 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.721+0000 7facefe00640 1 -- 172.21.15.121:0/1750716191 learned_addr learned my addr 172.21.15.121:0/1750716191 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:01.723 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.721+0000 7faceea00640 1 --2- 172.21.15.121:0/1750716191 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facf015ca40 0x7facf00ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:01.723 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.721+0000 7facef400640 1 -- 172.21.15.121:0/1750716191 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facf015ca40 msgr2=0x7facf00ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:01.723 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.721+0000 7facef400640 1 --2- 172.21.15.121:0/1750716191 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facf015ca40 0x7facf00ad3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:01.723 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.721+0000 7facef400640 1 -- 172.21.15.121:0/1750716191 >> v1:172.21.15.121:6789/0 conn(0x7facf00ad8e0 legacy=0x7facf00d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:01.723 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.721+0000 7facee000640 1 -- 172.21.15.121:0/1750716191 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1070788305 0 0) 0x7facf0141c60 con 0x7facf00ad8e0 2024-04-20T10:22:01.723 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.721+0000 7facef400640 1 -- 172.21.15.121:0/1750716191 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf015d550 con 0x7facf0156c20 2024-04-20T10:22:01.725 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.722+0000 7facef400640 1 --2- 172.21.15.121:0/1750716191 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7facf0156c20 0x7facf006b900 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7facd80097a0 tx=0x7facd802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca1565ac5d3a0924 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:01.725 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.723+0000 7facee000640 1 -- 172.21.15.121:0/1750716191 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd8035020 con 0x7facf0156c20 2024-04-20T10:22:01.725 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.723+0000 7facee000640 1 -- 172.21.15.121:0/1750716191 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facd8031830 con 0x7facf0156c20 2024-04-20T10:22:01.725 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.723+0000 7facee000640 1 -- 172.21.15.121:0/1750716191 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd8031c00 con 0x7facf0156c20 2024-04-20T10:22:01.725 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.723+0000 7facf6200640 1 -- 172.21.15.121:0/1750716191 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7facf0156c20 msgr2=0x7facf006b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:01.725 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.723+0000 7facf6200640 1 --2- 172.21.15.121:0/1750716191 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7facf0156c20 0x7facf006b900 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7facd80097a0 tx=0x7facd802b8d0 comp rx=0 tx=0).stop 2024-04-20T10:22:01.725 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.723+0000 7facf6200640 1 -- 172.21.15.121:0/1750716191 shutdown_connections 2024-04-20T10:22:01.726 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.723+0000 7facf6200640 1 --2- 172.21.15.121:0/1750716191 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facf015ca40 0x7facf00ad3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:01.726 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.723+0000 7facf6200640 1 --2- 172.21.15.121:0/1750716191 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7facf0156c20 0x7facf006b900 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:01.726 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.723+0000 7facf6200640 1 -- 172.21.15.121:0/1750716191 >> 172.21.15.121:0/1750716191 conn(0x7facf00b3350 msgr2=0x7facf00b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:01.726 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.724+0000 7facf6200640 1 -- 172.21.15.121:0/1750716191 shutdown_connections 2024-04-20T10:22:01.726 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.724+0000 7facf6200640 1 -- 172.21.15.121:0/1750716191 wait complete. 2024-04-20T10:22:01.729 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.725+0000 7facf6200640 1 Processor -- start 2024-04-20T10:22:01.729 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.725+0000 7facf6200640 1 -- start start 2024-04-20T10:22:01.729 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.726+0000 7facf6200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facf00ad8e0 0x7facf00730a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:01.729 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.726+0000 7facf6200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7facf0156c20 0x7facf00735e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:01.729 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.726+0000 7facf6200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facf015ca40 0x7facf0150850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:01.729 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.726+0000 7facf6200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7facf006bfc0 con 0x7facf0156c20 2024-04-20T10:22:01.729 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.726+0000 7facf6200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7facf006be40 con 0x7facf00ad8e0 2024-04-20T10:22:01.729 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.726+0000 7facf6200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7facf00d0a70 con 0x7facf015ca40 2024-04-20T10:22:01.729 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.727+0000 7faceea00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7facf0156c20 0x7facf00735e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:01.729 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.727+0000 7faceea00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7facf0156c20 0x7facf00735e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43100/0 (socket says 172.21.15.121:43100) 2024-04-20T10:22:01.729 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.727+0000 7faceea00640 1 -- 172.21.15.121:0/2279351356 learned_addr learned my addr 172.21.15.121:0/2279351356 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:01.729 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.727+0000 7facefe00640 1 --2- 172.21.15.121:0/2279351356 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facf015ca40 0x7facf0150850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:01.730 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.727+0000 7facef400640 1 --2- 172.21.15.121:0/2279351356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facf00ad8e0 0x7facf00730a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:01.730 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.727+0000 7faceea00640 1 -- 172.21.15.121:0/2279351356 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facf015ca40 msgr2=0x7facf0150850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:01.730 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.727+0000 7faceea00640 1 --2- 172.21.15.121:0/2279351356 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facf015ca40 0x7facf0150850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:01.730 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.727+0000 7faceea00640 1 -- 172.21.15.121:0/2279351356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facf00ad8e0 msgr2=0x7facf00730a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:01.730 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.727+0000 7faceea00640 1 --2- 172.21.15.121:0/2279351356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facf00ad8e0 0x7facf00730a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:01.730 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.727+0000 7faceea00640 1 -- 172.21.15.121:0/2279351356 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf00b1920 con 0x7facf0156c20 2024-04-20T10:22:01.730 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.727+0000 7facef400640 1 --2- 172.21.15.121:0/2279351356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facf00ad8e0 0x7facf00730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:01.730 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.727+0000 7faceea00640 1 --2- 172.21.15.121:0/2279351356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7facf0156c20 0x7facf00735e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7face0007e60 tx=0x7face000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:01.730 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.728+0000 7face7400640 1 -- 172.21.15.121:0/2279351356 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face0014620 con 0x7facf0156c20 2024-04-20T10:22:01.730 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.728+0000 7face7400640 1 -- 172.21.15.121:0/2279351356 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7face0019070 con 0x7facf0156c20 2024-04-20T10:22:01.730 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.728+0000 7face7400640 1 -- 172.21.15.121:0/2279351356 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face0014a20 con 0x7facf0156c20 2024-04-20T10:22:01.730 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.728+0000 7facf6200640 1 -- 172.21.15.121:0/2279351356 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facf0160820 con 0x7facf0156c20 2024-04-20T10:22:01.730 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.728+0000 7facf6200640 1 -- 172.21.15.121:0/2279351356 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facf0072600 con 0x7facf0156c20 2024-04-20T10:22:01.731 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.729+0000 7facf6200640 1 -- 172.21.15.121:0/2279351356 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facf0068980 con 0x7facf0156c20 2024-04-20T10:22:01.731 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.730+0000 7face7400640 1 -- 172.21.15.121:0/2279351356 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7face0014bc0 con 0x7facf0156c20 2024-04-20T10:22:01.731 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.731+0000 7face7400640 1 --2- 172.21.15.121:0/2279351356 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7faccc06c8f0 0x7faccc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:01.732 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.731+0000 7face7400640 1 -- 172.21.15.121:0/2279351356 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7face008c870 con 0x7facf0156c20 2024-04-20T10:22:01.732 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.731+0000 7facef400640 1 --2- 172.21.15.121:0/2279351356 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7faccc06c8f0 0x7faccc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:01.732 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.732+0000 7facef400640 1 --2- 172.21.15.121:0/2279351356 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7faccc06c8f0 0x7faccc06edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7facd8036000 tx=0x7facd8008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:01.736 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.735+0000 7face7400640 1 -- 172.21.15.121:0/2279351356 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7face005a890 con 0x7facf0156c20 2024-04-20T10:22:01.934 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.934+0000 7facf6200640 1 -- 172.21.15.121:0/2279351356 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7facf00cfd90 con 0x7facf0156c20 2024-04-20T10:22:01.935 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.935+0000 7face7400640 1 -- 172.21.15.121:0/2279351356 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7face005a190 con 0x7facf0156c20 2024-04-20T10:22:01.935 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 32 2024-04-20T10:22:01.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.937+0000 7facf6200640 1 -- 172.21.15.121:0/2279351356 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7faccc06c8f0 msgr2=0x7faccc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:01.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.937+0000 7facf6200640 1 --2- 172.21.15.121:0/2279351356 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7faccc06c8f0 0x7faccc06edb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7facd8036000 tx=0x7facd8008030 comp rx=0 tx=0).stop 2024-04-20T10:22:01.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.937+0000 7facf6200640 1 -- 172.21.15.121:0/2279351356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7facf0156c20 msgr2=0x7facf00735e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:01.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.937+0000 7facf6200640 1 --2- 172.21.15.121:0/2279351356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7facf0156c20 0x7facf00735e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7face0007e60 tx=0x7face000e4b0 comp rx=0 tx=0).stop 2024-04-20T10:22:01.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.937+0000 7facf6200640 1 -- 172.21.15.121:0/2279351356 shutdown_connections 2024-04-20T10:22:01.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.937+0000 7facf6200640 1 --2- 172.21.15.121:0/2279351356 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7facf015ca40 0x7facf0150850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:01.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.937+0000 7facf6200640 1 --2- 172.21.15.121:0/2279351356 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7facf0156c20 0x7facf00735e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:01.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.937+0000 7facf6200640 1 --2- 172.21.15.121:0/2279351356 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7faccc06c8f0 0x7faccc06edb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:01.939 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.937+0000 7facf6200640 1 --2- 172.21.15.121:0/2279351356 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7facf00ad8e0 0x7facf00730a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:01.939 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.937+0000 7facf6200640 1 -- 172.21.15.121:0/2279351356 >> 172.21.15.121:0/2279351356 conn(0x7facf00b3350 msgr2=0x7facf015db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:01.939 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.938+0000 7facf6200640 1 -- 172.21.15.121:0/2279351356 shutdown_connections 2024-04-20T10:22:01.939 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:01.938+0000 7facf6200640 1 -- 172.21.15.121:0/2279351356 wait complete. 2024-04-20T10:22:01.957 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:22:02.154 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.153+0000 7f10cd600640 1 Processor -- start 2024-04-20T10:22:02.154 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.153+0000 7f10cd600640 1 -- start start 2024-04-20T10:22:02.156 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.155+0000 7f10cd600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f10c8071a60 0x7f10c8167390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:02.156 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.155+0000 7f10cd600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10c815cc20 con 0x7f10c8073b80 2024-04-20T10:22:02.156 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.155+0000 7f10cd600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10c815cec0 con 0x7f10c815a130 2024-04-20T10:22:02.156 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.155+0000 7f10cd600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f10c815d7f0 con 0x7f10c8071a60 2024-04-20T10:22:02.156 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.155+0000 7f10ccc00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f10c8071a60 0x7f10c8167390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:02.157 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.155+0000 7f10c7400640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f10c8073b80 0x7f10c806f160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:40516/0 (socket says 172.21.15.121:40516) 2024-04-20T10:22:02.157 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.155+0000 7f10c7400640 1 -- 172.21.15.121:0/573919044 learned_addr learned my addr 172.21.15.121:0/573919044 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:02.157 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.156+0000 7f10ccc00640 1 -- 172.21.15.121:0/573919044 >> v1:172.21.15.123:6789/0 conn(0x7f10c815a130 legacy=0x7f10c8073470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:02.157 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.156+0000 7f10ccc00640 1 -- 172.21.15.121:0/573919044 >> v1:172.21.15.121:6789/0 conn(0x7f10c8073b80 legacy=0x7f10c806f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:02.157 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.156+0000 7f10ccc00640 1 -- 172.21.15.121:0/573919044 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c815d1a0 con 0x7f10c8071a60 2024-04-20T10:22:02.157 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.157+0000 7f10ccc00640 1 --2- 172.21.15.121:0/573919044 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10c8071a60 0x7f10c8167390 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f10c0009d90 tx=0x7f10c002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e6398303fac36f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:02.157 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.157+0000 7f10c6a00640 1 -- 172.21.15.121:0/573919044 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c0039070 con 0x7f10c8071a60 2024-04-20T10:22:02.158 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.157+0000 7f10c6a00640 1 -- 172.21.15.121:0/573919044 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10c002cbd0 con 0x7f10c8071a60 2024-04-20T10:22:02.158 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.157+0000 7f10c6a00640 1 -- 172.21.15.121:0/573919044 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c0034440 con 0x7f10c8071a60 2024-04-20T10:22:02.158 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.158+0000 7f10cd600640 1 -- 172.21.15.121:0/573919044 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10c8071a60 msgr2=0x7f10c8167390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:02.158 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.158+0000 7f10cd600640 1 --2- 172.21.15.121:0/573919044 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10c8071a60 0x7f10c8167390 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f10c0009d90 tx=0x7f10c002be20 comp rx=0 tx=0).stop 2024-04-20T10:22:02.159 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.158+0000 7f10cd600640 1 -- 172.21.15.121:0/573919044 shutdown_connections 2024-04-20T10:22:02.159 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.158+0000 7f10cd600640 1 --2- 172.21.15.121:0/573919044 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10c8071a60 0x7f10c8167390 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:02.159 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.158+0000 7f10cd600640 1 -- 172.21.15.121:0/573919044 >> 172.21.15.121:0/573919044 conn(0x7f10c80cffa0 msgr2=0x7f10c80d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:02.159 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.158+0000 7f10cd600640 1 -- 172.21.15.121:0/573919044 shutdown_connections 2024-04-20T10:22:02.159 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.158+0000 7f10cd600640 1 -- 172.21.15.121:0/573919044 wait complete. 2024-04-20T10:22:02.160 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.160+0000 7f10cd600640 1 Processor -- start 2024-04-20T10:22:02.160 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.160+0000 7f10cd600640 1 -- start start 2024-04-20T10:22:02.164 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.161+0000 7f10cd600640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10c8071a60 0x7f10c8150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:02.164 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.161+0000 7f10cd600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10c8073b80 0x7f10c80b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:02.164 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.161+0000 7f10cd600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10c815a130 0x7f10c80b50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:02.164 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.161+0000 7f10cd600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f10c806f9f0 con 0x7f10c815a130 2024-04-20T10:22:02.164 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.161+0000 7f10cd600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f10c806f870 con 0x7f10c8073b80 2024-04-20T10:22:02.165 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.161+0000 7f10cd600640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f10c806fb70 con 0x7f10c8071a60 2024-04-20T10:22:02.166 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.161+0000 7f10ccc00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10c815a130 0x7f10c80b50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:02.166 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.161+0000 7f10ccc00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10c815a130 0x7f10c80b50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43116/0 (socket says 172.21.15.121:43116) 2024-04-20T10:22:02.166 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.161+0000 7f10ccc00640 1 -- 172.21.15.121:0/1335420272 learned_addr learned my addr 172.21.15.121:0/1335420272 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:02.167 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.161+0000 7f10ccc00640 1 -- 172.21.15.121:0/1335420272 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10c8071a60 msgr2=0x7f10c8150a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:02.167 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.161+0000 7f10ccc00640 1 --2- 172.21.15.121:0/1335420272 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10c8071a60 0x7f10c8150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:02.167 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.161+0000 7f10c7400640 1 --2- 172.21.15.121:0/1335420272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10c8073b80 0x7f10c80b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:02.167 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.161+0000 7f10ccc00640 1 -- 172.21.15.121:0/1335420272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10c8073b80 msgr2=0x7f10c80b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:02.167 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.161+0000 7f10ccc00640 1 --2- 172.21.15.121:0/1335420272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10c8073b80 0x7f10c80b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:02.167 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.161+0000 7f10ccc00640 1 -- 172.21.15.121:0/1335420272 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c80cb980 con 0x7f10c815a130 2024-04-20T10:22:02.167 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.161+0000 7f10c7400640 1 --2- 172.21.15.121:0/1335420272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10c8073b80 0x7f10c80b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:02.167 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.162+0000 7f10ccc00640 1 --2- 172.21.15.121:0/1335420272 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10c815a130 0x7f10c80b50f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f10c0009d90 tx=0x7f10c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:02.167 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.162+0000 7f10bfe00640 1 -- 172.21.15.121:0/1335420272 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c0039050 con 0x7f10c815a130 2024-04-20T10:22:02.167 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.162+0000 7f10cd600640 1 -- 172.21.15.121:0/1335420272 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10c80cbc60 con 0x7f10c815a130 2024-04-20T10:22:02.167 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.162+0000 7f10bfe00640 1 -- 172.21.15.121:0/1335420272 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10c0045070 con 0x7f10c815a130 2024-04-20T10:22:02.167 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.162+0000 7f10bfe00640 1 -- 172.21.15.121:0/1335420272 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c003e830 con 0x7f10c815a130 2024-04-20T10:22:02.167 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.162+0000 7f10cd600640 1 -- 172.21.15.121:0/1335420272 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10c80cb3c0 con 0x7f10c815a130 2024-04-20T10:22:02.168 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.163+0000 7f10cd600640 1 -- 172.21.15.121:0/1335420272 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10c8075b30 con 0x7f10c815a130 2024-04-20T10:22:02.168 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.164+0000 7f10bfe00640 1 -- 172.21.15.121:0/1335420272 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f10c003e9d0 con 0x7f10c815a130 2024-04-20T10:22:02.169 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.165+0000 7f10bfe00640 1 --2- 172.21.15.121:0/1335420272 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f10a806c8f0 0x7f10a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:02.169 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.166+0000 7f10bfe00640 1 -- 172.21.15.121:0/1335420272 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f10c00aedf0 con 0x7f10c815a130 2024-04-20T10:22:02.169 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.166+0000 7f10c7e00640 1 --2- 172.21.15.121:0/1335420272 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f10a806c8f0 0x7f10a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:02.169 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.166+0000 7f10c7e00640 1 --2- 172.21.15.121:0/1335420272 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f10a806c8f0 0x7f10a806edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f10b8006fd0 tx=0x7f10b8006bc0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:02.170 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.170+0000 7f10bfe00640 1 -- 172.21.15.121:0/1335420272 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f10c007ceb0 con 0x7f10c815a130 2024-04-20T10:22:02.369 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.368+0000 7f10cd600640 1 -- 172.21.15.121:0/1335420272 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f10c806d010 con 0x7f10c815a130 2024-04-20T10:22:02.370 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.370+0000 7f10bfe00640 1 -- 172.21.15.121:0/1335420272 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f10c007c7b0 con 0x7f10c815a130 2024-04-20T10:22:02.370 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2024-04-20T10:22:02.373 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.372+0000 7f10cd600640 1 -- 172.21.15.121:0/1335420272 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f10a806c8f0 msgr2=0x7f10a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:02.373 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.372+0000 7f10cd600640 1 --2- 172.21.15.121:0/1335420272 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f10a806c8f0 0x7f10a806edb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f10b8006fd0 tx=0x7f10b8006bc0 comp rx=0 tx=0).stop 2024-04-20T10:22:02.373 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.372+0000 7f10cd600640 1 -- 172.21.15.121:0/1335420272 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10c815a130 msgr2=0x7f10c80b50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:02.373 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.372+0000 7f10cd600640 1 --2- 172.21.15.121:0/1335420272 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10c815a130 0x7f10c80b50f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f10c0009d90 tx=0x7f10c0004a40 comp rx=0 tx=0).stop 2024-04-20T10:22:02.373 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.372+0000 7f10cd600640 1 -- 172.21.15.121:0/1335420272 shutdown_connections 2024-04-20T10:22:02.373 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.372+0000 7f10cd600640 1 --2- 172.21.15.121:0/1335420272 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f10c815a130 0x7f10c80b50f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:02.373 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.372+0000 7f10cd600640 1 --2- 172.21.15.121:0/1335420272 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f10a806c8f0 0x7f10a806edb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:02.373 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.372+0000 7f10cd600640 1 --2- 172.21.15.121:0/1335420272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10c8073b80 0x7f10c80b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:02.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.372+0000 7f10cd600640 1 --2- 172.21.15.121:0/1335420272 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f10c8071a60 0x7f10c8150a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:02.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.372+0000 7f10cd600640 1 -- 172.21.15.121:0/1335420272 >> 172.21.15.121:0/1335420272 conn(0x7f10c80cffa0 msgr2=0x7f10c815e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:02.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.372+0000 7f10cd600640 1 -- 172.21.15.121:0/1335420272 shutdown_connections 2024-04-20T10:22:02.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.372+0000 7f10cd600640 1 -- 172.21.15.121:0/1335420272 wait complete. 2024-04-20T10:22:02.391 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T10:22:02.595 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.594+0000 7ff354a00640 1 Processor -- start 2024-04-20T10:22:02.595 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.594+0000 7ff354a00640 1 -- start start 2024-04-20T10:22:02.596 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.596+0000 7ff354a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff3500d3b20 0x7ff3500b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:02.597 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.596+0000 7ff354a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3500d0980 con 0x7ff350161a50 2024-04-20T10:22:02.597 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.596+0000 7ff354a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff350162490 con 0x7ff35006c010 2024-04-20T10:22:02.597 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.596+0000 7ff354a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ff3500d0c60 con 0x7ff3500d3b20 2024-04-20T10:22:02.597 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.596+0000 7ff34e800640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7ff350161a50 0x7ff35006b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:40526/0 (socket says 172.21.15.121:40526) 2024-04-20T10:22:02.597 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.596+0000 7ff34e800640 1 -- 172.21.15.121:0/2185806462 learned_addr learned my addr 172.21.15.121:0/2185806462 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:02.599 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.597+0000 7ff34de00640 1 --2- 172.21.15.121:0/2185806462 >> v2:172.21.15.123:3300/0 conn(0x7ff3500d3b20 0x7ff3500b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:02.599 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.597+0000 7ff34d400640 1 -- 172.21.15.121:0/2185806462 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2845066282 0 0) 0x7ff3500d0980 con 0x7ff350161a50 2024-04-20T10:22:02.599 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.597+0000 7ff34d400640 1 -- 172.21.15.121:0/2185806462 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff334003520 con 0x7ff350161a50 2024-04-20T10:22:02.599 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.597+0000 7ff34d400640 1 -- 172.21.15.121:0/2185806462 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 105341803 0 0) 0x7ff334003520 con 0x7ff350161a50 2024-04-20T10:22:02.599 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.597+0000 7ff34d400640 1 -- 172.21.15.121:0/2185806462 >> v2:172.21.15.123:3300/0 conn(0x7ff3500d3b20 msgr2=0x7ff3500b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:02.600 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.597+0000 7ff34d400640 1 --2- 172.21.15.121:0/2185806462 >> v2:172.21.15.123:3300/0 conn(0x7ff3500d3b20 0x7ff3500b3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:02.600 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.597+0000 7ff34d400640 1 -- 172.21.15.121:0/2185806462 >> v1:172.21.15.123:6789/0 conn(0x7ff35006c010 legacy=0x7ff3500ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:02.600 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.597+0000 7ff34d400640 1 -- 172.21.15.121:0/2185806462 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff350162770 con 0x7ff350161a50 2024-04-20T10:22:02.600 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.597+0000 7ff34d400640 1 -- 172.21.15.121:0/2185806462 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7ff3400019f0 con 0x7ff350161a50 2024-04-20T10:22:02.600 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.597+0000 7ff34d400640 1 -- 172.21.15.121:0/2185806462 >> v1:172.21.15.121:6789/0 conn(0x7ff350161a50 legacy=0x7ff35006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:02.600 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.597+0000 7ff34d400640 1 --2- 172.21.15.121:0/2185806462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3340039b0 0x7ff334003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:02.600 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.597+0000 7ff34d400640 1 --2- 172.21.15.121:0/2185806462 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff334004800 0x7ff334004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:02.600 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.597+0000 7ff34d400640 1 --2- 172.21.15.121:0/2185806462 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff3340051c0 0x7ff33400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:02.600 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.597+0000 7ff34d400640 1 -- 172.21.15.121:0/2185806462 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff3500d0c60 con 0x7ff334004800 2024-04-20T10:22:02.601 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.597+0000 7ff34d400640 1 -- 172.21.15.121:0/2185806462 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff33400c090 con 0x7ff3340039b0 2024-04-20T10:22:02.601 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.597+0000 7ff34d400640 1 -- 172.21.15.121:0/2185806462 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ff33400c290 con 0x7ff3340051c0 2024-04-20T10:22:02.601 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.598+0000 7ff34de00640 1 --2- 172.21.15.121:0/2185806462 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff334004800 0x7ff334004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:02.601 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.598+0000 7ff34e800640 1 --2- 172.21.15.121:0/2185806462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3340039b0 0x7ff334003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:02.601 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.598+0000 7ff34de00640 1 -- 172.21.15.121:0/2185806462 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff3340051c0 msgr2=0x7ff33400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:02.601 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.598+0000 7ff34de00640 1 --2- 172.21.15.121:0/2185806462 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff3340051c0 0x7ff33400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:02.601 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.598+0000 7ff34de00640 1 -- 172.21.15.121:0/2185806462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3340039b0 msgr2=0x7ff334003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:02.601 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.598+0000 7ff34de00640 1 --2- 172.21.15.121:0/2185806462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3340039b0 0x7ff334003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:02.601 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.598+0000 7ff34de00640 1 -- 172.21.15.121:0/2185806462 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff334003520 con 0x7ff334004800 2024-04-20T10:22:02.601 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.598+0000 7ff34e800640 1 --2- 172.21.15.121:0/2185806462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3340039b0 0x7ff334003df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:02.601 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.598+0000 7ff34f200640 1 -- 172.21.15.121:0/2185806462 reap_dead start 2024-04-20T10:22:02.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.599+0000 7ff34de00640 1 --2- 172.21.15.121:0/2185806462 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff334004800 0x7ff334004c80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff338009a90 tx=0x7ff334005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d91caf3e71dc69 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:02.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.600+0000 7ff34d400640 1 -- 172.21.15.121:0/2185806462 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff338039070 con 0x7ff334004800 2024-04-20T10:22:02.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.600+0000 7ff34d400640 1 -- 172.21.15.121:0/2185806462 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff33802c6b0 con 0x7ff334004800 2024-04-20T10:22:02.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.600+0000 7ff34d400640 1 -- 172.21.15.121:0/2185806462 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff338009690 con 0x7ff334004800 2024-04-20T10:22:02.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.600+0000 7ff354a00640 1 -- 172.21.15.121:0/2185806462 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff334004800 msgr2=0x7ff334004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:02.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.600+0000 7ff354a00640 1 --2- 172.21.15.121:0/2185806462 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff334004800 0x7ff334004c80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff338009a90 tx=0x7ff334005830 comp rx=0 tx=0).stop 2024-04-20T10:22:02.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.600+0000 7ff354a00640 1 -- 172.21.15.121:0/2185806462 shutdown_connections 2024-04-20T10:22:02.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.600+0000 7ff354a00640 1 --2- 172.21.15.121:0/2185806462 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff334004800 0x7ff334004c80 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:02.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.600+0000 7ff354a00640 1 -- 172.21.15.121:0/2185806462 >> 172.21.15.121:0/2185806462 conn(0x7ff3500d0ed0 msgr2=0x7ff3500d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:02.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.600+0000 7ff354a00640 1 -- 172.21.15.121:0/2185806462 shutdown_connections 2024-04-20T10:22:02.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.600+0000 7ff354a00640 1 -- 172.21.15.121:0/2185806462 wait complete. 2024-04-20T10:22:02.606 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.603+0000 7ff354a00640 1 Processor -- start 2024-04-20T10:22:02.606 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.603+0000 7ff354a00640 1 -- start start 2024-04-20T10:22:02.606 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.604+0000 7ff354a00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff334004800 0x7ff3500ac4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:02.606 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.604+0000 7ff354a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff350160560 0x7ff3500ade70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:02.606 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.604+0000 7ff354a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3500ae3b0 0x7ff3500ae830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:02.606 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.604+0000 7ff354a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff350162ae0 con 0x7ff3500ae3b0 2024-04-20T10:22:02.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.604+0000 7ff354a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff3500d0690 con 0x7ff350160560 2024-04-20T10:22:02.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.604+0000 7ff354a00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ff350161e30 con 0x7ff334004800 2024-04-20T10:22:02.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.604+0000 7ff34f200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3500ae3b0 0x7ff3500ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:02.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.604+0000 7ff34f200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3500ae3b0 0x7ff3500ae830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43140/0 (socket says 172.21.15.121:43140) 2024-04-20T10:22:02.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.604+0000 7ff34f200640 1 -- 172.21.15.121:0/3669361695 learned_addr learned my addr 172.21.15.121:0/3669361695 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:02.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.605+0000 7ff34de00640 1 --2- 172.21.15.121:0/3669361695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff350160560 0x7ff3500ade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:02.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.605+0000 7ff34e800640 1 --2- 172.21.15.121:0/3669361695 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff334004800 0x7ff3500ac4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:02.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.605+0000 7ff34f200640 1 -- 172.21.15.121:0/3669361695 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff334004800 msgr2=0x7ff3500ac4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:02.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.605+0000 7ff34f200640 1 --2- 172.21.15.121:0/3669361695 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff334004800 0x7ff3500ac4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:02.608 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.605+0000 7ff34f200640 1 -- 172.21.15.121:0/3669361695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff350160560 msgr2=0x7ff3500ade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:02.608 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.605+0000 7ff34f200640 1 --2- 172.21.15.121:0/3669361695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff350160560 0x7ff3500ade70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:02.608 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.605+0000 7ff34f200640 1 -- 172.21.15.121:0/3669361695 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3500cc570 con 0x7ff3500ae3b0 2024-04-20T10:22:02.608 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.605+0000 7ff34f200640 1 --2- 172.21.15.121:0/3669361695 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3500ae3b0 0x7ff3500ae830 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff350162a90 tx=0x7ff34800c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:02.608 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.605+0000 7ff346a00640 1 -- 172.21.15.121:0/3669361695 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3480075e0 con 0x7ff3500ae3b0 2024-04-20T10:22:02.608 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.605+0000 7ff346a00640 1 -- 172.21.15.121:0/3669361695 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3480057c0 con 0x7ff3500ae3b0 2024-04-20T10:22:02.608 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.605+0000 7ff346a00640 1 -- 172.21.15.121:0/3669361695 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff348005bb0 con 0x7ff3500ae3b0 2024-04-20T10:22:02.608 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.605+0000 7ff354a00640 1 -- 172.21.15.121:0/3669361695 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3500acbc0 con 0x7ff3500ae3b0 2024-04-20T10:22:02.608 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.606+0000 7ff354a00640 1 -- 172.21.15.121:0/3669361695 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3500ad180 con 0x7ff3500ae3b0 2024-04-20T10:22:02.609 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.607+0000 7ff354a00640 1 -- 172.21.15.121:0/3669361695 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff350075b30 con 0x7ff3500ae3b0 2024-04-20T10:22:02.609 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.608+0000 7ff346a00640 1 -- 172.21.15.121:0/3669361695 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff348011070 con 0x7ff3500ae3b0 2024-04-20T10:22:02.609 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.609+0000 7ff346a00640 1 --2- 172.21.15.121:0/3669361695 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff32c06c970 0x7ff32c06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:02.610 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.609+0000 7ff346a00640 1 -- 172.21.15.121:0/3669361695 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ff34808b330 con 0x7ff3500ae3b0 2024-04-20T10:22:02.610 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.609+0000 7ff34e800640 1 --2- 172.21.15.121:0/3669361695 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff32c06c970 0x7ff32c06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:02.610 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.610+0000 7ff34e800640 1 --2- 172.21.15.121:0/3669361695 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff32c06c970 0x7ff32c06ee30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff340002230 tx=0x7ff340009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:02.613 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.613+0000 7ff346a00640 1 -- 172.21.15.121:0/3669361695 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff348019060 con 0x7ff3500ae3b0 2024-04-20T10:22:02.814 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.813+0000 7ff354a00640 1 -- 172.21.15.121:0/3669361695 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ff3500d3f10 con 0x7ff3500ae3b0 2024-04-20T10:22:02.814 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.814+0000 7ff346a00640 1 -- 172.21.15.121:0/3669361695 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7ff34800f260 con 0x7ff3500ae3b0 2024-04-20T10:22:02.814 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 32 2024-04-20T10:22:02.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.816+0000 7ff354a00640 1 -- 172.21.15.121:0/3669361695 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff32c06c970 msgr2=0x7ff32c06ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:02.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.816+0000 7ff354a00640 1 --2- 172.21.15.121:0/3669361695 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff32c06c970 0x7ff32c06ee30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff340002230 tx=0x7ff340009040 comp rx=0 tx=0).stop 2024-04-20T10:22:02.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.816+0000 7ff354a00640 1 -- 172.21.15.121:0/3669361695 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3500ae3b0 msgr2=0x7ff3500ae830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:02.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.816+0000 7ff354a00640 1 --2- 172.21.15.121:0/3669361695 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3500ae3b0 0x7ff3500ae830 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff350162a90 tx=0x7ff34800c820 comp rx=0 tx=0).stop 2024-04-20T10:22:02.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.816+0000 7ff354a00640 1 -- 172.21.15.121:0/3669361695 shutdown_connections 2024-04-20T10:22:02.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.816+0000 7ff354a00640 1 --2- 172.21.15.121:0/3669361695 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3500ae3b0 0x7ff3500ae830 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:02.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.816+0000 7ff354a00640 1 --2- 172.21.15.121:0/3669361695 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff32c06c970 0x7ff32c06ee30 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:02.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.816+0000 7ff354a00640 1 --2- 172.21.15.121:0/3669361695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff350160560 0x7ff3500ade70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:02.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.816+0000 7ff354a00640 1 --2- 172.21.15.121:0/3669361695 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff334004800 0x7ff3500ac4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:02.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.817+0000 7ff354a00640 1 -- 172.21.15.121:0/3669361695 >> 172.21.15.121:0/3669361695 conn(0x7ff3500d0ed0 msgr2=0x7ff3500b8a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:02.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.817+0000 7ff354a00640 1 -- 172.21.15.121:0/3669361695 shutdown_connections 2024-04-20T10:22:02.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:02.817+0000 7ff354a00640 1 -- 172.21.15.121:0/3669361695 wait complete. 2024-04-20T10:22:02.835 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:22:03.035 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.035+0000 7ff357e00640 1 Processor -- start 2024-04-20T10:22:03.036 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.035+0000 7ff357e00640 1 -- start start 2024-04-20T10:22:03.037 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.036+0000 7ff357e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff35006dcb0 0x7ff350071b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:03.037 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.036+0000 7ff357e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3500ade40 con 0x7ff350072090 2024-04-20T10:22:03.037 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.036+0000 7ff357e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3500ae0e0 con 0x7ff35015b3b0 2024-04-20T10:22:03.037 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.036+0000 7ff357e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ff35006f1b0 con 0x7ff35006dcb0 2024-04-20T10:22:03.039 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.037+0000 7ff355e00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7ff350072090 0x7ff3500d0360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:40532/0 (socket says 172.21.15.121:40532) 2024-04-20T10:22:03.039 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.037+0000 7ff355e00640 1 -- 172.21.15.121:0/2290751711 learned_addr learned my addr 172.21.15.121:0/2290751711 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:03.039 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.037+0000 7ff354a00640 1 --2- 172.21.15.121:0/2290751711 >> v2:172.21.15.123:3300/0 conn(0x7ff35006dcb0 0x7ff350071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:03.039 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.037+0000 7ff34fe00640 1 -- 172.21.15.121:0/2290751711 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1653690401 0 0) 0x7ff3500ade40 con 0x7ff350072090 2024-04-20T10:22:03.039 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.037+0000 7ff34fe00640 1 -- 172.21.15.121:0/2290751711 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff338003520 con 0x7ff350072090 2024-04-20T10:22:03.039 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.037+0000 7ff34fe00640 1 -- 172.21.15.121:0/2290751711 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4285135167 0 0) 0x7ff338003520 con 0x7ff350072090 2024-04-20T10:22:03.039 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.037+0000 7ff34fe00640 1 -- 172.21.15.121:0/2290751711 >> v2:172.21.15.123:3300/0 conn(0x7ff35006dcb0 msgr2=0x7ff350071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:03.039 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.037+0000 7ff34fe00640 1 --2- 172.21.15.121:0/2290751711 >> v2:172.21.15.123:3300/0 conn(0x7ff35006dcb0 0x7ff350071b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:03.039 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.038+0000 7ff34fe00640 1 -- 172.21.15.121:0/2290751711 >> v1:172.21.15.123:6789/0 conn(0x7ff35015b3b0 legacy=0x7ff35006eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:03.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.038+0000 7ff34fe00640 1 -- 172.21.15.121:0/2290751711 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff35015b790 con 0x7ff350072090 2024-04-20T10:22:03.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.038+0000 7ff34fe00640 1 -- 172.21.15.121:0/2290751711 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7ff3480019f0 con 0x7ff350072090 2024-04-20T10:22:03.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.038+0000 7ff34fe00640 1 -- 172.21.15.121:0/2290751711 >> v1:172.21.15.121:6789/0 conn(0x7ff350072090 legacy=0x7ff3500d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:03.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.038+0000 7ff34fe00640 1 --2- 172.21.15.121:0/2290751711 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff3380039b0 0x7ff338003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:03.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.038+0000 7ff34fe00640 1 --2- 172.21.15.121:0/2290751711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff338004800 0x7ff338004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:03.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.038+0000 7ff34fe00640 1 --2- 172.21.15.121:0/2290751711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3380051c0 0x7ff33800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:03.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.038+0000 7ff34fe00640 1 -- 172.21.15.121:0/2290751711 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff35006f1b0 con 0x7ff3380051c0 2024-04-20T10:22:03.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.038+0000 7ff34fe00640 1 -- 172.21.15.121:0/2290751711 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff33800c060 con 0x7ff338004800 2024-04-20T10:22:03.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.038+0000 7ff34fe00640 1 -- 172.21.15.121:0/2290751711 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ff33800c260 con 0x7ff3380039b0 2024-04-20T10:22:03.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.038+0000 7ff355e00640 1 --2- 172.21.15.121:0/2290751711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3380051c0 0x7ff33800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:03.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.038+0000 7ff354a00640 1 --2- 172.21.15.121:0/2290751711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff338004800 0x7ff338004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:03.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.038+0000 7ff355400640 1 --2- 172.21.15.121:0/2290751711 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff3380039b0 0x7ff338003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:03.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.038+0000 7ff355e00640 1 -- 172.21.15.121:0/2290751711 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff3380039b0 msgr2=0x7ff338003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:03.041 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.038+0000 7ff355e00640 1 --2- 172.21.15.121:0/2290751711 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff3380039b0 0x7ff338003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:03.041 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.038+0000 7ff355e00640 1 -- 172.21.15.121:0/2290751711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff338004800 msgr2=0x7ff338004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:03.041 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.038+0000 7ff355e00640 1 --2- 172.21.15.121:0/2290751711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff338004800 0x7ff338004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:03.041 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.039+0000 7ff355e00640 1 -- 172.21.15.121:0/2290751711 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff338003520 con 0x7ff3380051c0 2024-04-20T10:22:03.042 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.039+0000 7ff355e00640 1 -- 172.21.15.121:0/2290751711 reap_dead start 2024-04-20T10:22:03.042 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.040+0000 7ff355e00640 1 --2- 172.21.15.121:0/2290751711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3380051c0 0x7ff33800b9c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff34800a850 tx=0x7ff338005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e468351e59517118 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:03.042 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.040+0000 7ff34fe00640 1 -- 172.21.15.121:0/2290751711 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff34803a070 con 0x7ff3380051c0 2024-04-20T10:22:03.042 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.040+0000 7ff34fe00640 1 -- 172.21.15.121:0/2290751711 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff348035660 con 0x7ff3380051c0 2024-04-20T10:22:03.042 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.040+0000 7ff34fe00640 1 -- 172.21.15.121:0/2290751711 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff34800a4a0 con 0x7ff3380051c0 2024-04-20T10:22:03.042 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.040+0000 7ff357e00640 1 -- 172.21.15.121:0/2290751711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3380051c0 msgr2=0x7ff33800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:03.043 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.040+0000 7ff357e00640 1 --2- 172.21.15.121:0/2290751711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3380051c0 0x7ff33800b9c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff34800a850 tx=0x7ff338005830 comp rx=0 tx=0).stop 2024-04-20T10:22:03.043 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.041+0000 7ff357e00640 1 -- 172.21.15.121:0/2290751711 shutdown_connections 2024-04-20T10:22:03.043 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.041+0000 7ff357e00640 1 --2- 172.21.15.121:0/2290751711 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3380051c0 0x7ff33800b9c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:03.043 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.041+0000 7ff357e00640 1 -- 172.21.15.121:0/2290751711 >> 172.21.15.121:0/2290751711 conn(0x7ff3500b3350 msgr2=0x7ff3500b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:03.043 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.041+0000 7ff357e00640 1 -- 172.21.15.121:0/2290751711 shutdown_connections 2024-04-20T10:22:03.043 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.041+0000 7ff357e00640 1 -- 172.21.15.121:0/2290751711 wait complete. 2024-04-20T10:22:03.043 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.042+0000 7ff357e00640 1 Processor -- start 2024-04-20T10:22:03.043 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.043+0000 7ff357e00640 1 -- start start 2024-04-20T10:22:03.045 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.044+0000 7ff357e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3380051c0 0x7ff35015d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:03.045 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.044+0000 7ff357e00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff35014cdf0 0x7ff3501562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:03.045 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.044+0000 7ff357e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3501567f0 0x7ff3500af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:03.045 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.044+0000 7ff357e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff3500ae940 con 0x7ff3501567f0 2024-04-20T10:22:03.045 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.044+0000 7ff357e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff3500ae3e0 con 0x7ff3380051c0 2024-04-20T10:22:03.045 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.044+0000 7ff357e00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ff3500aec20 con 0x7ff35014cdf0 2024-04-20T10:22:03.045 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.044+0000 7ff355e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3501567f0 0x7ff3500af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:03.045 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.044+0000 7ff355e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3501567f0 0x7ff3500af2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43150/0 (socket says 172.21.15.121:43150) 2024-04-20T10:22:03.045 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.044+0000 7ff355e00640 1 -- 172.21.15.121:0/172881308 learned_addr learned my addr 172.21.15.121:0/172881308 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:03.045 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.044+0000 7ff355400640 1 --2- 172.21.15.121:0/172881308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3380051c0 0x7ff35015d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:03.046 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.044+0000 7ff355e00640 1 -- 172.21.15.121:0/172881308 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff35014cdf0 msgr2=0x7ff3501562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:03.046 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.044+0000 7ff355e00640 1 --2- 172.21.15.121:0/172881308 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff35014cdf0 0x7ff3501562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:03.046 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.044+0000 7ff355e00640 1 -- 172.21.15.121:0/172881308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3380051c0 msgr2=0x7ff35015d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:03.046 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.044+0000 7ff355e00640 1 --2- 172.21.15.121:0/172881308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3380051c0 0x7ff35015d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:03.046 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.044+0000 7ff355e00640 1 -- 172.21.15.121:0/172881308 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff35015bd00 con 0x7ff3501567f0 2024-04-20T10:22:03.046 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.045+0000 7ff355e00640 1 --2- 172.21.15.121:0/172881308 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3501567f0 0x7ff3500af2b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff348000f80 tx=0x7ff348001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:03.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.045+0000 7ff34d600640 1 -- 172.21.15.121:0/172881308 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff34803a040 con 0x7ff3501567f0 2024-04-20T10:22:03.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.045+0000 7ff34d600640 1 -- 172.21.15.121:0/172881308 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff348044070 con 0x7ff3501567f0 2024-04-20T10:22:03.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.045+0000 7ff34d600640 1 -- 172.21.15.121:0/172881308 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff34800a480 con 0x7ff3501567f0 2024-04-20T10:22:03.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.045+0000 7ff357e00640 1 -- 172.21.15.121:0/172881308 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff35015bfe0 con 0x7ff3501567f0 2024-04-20T10:22:03.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.045+0000 7ff357e00640 1 -- 172.21.15.121:0/172881308 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff35015c5a0 con 0x7ff3501567f0 2024-04-20T10:22:03.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.046+0000 7ff357e00640 1 -- 172.21.15.121:0/172881308 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff350068980 con 0x7ff3501567f0 2024-04-20T10:22:03.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.047+0000 7ff34d600640 1 -- 172.21.15.121:0/172881308 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff348049460 con 0x7ff3501567f0 2024-04-20T10:22:03.048 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.048+0000 7ff34d600640 1 --2- 172.21.15.121:0/172881308 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff33006c920 0x7ff33006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:03.049 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.048+0000 7ff34d600640 1 -- 172.21.15.121:0/172881308 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ff3480ada80 con 0x7ff3501567f0 2024-04-20T10:22:03.049 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.048+0000 7ff355400640 1 --2- 172.21.15.121:0/172881308 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff33006c920 0x7ff33006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:03.049 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.049+0000 7ff355400640 1 --2- 172.21.15.121:0/172881308 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff33006c920 0x7ff33006ede0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff344000c00 tx=0x7ff344009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:03.053 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.053+0000 7ff34d600640 1 -- 172.21.15.121:0/172881308 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff34804e020 con 0x7ff3501567f0 2024-04-20T10:22:03.262 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.261+0000 7ff357e00640 1 -- 172.21.15.121:0/172881308 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff350159f10 con 0x7ff3501567f0 2024-04-20T10:22:03.264 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.264+0000 7ff34d600640 1 -- 172.21.15.121:0/172881308 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18940 (secure 0 0 0) 0x7ff34807b460 con 0x7ff3501567f0 2024-04-20T10:22:03.265 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:22:03.265 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":27,"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","created":"2024-04-20T10:20:38.076076+0000","modified":"2024-04-20T10:21:44.715898+0000","last_up_change":"2024-04-20T10:20:50.162700+0000","last_in_change":"2024-04-20T10:20:41.510382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:20:51.141365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:21:00.436937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:21:01.026957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:21:04.862693+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"629a7b8c-a40a-4a2a-a3cc-d9f357774310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6809","nonce":1044588835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6811","nonce":1044588835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6815","nonce":1044588835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6813","nonce":1044588835}]},"public_addr":"172.21.15.121:6809/1044588835","cluster_addr":"172.21.15.121:6811/1044588835","heartbeat_back_addr":"172.21.15.121:6815/1044588835","heartbeat_front_addr":"172.21.15.121:6813/1044588835","state":["exists","up"]},{"osd":1,"uuid":"f2570399-aa50-4ff3-bf5d-3d09207290db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6801","nonce":3505199624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6803","nonce":3505199624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6807","nonce":3505199624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6805","nonce":3505199624}]},"public_addr":"172.21.15.121:6801/3505199624","cluster_addr":"172.21.15.121:6803/3505199624","heartbeat_back_addr":"172.21.15.121:6807/3505199624","heartbeat_front_addr":"172.21.15.121:6805/3505199624","state":["exists","up"]},{"osd":2,"uuid":"50893807-dc30-49f4-b41f-739366626658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6817","nonce":3656724207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6819","nonce":3656724207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6827","nonce":3656724207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6822","nonce":3656724207}]},"public_addr":"172.21.15.121:6817/3656724207","cluster_addr":"172.21.15.121:6819/3656724207","heartbeat_back_addr":"172.21.15.121:6827/3656724207","heartbeat_front_addr":"172.21.15.121:6822/3656724207","state":["exists","up"]},{"osd":3,"uuid":"dadb01bc-5491-4ffc-8824-d48a057240b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6823","nonce":2232342832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6826","nonce":2232342832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6831","nonce":2232342832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6829","nonce":2232342832}]},"public_addr":"172.21.15.121:6823/2232342832","cluster_addr":"172.21.15.121:6826/2232342832","heartbeat_back_addr":"172.21.15.121:6831/2232342832","heartbeat_front_addr":"172.21.15.121:6829/2232342832","state":["exists","up"]},{"osd":4,"uuid":"b3f3b4a1-b375-4526-b69e-344ab24c2da1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6801","nonce":99353111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6803","nonce":99353111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6807","nonce":99353111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6805","nonce":99353111}]},"public_addr":"172.21.15.123:6801/99353111","cluster_addr":"172.21.15.123:6803/99353111","heartbeat_back_addr":"172.21.15.123:6807/99353111","heartbeat_front_addr":"172.21.15.123:6805/99353111","state":["exists","up"]},{"osd":5,"uuid":"2ee24a08-1624-4961-9642-d7b510282484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6818","nonce":2504374688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6822","nonce":2504374688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6830","nonce":2504374688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6826","nonce":2504374688}]},"public_addr":"172.21.15.123:6818/2504374688","cluster_addr":"172.21.15.123:6822/2504374688","heartbeat_back_addr":"172.21.15.123:6830/2504374688","heartbeat_front_addr":"172.21.15.123:6826/2504374688","state":["exists","up"]},{"osd":6,"uuid":"186118cc-6128-49de-acca-55260945c1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6819","nonce":4139299599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6823","nonce":4139299599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6829","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6831","nonce":4139299599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6827","nonce":4139299599}]},"public_addr":"172.21.15.123:6819/4139299599","cluster_addr":"172.21.15.123:6823/4139299599","heartbeat_back_addr":"172.21.15.123:6831/4139299599","heartbeat_front_addr":"172.21.15.123:6827/4139299599","state":["exists","up"]},{"osd":7,"uuid":"a00aac6e-4bd9-415a-a93d-ae85210865ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6809","nonce":4043981069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6811","nonce":4043981069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6815","nonce":4043981069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6813","nonce":4043981069}]},"public_addr":"172.21.15.123:6809/4043981069","cluster_addr":"172.21.15.123:6811/4043981069","heartbeat_back_addr":"172.21.15.123:6815/4043981069","heartbeat_front_addr":"172.21.15.123:6813/4043981069","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.904540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.958585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.616367+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.885244+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.103127+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.167248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.884348+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.862356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.23","mappings":[{"from":3,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:22:03.267 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.266+0000 7ff357e00640 1 -- 172.21.15.121:0/172881308 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff33006c920 msgr2=0x7ff33006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:03.268 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.266+0000 7ff357e00640 1 --2- 172.21.15.121:0/172881308 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff33006c920 0x7ff33006ede0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff344000c00 tx=0x7ff344009040 comp rx=0 tx=0).stop 2024-04-20T10:22:03.268 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.266+0000 7ff357e00640 1 -- 172.21.15.121:0/172881308 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3501567f0 msgr2=0x7ff3500af2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:03.268 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.266+0000 7ff357e00640 1 --2- 172.21.15.121:0/172881308 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3501567f0 0x7ff3500af2b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff348000f80 tx=0x7ff348001dc0 comp rx=0 tx=0).stop 2024-04-20T10:22:03.268 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.267+0000 7ff357e00640 1 -- 172.21.15.121:0/172881308 shutdown_connections 2024-04-20T10:22:03.268 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.267+0000 7ff357e00640 1 --2- 172.21.15.121:0/172881308 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff3501567f0 0x7ff3500af2b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:03.268 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.267+0000 7ff357e00640 1 --2- 172.21.15.121:0/172881308 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff35014cdf0 0x7ff3501562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:03.268 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.267+0000 7ff357e00640 1 --2- 172.21.15.121:0/172881308 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff33006c920 0x7ff33006ede0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:03.268 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.267+0000 7ff357e00640 1 --2- 172.21.15.121:0/172881308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3380051c0 0x7ff35015d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:03.268 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.267+0000 7ff357e00640 1 -- 172.21.15.121:0/172881308 >> 172.21.15.121:0/172881308 conn(0x7ff3500b3350 msgr2=0x7ff35006e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:03.268 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.267+0000 7ff357e00640 1 -- 172.21.15.121:0/172881308 shutdown_connections 2024-04-20T10:22:03.268 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.267+0000 7ff357e00640 1 -- 172.21.15.121:0/172881308 wait complete. 2024-04-20T10:22:03.286 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:20:51.141365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:21:00.436937+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:21:01.026957+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T10:21:04.862693+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T10:22:03.287 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:22:03.491 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.490+0000 7f7d85e00640 1 Processor -- start 2024-04-20T10:22:03.491 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.490+0000 7f7d85e00640 1 -- start start 2024-04-20T10:22:03.492 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.492+0000 7f7d85e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7d80071a60 0x7f7d80167390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:03.493 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.492+0000 7f7d85e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d8015cc20 con 0x7f7d80073b80 2024-04-20T10:22:03.493 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.492+0000 7f7d85e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d8015cec0 con 0x7f7d8015a130 2024-04-20T10:22:03.493 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.492+0000 7f7d85e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f7d8015d7f0 con 0x7f7d80071a60 2024-04-20T10:22:03.493 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.492+0000 7f7d77e00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f7d80073b80 0x7f7d8006f160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:40538/0 (socket says 172.21.15.121:40538) 2024-04-20T10:22:03.493 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.492+0000 7f7d77e00640 1 -- 172.21.15.121:0/1254791422 learned_addr learned my addr 172.21.15.121:0/1254791422 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:03.495 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.493+0000 7f7d85400640 1 --2- 172.21.15.121:0/1254791422 >> v2:172.21.15.123:3300/0 conn(0x7f7d80071a60 0x7f7d80167390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:03.495 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.493+0000 7f7d7fe00640 1 -- 172.21.15.121:0/1254791422 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2741739197 0 0) 0x7f7d8015cc20 con 0x7f7d80073b80 2024-04-20T10:22:03.495 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.493+0000 7f7d7fe00640 1 -- 172.21.15.121:0/1254791422 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d64003520 con 0x7f7d80073b80 2024-04-20T10:22:03.495 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.493+0000 7f7d7fe00640 1 -- 172.21.15.121:0/1254791422 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1163043749 0 0) 0x7f7d64003520 con 0x7f7d80073b80 2024-04-20T10:22:03.496 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.493+0000 7f7d7fe00640 1 -- 172.21.15.121:0/1254791422 >> v2:172.21.15.123:3300/0 conn(0x7f7d80071a60 msgr2=0x7f7d80167390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:03.496 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.493+0000 7f7d7fe00640 1 --2- 172.21.15.121:0/1254791422 >> v2:172.21.15.123:3300/0 conn(0x7f7d80071a60 0x7f7d80167390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:03.496 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.493+0000 7f7d7fe00640 1 -- 172.21.15.121:0/1254791422 >> v1:172.21.15.123:6789/0 conn(0x7f7d8015a130 legacy=0x7f7d80073470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:03.496 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.493+0000 7f7d7fe00640 1 -- 172.21.15.121:0/1254791422 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d8015d1a0 con 0x7f7d80073b80 2024-04-20T10:22:03.496 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.493+0000 7f7d7fe00640 1 -- 172.21.15.121:0/1254791422 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f7d68001670 con 0x7f7d80073b80 2024-04-20T10:22:03.496 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.493+0000 7f7d7fe00640 1 -- 172.21.15.121:0/1254791422 >> v1:172.21.15.121:6789/0 conn(0x7f7d80073b80 legacy=0x7f7d8006f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:03.496 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.493+0000 7f7d7fe00640 1 --2- 172.21.15.121:0/1254791422 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d64003a00 0x7f7d64003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:03.496 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.493+0000 7f7d7fe00640 1 --2- 172.21.15.121:0/1254791422 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d64004800 0x7f7d64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:03.496 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.493+0000 7f7d7fe00640 1 --2- 172.21.15.121:0/1254791422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d64005210 0x7f7d6400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:03.496 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.493+0000 7f7d7fe00640 1 -- 172.21.15.121:0/1254791422 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8015d7f0 con 0x7f7d64004800 2024-04-20T10:22:03.496 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.493+0000 7f7d7fe00640 1 -- 172.21.15.121:0/1254791422 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6400c160 con 0x7f7d64005210 2024-04-20T10:22:03.496 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.493+0000 7f7d7fe00640 1 -- 172.21.15.121:0/1254791422 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f7d6400c360 con 0x7f7d64003a00 2024-04-20T10:22:03.496 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.494+0000 7f7d77e00640 1 --2- 172.21.15.121:0/1254791422 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d64004800 0x7f7d64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:03.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.494+0000 7f7d85400640 1 --2- 172.21.15.121:0/1254791422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d64005210 0x7f7d6400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:03.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.494+0000 7f7d84a00640 1 --2- 172.21.15.121:0/1254791422 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d64003a00 0x7f7d64003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:03.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.494+0000 7f7d77e00640 1 -- 172.21.15.121:0/1254791422 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d64003a00 msgr2=0x7f7d64003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:03.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.494+0000 7f7d77e00640 1 --2- 172.21.15.121:0/1254791422 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d64003a00 0x7f7d64003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:03.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.494+0000 7f7d77e00640 1 -- 172.21.15.121:0/1254791422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d64005210 msgr2=0x7f7d6400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:03.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.494+0000 7f7d77e00640 1 --2- 172.21.15.121:0/1254791422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d64005210 0x7f7d6400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:03.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.494+0000 7f7d77e00640 1 -- 172.21.15.121:0/1254791422 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d64003520 con 0x7f7d64004800 2024-04-20T10:22:03.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.494+0000 7f7d85400640 1 --2- 172.21.15.121:0/1254791422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d64005210 0x7f7d6400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:03.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.494+0000 7f7d85400640 1 -- 172.21.15.121:0/1254791422 reap_dead start 2024-04-20T10:22:03.498 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.495+0000 7f7d77e00640 1 --2- 172.21.15.121:0/1254791422 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d64004800 0x7f7d64004c80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7d6800c160 tx=0x7f7d64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a949d599b7ef0f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:03.499 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.496+0000 7f7d7fe00640 1 -- 172.21.15.121:0/1254791422 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d68007c60 con 0x7f7d64004800 2024-04-20T10:22:03.500 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.496+0000 7f7d7fe00640 1 -- 172.21.15.121:0/1254791422 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d68030070 con 0x7f7d64004800 2024-04-20T10:22:03.500 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.496+0000 7f7d7fe00640 1 -- 172.21.15.121:0/1254791422 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d68005d90 con 0x7f7d64004800 2024-04-20T10:22:03.500 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.496+0000 7f7d85e00640 1 -- 172.21.15.121:0/1254791422 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d64004800 msgr2=0x7f7d64004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:03.500 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.496+0000 7f7d85e00640 1 --2- 172.21.15.121:0/1254791422 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d64004800 0x7f7d64004c80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7d6800c160 tx=0x7f7d64005830 comp rx=0 tx=0).stop 2024-04-20T10:22:03.500 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.496+0000 7f7d85e00640 1 -- 172.21.15.121:0/1254791422 shutdown_connections 2024-04-20T10:22:03.500 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.496+0000 7f7d85e00640 1 --2- 172.21.15.121:0/1254791422 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d64004800 0x7f7d64004c80 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:03.500 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.496+0000 7f7d85e00640 1 -- 172.21.15.121:0/1254791422 >> 172.21.15.121:0/1254791422 conn(0x7f7d800cffa0 msgr2=0x7f7d800d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:03.500 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.496+0000 7f7d85e00640 1 -- 172.21.15.121:0/1254791422 shutdown_connections 2024-04-20T10:22:03.500 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.496+0000 7f7d85e00640 1 -- 172.21.15.121:0/1254791422 wait complete. 2024-04-20T10:22:03.500 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.498+0000 7f7d85e00640 1 Processor -- start 2024-04-20T10:22:03.500 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.498+0000 7f7d85e00640 1 -- start start 2024-04-20T10:22:03.502 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.499+0000 7f7d85e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d64004800 0x7f7d801549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:03.503 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.499+0000 7f7d85e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d800b0e60 0x7f7d800b4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:03.503 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.499+0000 7f7d85e00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d800b12e0 0x7f7d8015af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:03.503 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.499+0000 7f7d85e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7d800735f0 con 0x7f7d64004800 2024-04-20T10:22:03.503 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.499+0000 7f7d85e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7d80073470 con 0x7f7d800b0e60 2024-04-20T10:22:03.503 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.499+0000 7f7d85e00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f7d80073770 con 0x7f7d800b12e0 2024-04-20T10:22:03.503 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.499+0000 7f7d84a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d64004800 0x7f7d801549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:03.503 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.500+0000 7f7d84a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d64004800 0x7f7d801549b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43166/0 (socket says 172.21.15.121:43166) 2024-04-20T10:22:03.503 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.500+0000 7f7d84a00640 1 -- 172.21.15.121:0/3324104950 learned_addr learned my addr 172.21.15.121:0/3324104950 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:03.503 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.500+0000 7f7d77e00640 1 --2- 172.21.15.121:0/3324104950 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d800b0e60 0x7f7d800b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:03.503 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.500+0000 7f7d85400640 1 --2- 172.21.15.121:0/3324104950 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d800b12e0 0x7f7d8015af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:03.503 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.500+0000 7f7d84a00640 1 -- 172.21.15.121:0/3324104950 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d800b12e0 msgr2=0x7f7d8015af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:03.503 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.500+0000 7f7d84a00640 1 --2- 172.21.15.121:0/3324104950 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d800b12e0 0x7f7d8015af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:03.504 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.500+0000 7f7d84a00640 1 -- 172.21.15.121:0/3324104950 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d800b0e60 msgr2=0x7f7d800b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:03.504 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.500+0000 7f7d84a00640 1 --2- 172.21.15.121:0/3324104950 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d800b0e60 0x7f7d800b4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:03.504 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.500+0000 7f7d84a00640 1 -- 172.21.15.121:0/3324104950 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d801553b0 con 0x7f7d64004800 2024-04-20T10:22:03.504 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.500+0000 7f7d77e00640 1 --2- 172.21.15.121:0/3324104950 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d800b0e60 0x7f7d800b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:03.504 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.500+0000 7f7d84a00640 1 --2- 172.21.15.121:0/3324104950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d64004800 0x7f7d801549b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7d7000d570 tx=0x7f7d7000da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:03.504 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.501+0000 7f7d7d600640 1 -- 172.21.15.121:0/3324104950 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d70003610 con 0x7f7d64004800 2024-04-20T10:22:03.504 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.501+0000 7f7d7d600640 1 -- 172.21.15.121:0/3324104950 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d70017070 con 0x7f7d64004800 2024-04-20T10:22:03.504 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.501+0000 7f7d7d600640 1 -- 172.21.15.121:0/3324104950 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d700039f0 con 0x7f7d64004800 2024-04-20T10:22:03.504 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.501+0000 7f7d85e00640 1 -- 172.21.15.121:0/3324104950 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d800afae0 con 0x7f7d64004800 2024-04-20T10:22:03.504 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.501+0000 7f7d85e00640 1 -- 172.21.15.121:0/3324104950 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d800b00a0 con 0x7f7d64004800 2024-04-20T10:22:03.504 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.502+0000 7f7d85e00640 1 -- 172.21.15.121:0/3324104950 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d80075b30 con 0x7f7d64004800 2024-04-20T10:22:03.505 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.503+0000 7f7d7d600640 1 -- 172.21.15.121:0/3324104950 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7d70003b90 con 0x7f7d64004800 2024-04-20T10:22:03.505 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.504+0000 7f7d7d600640 1 --2- 172.21.15.121:0/3324104950 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7d5c06c8f0 0x7f7d5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:03.505 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.504+0000 7f7d7d600640 1 -- 172.21.15.121:0/3324104950 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f7d7008bd30 con 0x7f7d64004800 2024-04-20T10:22:03.505 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.504+0000 7f7d77e00640 1 --2- 172.21.15.121:0/3324104950 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7d5c06c8f0 0x7f7d5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:03.505 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.505+0000 7f7d77e00640 1 --2- 172.21.15.121:0/3324104950 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7d5c06c8f0 0x7f7d5c06edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7d64004780 tx=0x7f7d68009030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:03.509 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.509+0000 7f7d7d600640 1 -- 172.21.15.121:0/3324104950 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7d70059eb0 con 0x7f7d64004800 2024-04-20T10:22:03.708 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.708+0000 7f7d85e00640 1 -- 172.21.15.121:0/3324104950 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7d8006d010 con 0x7f7d64004800 2024-04-20T10:22:03.709 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.709+0000 7f7d7d600640 1 -- 172.21.15.121:0/3324104950 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f7d70060080 con 0x7f7d64004800 2024-04-20T10:22:03.709 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2024-04-20T10:22:03.713 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.711+0000 7f7d85e00640 1 -- 172.21.15.121:0/3324104950 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7d5c06c8f0 msgr2=0x7f7d5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:03.713 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.711+0000 7f7d85e00640 1 --2- 172.21.15.121:0/3324104950 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7d5c06c8f0 0x7f7d5c06edb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7d64004780 tx=0x7f7d68009030 comp rx=0 tx=0).stop 2024-04-20T10:22:03.713 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.711+0000 7f7d85e00640 1 -- 172.21.15.121:0/3324104950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d64004800 msgr2=0x7f7d801549b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:03.713 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.711+0000 7f7d85e00640 1 --2- 172.21.15.121:0/3324104950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d64004800 0x7f7d801549b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7d7000d570 tx=0x7f7d7000da30 comp rx=0 tx=0).stop 2024-04-20T10:22:03.713 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.712+0000 7f7d85e00640 1 -- 172.21.15.121:0/3324104950 shutdown_connections 2024-04-20T10:22:03.713 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.712+0000 7f7d85e00640 1 --2- 172.21.15.121:0/3324104950 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7d800b12e0 0x7f7d8015af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:03.713 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.712+0000 7f7d85e00640 1 --2- 172.21.15.121:0/3324104950 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7d5c06c8f0 0x7f7d5c06edb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:03.713 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.712+0000 7f7d85e00640 1 --2- 172.21.15.121:0/3324104950 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d800b0e60 0x7f7d800b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:03.713 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.712+0000 7f7d85e00640 1 --2- 172.21.15.121:0/3324104950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7d64004800 0x7f7d801549b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:03.713 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.712+0000 7f7d85e00640 1 -- 172.21.15.121:0/3324104950 >> 172.21.15.121:0/3324104950 conn(0x7f7d800cffa0 msgr2=0x7f7d8015dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:03.713 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.712+0000 7f7d85e00640 1 -- 172.21.15.121:0/3324104950 shutdown_connections 2024-04-20T10:22:03.713 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.712+0000 7f7d85e00640 1 -- 172.21.15.121:0/3324104950 wait complete. 2024-04-20T10:22:03.732 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:22:03.929 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.928+0000 7fd07fe00640 1 Processor -- start 2024-04-20T10:22:03.929 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.929+0000 7fd07fe00640 1 -- start start 2024-04-20T10:22:03.931 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.930+0000 7fd07fe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd080158ca0 0x7fd08006db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:03.931 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.930+0000 7fd07fe00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd08015f2b0 0x7fd0801460f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:03.931 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.930+0000 7fd07fe00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd08015f960 con 0x7fd08006e0d0 2024-04-20T10:22:03.931 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.930+0000 7fd07fe00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fd08015f720 con 0x7fd08015f2b0 2024-04-20T10:22:03.931 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.930+0000 7fd07fe00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fd08006b9a0 con 0x7fd080158ca0 2024-04-20T10:22:03.931 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.930+0000 7fd07f400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd08015f2b0 0x7fd0801460f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:03.931 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.931+0000 7fd07f400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fd08015f2b0 0x7fd0801460f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43174/0 (socket says 172.21.15.121:43174) 2024-04-20T10:22:03.931 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.931+0000 7fd07f400640 1 -- 172.21.15.121:0/658131133 learned_addr learned my addr 172.21.15.121:0/658131133 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:03.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.931+0000 7fd07ea00640 1 --2- 172.21.15.121:0/658131133 >> v2:172.21.15.123:3300/0 conn(0x7fd080158ca0 0x7fd08006db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:03.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.931+0000 7fd07f400640 1 -- 172.21.15.121:0/658131133 >> v2:172.21.15.123:3300/0 conn(0x7fd080158ca0 msgr2=0x7fd08006db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:03.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.931+0000 7fd07f400640 1 --2- 172.21.15.121:0/658131133 >> v2:172.21.15.123:3300/0 conn(0x7fd080158ca0 0x7fd08006db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:03.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.931+0000 7fd07f400640 1 -- 172.21.15.121:0/658131133 >> v1:172.21.15.123:6789/0 conn(0x7fd08006e0d0 legacy=0x7fd08015eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:03.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.931+0000 7fd07f400640 1 -- 172.21.15.121:0/658131133 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd08006bc80 con 0x7fd08015f2b0 2024-04-20T10:22:03.933 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.932+0000 7fd07f400640 1 --2- 172.21.15.121:0/658131133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd08015f2b0 0x7fd0801460f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd08006eb10 tx=0x7fd07802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf60fc64933823d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:03.933 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.932+0000 7fd07e000640 1 -- 172.21.15.121:0/658131133 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07802ca80 con 0x7fd08015f2b0 2024-04-20T10:22:03.934 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.932+0000 7fd07e000640 1 -- 172.21.15.121:0/658131133 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd07802cc20 con 0x7fd08015f2b0 2024-04-20T10:22:03.934 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.932+0000 7fd07e000640 1 -- 172.21.15.121:0/658131133 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0780343f0 con 0x7fd08015f2b0 2024-04-20T10:22:03.934 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.933+0000 7fd07fe00640 1 -- 172.21.15.121:0/658131133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd08015f2b0 msgr2=0x7fd0801460f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:03.934 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.933+0000 7fd07fe00640 1 --2- 172.21.15.121:0/658131133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd08015f2b0 0x7fd0801460f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd08006eb10 tx=0x7fd07802bee0 comp rx=0 tx=0).stop 2024-04-20T10:22:03.934 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.933+0000 7fd07fe00640 1 -- 172.21.15.121:0/658131133 shutdown_connections 2024-04-20T10:22:03.934 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.933+0000 7fd07fe00640 1 --2- 172.21.15.121:0/658131133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd08015f2b0 0x7fd0801460f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:03.934 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.933+0000 7fd07fe00640 1 --2- 172.21.15.121:0/658131133 >> v2:172.21.15.123:3300/0 conn(0x7fd080158ca0 0x7fd08006db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:03.934 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.933+0000 7fd07fe00640 1 -- 172.21.15.121:0/658131133 >> 172.21.15.121:0/658131133 conn(0x7fd0800b5270 msgr2=0x7fd0800b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:03.934 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.933+0000 7fd07fe00640 1 -- 172.21.15.121:0/658131133 shutdown_connections 2024-04-20T10:22:03.934 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.933+0000 7fd07fe00640 1 -- 172.21.15.121:0/658131133 wait complete. 2024-04-20T10:22:03.935 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.935+0000 7fd07fe00640 1 Processor -- start 2024-04-20T10:22:03.935 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.935+0000 7fd07fe00640 1 -- start start 2024-04-20T10:22:03.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.936+0000 7fd07fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd08006e0d0 0x7fd080099770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:03.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.936+0000 7fd07fe00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd080158ca0 0x7fd0800ae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:03.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.936+0000 7fd07fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd08015f2b0 0x7fd0800ae640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:03.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.936+0000 7fd07fe00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd08006c760 con 0x7fd08015f2b0 2024-04-20T10:22:03.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.936+0000 7fd07fe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd08006c240 con 0x7fd08006e0d0 2024-04-20T10:22:03.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.936+0000 7fd07fe00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fd0801426c0 con 0x7fd080158ca0 2024-04-20T10:22:03.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.936+0000 7fd07f400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd08015f2b0 0x7fd0800ae640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:03.939 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.936+0000 7fd07f400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd08015f2b0 0x7fd0800ae640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43180/0 (socket says 172.21.15.121:43180) 2024-04-20T10:22:03.939 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.936+0000 7fd07f400640 1 -- 172.21.15.121:0/2188330204 learned_addr learned my addr 172.21.15.121:0/2188330204 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:03.939 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.937+0000 7fd07ea00640 1 --2- 172.21.15.121:0/2188330204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd08006e0d0 0x7fd080099770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:03.939 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.937+0000 7fd07f400640 1 -- 172.21.15.121:0/2188330204 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd080158ca0 msgr2=0x7fd0800ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:03.939 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.937+0000 7fd077e00640 1 --2- 172.21.15.121:0/2188330204 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd080158ca0 0x7fd0800ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:03.939 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.937+0000 7fd07f400640 1 --2- 172.21.15.121:0/2188330204 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd080158ca0 0x7fd0800ae100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:03.939 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.937+0000 7fd07f400640 1 -- 172.21.15.121:0/2188330204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd08006e0d0 msgr2=0x7fd080099770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:03.939 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.937+0000 7fd07f400640 1 --2- 172.21.15.121:0/2188330204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd08006e0d0 0x7fd080099770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:03.939 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.937+0000 7fd07f400640 1 -- 172.21.15.121:0/2188330204 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd080142960 con 0x7fd08015f2b0 2024-04-20T10:22:03.939 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.937+0000 7fd07f400640 1 --2- 172.21.15.121:0/2188330204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd08015f2b0 0x7fd0800ae640 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd078036040 tx=0x7fd078004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:03.939 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.937+0000 7fd076a00640 1 -- 172.21.15.121:0/2188330204 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd078043020 con 0x7fd08015f2b0 2024-04-20T10:22:03.939 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.937+0000 7fd076a00640 1 -- 172.21.15.121:0/2188330204 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd07803e450 con 0x7fd08015f2b0 2024-04-20T10:22:03.939 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.937+0000 7fd076a00640 1 -- 172.21.15.121:0/2188330204 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07803e860 con 0x7fd08015f2b0 2024-04-20T10:22:03.939 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.937+0000 7fd07fe00640 1 -- 172.21.15.121:0/2188330204 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd080142c40 con 0x7fd08015f2b0 2024-04-20T10:22:03.939 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.938+0000 7fd07fe00640 1 -- 172.21.15.121:0/2188330204 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd080143200 con 0x7fd08015f2b0 2024-04-20T10:22:03.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.938+0000 7fd07fe00640 1 -- 172.21.15.121:0/2188330204 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0801435e0 con 0x7fd08015f2b0 2024-04-20T10:22:03.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.939+0000 7fd076a00640 1 -- 172.21.15.121:0/2188330204 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd078046020 con 0x7fd08015f2b0 2024-04-20T10:22:03.941 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.940+0000 7fd076a00640 1 --2- 172.21.15.121:0/2188330204 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fd05c06c920 0x7fd05c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:03.941 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.940+0000 7fd076a00640 1 -- 172.21.15.121:0/2188330204 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fd0780ae640 con 0x7fd08015f2b0 2024-04-20T10:22:03.941 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.941+0000 7fd07ea00640 1 --2- 172.21.15.121:0/2188330204 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fd05c06c920 0x7fd05c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:03.941 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.941+0000 7fd07ea00640 1 --2- 172.21.15.121:0/2188330204 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fd05c06c920 0x7fd05c06ede0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd07000a970 tx=0x7fd070008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:03.945 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:03.945+0000 7fd076a00640 1 -- 172.21.15.121:0/2188330204 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd0801435e0 con 0x7fd08015f2b0 2024-04-20T10:22:04.147 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.146+0000 7fd07fe00640 1 -- 172.21.15.121:0/2188330204 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd080075b30 con 0x7fd08015f2b0 2024-04-20T10:22:04.147 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.147+0000 7fd076a00640 1 -- 172.21.15.121:0/2188330204 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fd07804d070 con 0x7fd08015f2b0 2024-04-20T10:22:04.148 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 32 2024-04-20T10:22:04.150 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.149+0000 7fd07fe00640 1 -- 172.21.15.121:0/2188330204 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fd05c06c920 msgr2=0x7fd05c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:04.150 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.149+0000 7fd07fe00640 1 --2- 172.21.15.121:0/2188330204 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fd05c06c920 0x7fd05c06ede0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd07000a970 tx=0x7fd070008040 comp rx=0 tx=0).stop 2024-04-20T10:22:04.150 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.149+0000 7fd07fe00640 1 -- 172.21.15.121:0/2188330204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd08015f2b0 msgr2=0x7fd0800ae640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:04.150 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.149+0000 7fd07fe00640 1 --2- 172.21.15.121:0/2188330204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd08015f2b0 0x7fd0800ae640 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd078036040 tx=0x7fd078004a40 comp rx=0 tx=0).stop 2024-04-20T10:22:04.151 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.150+0000 7fd07fe00640 1 -- 172.21.15.121:0/2188330204 shutdown_connections 2024-04-20T10:22:04.151 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.150+0000 7fd07fe00640 1 --2- 172.21.15.121:0/2188330204 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd08015f2b0 0x7fd0800ae640 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:04.151 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.150+0000 7fd07fe00640 1 --2- 172.21.15.121:0/2188330204 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd080158ca0 0x7fd0800ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:04.151 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.150+0000 7fd07fe00640 1 --2- 172.21.15.121:0/2188330204 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fd05c06c920 0x7fd05c06ede0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:04.151 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.150+0000 7fd07fe00640 1 --2- 172.21.15.121:0/2188330204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd08006e0d0 0x7fd080099770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:04.151 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.150+0000 7fd07fe00640 1 -- 172.21.15.121:0/2188330204 >> 172.21.15.121:0/2188330204 conn(0x7fd0800b5270 msgr2=0x7fd080071e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:04.151 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.150+0000 7fd07fe00640 1 -- 172.21.15.121:0/2188330204 shutdown_connections 2024-04-20T10:22:04.151 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.150+0000 7fd07fe00640 1 -- 172.21.15.121:0/2188330204 wait complete. 2024-04-20T10:22:04.169 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:22:04.364 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.363+0000 7efeb5400640 1 Processor -- start 2024-04-20T10:22:04.364 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.364+0000 7efeb5400640 1 -- start start 2024-04-20T10:22:04.365 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.365+0000 7efeb5400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efeb00d4270 0x7efeb00d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:04.365 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.365+0000 7efeb5400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7efeb00d0380 0x7efeb00d0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:04.365 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.365+0000 7efeb5400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efeb00d0d50 con 0x7efeb00d4650 2024-04-20T10:22:04.365 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.365+0000 7efeb5400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7efeb00d40c0 con 0x7efeb00d0380 2024-04-20T10:22:04.365 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.365+0000 7efeb5400640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7efeb00d3820 con 0x7efeb00d4270 2024-04-20T10:22:04.366 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.366+0000 7efeae800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efeb00d4270 0x7efeb00d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:04.366 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.366+0000 7efeaf200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7efeb00d0380 0x7efeb00d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:04.366 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.366+0000 7efeae800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efeb00d4270 0x7efeb00d1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.121:43022/0 (socket says 172.21.15.121:43022) 2024-04-20T10:22:04.366 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.366+0000 7efeae800640 1 -- 172.21.15.121:0/4104121256 learned_addr learned my addr 172.21.15.121:0/4104121256 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:04.367 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.366+0000 7efeaf200640 1 -- 172.21.15.121:0/4104121256 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efeb00d4270 msgr2=0x7efeb00d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:04.367 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.366+0000 7efeaf200640 1 --2- 172.21.15.121:0/4104121256 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efeb00d4270 0x7efeb00d1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:04.367 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.366+0000 7efeaf200640 1 -- 172.21.15.121:0/4104121256 >> v1:172.21.15.123:6789/0 conn(0x7efeb00d4650 legacy=0x7efeb00b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:04.367 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.367+0000 7efeaf200640 1 -- 172.21.15.121:0/4104121256 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb00d3b00 con 0x7efeb00d0380 2024-04-20T10:22:04.368 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.368+0000 7efeaf200640 1 --2- 172.21.15.121:0/4104121256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efeb00d0380 0x7efeb00d0810 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efea8009d90 tx=0x7efea802bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64e343e9f5740f5f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:04.368 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.368+0000 7efead400640 1 -- 172.21.15.121:0/4104121256 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea8039070 con 0x7efeb00d0380 2024-04-20T10:22:04.369 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.368+0000 7efead400640 1 -- 172.21.15.121:0/4104121256 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efea802cc50 con 0x7efeb00d0380 2024-04-20T10:22:04.369 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.368+0000 7efead400640 1 -- 172.21.15.121:0/4104121256 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea8034440 con 0x7efeb00d0380 2024-04-20T10:22:04.370 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.369+0000 7efeb5400640 1 -- 172.21.15.121:0/4104121256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efeb00d0380 msgr2=0x7efeb00d0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:04.370 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.369+0000 7efeb5400640 1 --2- 172.21.15.121:0/4104121256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efeb00d0380 0x7efeb00d0810 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efea8009d90 tx=0x7efea802bea0 comp rx=0 tx=0).stop 2024-04-20T10:22:04.370 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.369+0000 7efeb5400640 1 -- 172.21.15.121:0/4104121256 shutdown_connections 2024-04-20T10:22:04.370 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.369+0000 7efeb5400640 1 --2- 172.21.15.121:0/4104121256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efeb00d0380 0x7efeb00d0810 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:04.370 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.369+0000 7efeb5400640 1 --2- 172.21.15.121:0/4104121256 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efeb00d4270 0x7efeb00d1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:04.370 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.369+0000 7efeb5400640 1 -- 172.21.15.121:0/4104121256 >> 172.21.15.121:0/4104121256 conn(0x7efeb00b99c0 msgr2=0x7efeb00b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:04.370 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.369+0000 7efeb5400640 1 -- 172.21.15.121:0/4104121256 shutdown_connections 2024-04-20T10:22:04.370 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.369+0000 7efeb5400640 1 -- 172.21.15.121:0/4104121256 wait complete. 2024-04-20T10:22:04.371 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.371+0000 7efeb5400640 1 Processor -- start 2024-04-20T10:22:04.371 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.371+0000 7efeb5400640 1 -- start start 2024-04-20T10:22:04.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.372+0000 7efeb5400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efeb00d0380 0x7efeb006e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:04.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.372+0000 7efeb5400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efeb00d4270 0x7efeb00ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:04.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.372+0000 7efeb5400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efeb00d4650 0x7efeb00cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:04.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.372+0000 7efeb5400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efeb00b19a0 con 0x7efeb00d4650 2024-04-20T10:22:04.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.372+0000 7efeb5400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efeb00cfe60 con 0x7efeb00d4270 2024-04-20T10:22:04.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.372+0000 7efeb5400640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7efeb00d53b0 con 0x7efeb00d0380 2024-04-20T10:22:04.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.372+0000 7efeaf200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efeb00d4650 0x7efeb00cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:04.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.372+0000 7efeaf200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efeb00d4650 0x7efeb00cb6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43184/0 (socket says 172.21.15.121:43184) 2024-04-20T10:22:04.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.372+0000 7efeaf200640 1 -- 172.21.15.121:0/3939153892 learned_addr learned my addr 172.21.15.121:0/3939153892 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:04.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.373+0000 7efeae800640 1 --2- 172.21.15.121:0/3939153892 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efeb00d0380 0x7efeb006e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:04.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.373+0000 7efeade00640 1 --2- 172.21.15.121:0/3939153892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efeb00d4270 0x7efeb00ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:04.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.373+0000 7efeaf200640 1 -- 172.21.15.121:0/3939153892 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efeb00d0380 msgr2=0x7efeb006e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:04.375 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.373+0000 7efeaf200640 1 --2- 172.21.15.121:0/3939153892 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efeb00d0380 0x7efeb006e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:04.375 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.373+0000 7efeaf200640 1 -- 172.21.15.121:0/3939153892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efeb00d4270 msgr2=0x7efeb00ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:04.375 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.373+0000 7efeaf200640 1 --2- 172.21.15.121:0/3939153892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efeb00d4270 0x7efeb00ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:04.375 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.373+0000 7efeaf200640 1 -- 172.21.15.121:0/3939153892 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb0158600 con 0x7efeb00d4650 2024-04-20T10:22:04.375 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.373+0000 7efeae800640 1 --2- 172.21.15.121:0/3939153892 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efeb00d0380 0x7efeb006e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:22:04.375 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.373+0000 7efeaf200640 1 --2- 172.21.15.121:0/3939153892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efeb00d4650 0x7efeb00cb6b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7efea8009eb0 tx=0x7efea8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:04.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.374+0000 7efea6a00640 1 -- 172.21.15.121:0/3939153892 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea8044020 con 0x7efeb00d4650 2024-04-20T10:22:04.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.374+0000 7efea6a00640 1 -- 172.21.15.121:0/3939153892 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efea802ca50 con 0x7efeb00d4650 2024-04-20T10:22:04.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.374+0000 7efea6a00640 1 -- 172.21.15.121:0/3939153892 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea8039040 con 0x7efeb00d4650 2024-04-20T10:22:04.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.374+0000 7efeb5400640 1 -- 172.21.15.121:0/3939153892 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeb00cd720 con 0x7efeb00d4650 2024-04-20T10:22:04.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.374+0000 7efeb5400640 1 -- 172.21.15.121:0/3939153892 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeb00cdce0 con 0x7efeb00d4650 2024-04-20T10:22:04.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.375+0000 7efeb5400640 1 -- 172.21.15.121:0/3939153892 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeb01587d0 con 0x7efeb00d4650 2024-04-20T10:22:04.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.376+0000 7efea6a00640 1 -- 172.21.15.121:0/3939153892 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efea8047020 con 0x7efeb00d4650 2024-04-20T10:22:04.377 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.377+0000 7efea6a00640 1 --2- 172.21.15.121:0/3939153892 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7efe8c06c920 0x7efe8c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:04.377 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.377+0000 7efea6a00640 1 -- 172.21.15.121:0/3939153892 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7efea80ae500 con 0x7efeb00d4650 2024-04-20T10:22:04.378 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.377+0000 7efeae800640 1 --2- 172.21.15.121:0/3939153892 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7efe8c06c920 0x7efe8c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:04.378 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.378+0000 7efeae800640 1 --2- 172.21.15.121:0/3939153892 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7efe8c06c920 0x7efe8c06ede0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efea0009c20 tx=0x7efea0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:04.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.381+0000 7efea6a00640 1 -- 172.21.15.121:0/3939153892 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7efea804e070 con 0x7efeb00d4650 2024-04-20T10:22:04.569 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.568+0000 7efeb5400640 1 -- 172.21.15.121:0/3939153892 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7efeb00b6ec0 con 0x7efeb00d4650 2024-04-20T10:22:04.570 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.569+0000 7efea6a00640 1 -- 172.21.15.121:0/3939153892 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7efea807c5c0 con 0x7efeb00d4650 2024-04-20T10:22:04.570 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2024-04-20T10:22:04.572 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.571+0000 7efeb5400640 1 -- 172.21.15.121:0/3939153892 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7efe8c06c920 msgr2=0x7efe8c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:04.572 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.571+0000 7efeb5400640 1 --2- 172.21.15.121:0/3939153892 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7efe8c06c920 0x7efe8c06ede0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efea0009c20 tx=0x7efea0008040 comp rx=0 tx=0).stop 2024-04-20T10:22:04.572 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.571+0000 7efeb5400640 1 -- 172.21.15.121:0/3939153892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efeb00d4650 msgr2=0x7efeb00cb6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:04.573 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.571+0000 7efeb5400640 1 --2- 172.21.15.121:0/3939153892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efeb00d4650 0x7efeb00cb6b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7efea8009eb0 tx=0x7efea8004a40 comp rx=0 tx=0).stop 2024-04-20T10:22:04.573 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.572+0000 7efeb5400640 1 -- 172.21.15.121:0/3939153892 shutdown_connections 2024-04-20T10:22:04.573 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.572+0000 7efeb5400640 1 --2- 172.21.15.121:0/3939153892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efeb00d4650 0x7efeb00cb6b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:04.573 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.572+0000 7efeb5400640 1 --2- 172.21.15.121:0/3939153892 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7efe8c06c920 0x7efe8c06ede0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:04.573 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.572+0000 7efeb5400640 1 --2- 172.21.15.121:0/3939153892 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efeb00d4270 0x7efeb00ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:04.573 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.572+0000 7efeb5400640 1 --2- 172.21.15.121:0/3939153892 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efeb00d0380 0x7efeb006e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:04.573 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.572+0000 7efeb5400640 1 -- 172.21.15.121:0/3939153892 >> 172.21.15.121:0/3939153892 conn(0x7efeb00b99c0 msgr2=0x7efeb0160ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:04.573 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.572+0000 7efeb5400640 1 -- 172.21.15.121:0/3939153892 shutdown_connections 2024-04-20T10:22:04.573 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.572+0000 7efeb5400640 1 -- 172.21.15.121:0/3939153892 wait complete. 2024-04-20T10:22:04.590 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T10:22:04.795 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.794+0000 7f020b200640 1 Processor -- start 2024-04-20T10:22:04.795 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.794+0000 7f020b200640 1 -- start start 2024-04-20T10:22:04.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.796+0000 7f020b200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f02040ae0c0 0x7f02041621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:04.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.796+0000 7f020b200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f02040aebb0 0x7f020406b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:04.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.796+0000 7f020b200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02040b4af0 con 0x7f02041626f0 2024-04-20T10:22:04.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.796+0000 7f020b200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f0204162b60 con 0x7f02040aebb0 2024-04-20T10:22:04.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.796+0000 7f020b200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f02040b4dd0 con 0x7f02040ae0c0 2024-04-20T10:22:04.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.796+0000 7f0209200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f02040aebb0 0x7f020406b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:04.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.796+0000 7f0209200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f02040aebb0 0x7f020406b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43192/0 (socket says 172.21.15.121:43192) 2024-04-20T10:22:04.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.796+0000 7f0209200640 1 -- 172.21.15.121:0/976933805 learned_addr learned my addr 172.21.15.121:0/976933805 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:04.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.796+0000 7f0209200640 1 -- 172.21.15.121:0/976933805 >> v2:172.21.15.123:3300/0 conn(0x7f02040ae0c0 msgr2=0x7f02041621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:04.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.796+0000 7f0209200640 1 --2- 172.21.15.121:0/976933805 >> v2:172.21.15.123:3300/0 conn(0x7f02040ae0c0 0x7f02041621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:04.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.796+0000 7f0209200640 1 -- 172.21.15.121:0/976933805 >> v1:172.21.15.123:6789/0 conn(0x7f02041626f0 legacy=0x7f02040ae4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:04.798 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.796+0000 7f0209200640 1 -- 172.21.15.121:0/976933805 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02040b50b0 con 0x7f02040aebb0 2024-04-20T10:22:04.798 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.797+0000 7f0209200640 1 --2- 172.21.15.121:0/976933805 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02040aebb0 0x7f020406b900 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0204074450 tx=0x7f01fc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=810a47d58fbd541f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:04.798 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.798+0000 7f0203400640 1 -- 172.21.15.121:0/976933805 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01fc02ca80 con 0x7f02040aebb0 2024-04-20T10:22:04.798 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.798+0000 7f0203400640 1 -- 172.21.15.121:0/976933805 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01fc02cc20 con 0x7f02040aebb0 2024-04-20T10:22:04.798 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.798+0000 7f0203400640 1 -- 172.21.15.121:0/976933805 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01fc0343f0 con 0x7f02040aebb0 2024-04-20T10:22:04.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.798+0000 7f020b200640 1 -- 172.21.15.121:0/976933805 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02040aebb0 msgr2=0x7f020406b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:04.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.798+0000 7f020b200640 1 --2- 172.21.15.121:0/976933805 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02040aebb0 0x7f020406b900 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0204074450 tx=0x7f01fc02bee0 comp rx=0 tx=0).stop 2024-04-20T10:22:04.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.798+0000 7f020b200640 1 -- 172.21.15.121:0/976933805 shutdown_connections 2024-04-20T10:22:04.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.798+0000 7f020b200640 1 --2- 172.21.15.121:0/976933805 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02040aebb0 0x7f020406b900 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:04.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.798+0000 7f020b200640 1 --2- 172.21.15.121:0/976933805 >> v2:172.21.15.123:3300/0 conn(0x7f02040ae0c0 0x7f02041621b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:04.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.799+0000 7f020b200640 1 -- 172.21.15.121:0/976933805 >> 172.21.15.121:0/976933805 conn(0x7f02040d0560 msgr2=0x7f02040d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:04.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.799+0000 7f020b200640 1 -- 172.21.15.121:0/976933805 shutdown_connections 2024-04-20T10:22:04.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.799+0000 7f020b200640 1 -- 172.21.15.121:0/976933805 wait complete. 2024-04-20T10:22:04.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.800+0000 7f020b200640 1 Processor -- start 2024-04-20T10:22:04.801 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.800+0000 7f020b200640 1 -- start start 2024-04-20T10:22:04.803 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.801+0000 7f020b200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02040ae0c0 0x7f02040ce360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:04.803 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.801+0000 7f020b200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02040aebb0 0x7f0204145ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:04.803 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.801+0000 7f020b200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f02041626f0 0x7f0204145ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:04.803 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.801+0000 7f020b200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f02040b4710 con 0x7f02040aebb0 2024-04-20T10:22:04.803 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.801+0000 7f020b200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f02040b5670 con 0x7f02040ae0c0 2024-04-20T10:22:04.803 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.801+0000 7f020b200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f02040b4890 con 0x7f02041626f0 2024-04-20T10:22:04.803 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.802+0000 7f01fb400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02040aebb0 0x7f0204145ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:04.803 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.802+0000 7f01fb400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02040aebb0 0x7f0204145ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43204/0 (socket says 172.21.15.121:43204) 2024-04-20T10:22:04.803 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.802+0000 7f01fb400640 1 -- 172.21.15.121:0/2102459770 learned_addr learned my addr 172.21.15.121:0/2102459770 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:04.803 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.802+0000 7f0203e00640 1 --2- 172.21.15.121:0/2102459770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02040ae0c0 0x7f02040ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:04.803 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.802+0000 7f0209200640 1 --2- 172.21.15.121:0/2102459770 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f02041626f0 0x7f0204145ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:04.803 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.802+0000 7f01fb400640 1 -- 172.21.15.121:0/2102459770 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f02041626f0 msgr2=0x7f0204145ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:04.803 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.802+0000 7f01fb400640 1 --2- 172.21.15.121:0/2102459770 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f02041626f0 0x7f0204145ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:04.804 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.802+0000 7f01fb400640 1 -- 172.21.15.121:0/2102459770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02040ae0c0 msgr2=0x7f02040ce360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:04.804 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.802+0000 7f01fb400640 1 --2- 172.21.15.121:0/2102459770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02040ae0c0 0x7f02040ce360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:04.804 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.802+0000 7f01fb400640 1 -- 172.21.15.121:0/2102459770 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02040cd2c0 con 0x7f02040aebb0 2024-04-20T10:22:04.804 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.802+0000 7f01fb400640 1 --2- 172.21.15.121:0/2102459770 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02040aebb0 0x7f0204145ab0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00b410 tx=0x7f01ec00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:04.805 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.803+0000 7f0200c00640 1 -- 172.21.15.121:0/2102459770 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec015020 con 0x7f02040aebb0 2024-04-20T10:22:04.805 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.803+0000 7f0200c00640 1 -- 172.21.15.121:0/2102459770 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01ec011830 con 0x7f02040aebb0 2024-04-20T10:22:04.805 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.803+0000 7f020b200640 1 -- 172.21.15.121:0/2102459770 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02040cd5a0 con 0x7f02040aebb0 2024-04-20T10:22:04.805 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.803+0000 7f0200c00640 1 -- 172.21.15.121:0/2102459770 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec011c20 con 0x7f02040aebb0 2024-04-20T10:22:04.805 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.803+0000 7f020b200640 1 -- 172.21.15.121:0/2102459770 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02040cdb60 con 0x7f02040aebb0 2024-04-20T10:22:04.805 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.804+0000 7f020b200640 1 -- 172.21.15.121:0/2102459770 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f020406d150 con 0x7f02040aebb0 2024-04-20T10:22:04.805 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.805+0000 7f0200c00640 1 -- 172.21.15.121:0/2102459770 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f01ec018020 con 0x7f02040aebb0 2024-04-20T10:22:04.806 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.806+0000 7f0200c00640 1 --2- 172.21.15.121:0/2102459770 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f01e006c8f0 0x7f01e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:04.806 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.806+0000 7f0200c00640 1 -- 172.21.15.121:0/2102459770 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f01ec08be70 con 0x7f02040aebb0 2024-04-20T10:22:04.807 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.806+0000 7f0203e00640 1 --2- 172.21.15.121:0/2102459770 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f01e006c8f0 0x7f01e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:04.807 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.807+0000 7f0203e00640 1 --2- 172.21.15.121:0/2102459770 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f01e006c8f0 0x7f01e006edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f01f4009890 tx=0x7f01f4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:04.810 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:04.810+0000 7f0200c00640 1 -- 172.21.15.121:0/2102459770 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f020406d150 con 0x7f02040aebb0 2024-04-20T10:22:05.008 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.008+0000 7f020b200640 1 -- 172.21.15.121:0/2102459770 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f0204074700 con 0x7f02040aebb0 2024-04-20T10:22:05.009 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.009+0000 7f0200c00640 1 -- 172.21.15.121:0/2102459770 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f0204074700 con 0x7f02040aebb0 2024-04-20T10:22:05.009 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 32 2024-04-20T10:22:05.012 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.011+0000 7f020b200640 1 -- 172.21.15.121:0/2102459770 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f01e006c8f0 msgr2=0x7f01e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:05.012 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.011+0000 7f020b200640 1 --2- 172.21.15.121:0/2102459770 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f01e006c8f0 0x7f01e006edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f01f4009890 tx=0x7f01f4008040 comp rx=0 tx=0).stop 2024-04-20T10:22:05.012 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.011+0000 7f020b200640 1 -- 172.21.15.121:0/2102459770 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02040aebb0 msgr2=0x7f0204145ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:05.012 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.011+0000 7f020b200640 1 --2- 172.21.15.121:0/2102459770 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02040aebb0 0x7f0204145ab0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00b410 tx=0x7f01ec00b8d0 comp rx=0 tx=0).stop 2024-04-20T10:22:05.012 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.011+0000 7f020b200640 1 -- 172.21.15.121:0/2102459770 shutdown_connections 2024-04-20T10:22:05.012 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.011+0000 7f020b200640 1 --2- 172.21.15.121:0/2102459770 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f02041626f0 0x7f0204145ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:05.012 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.011+0000 7f020b200640 1 --2- 172.21.15.121:0/2102459770 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f02040aebb0 0x7f0204145ab0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:05.012 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.011+0000 7f020b200640 1 --2- 172.21.15.121:0/2102459770 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f01e006c8f0 0x7f01e006edb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:05.012 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.011+0000 7f020b200640 1 --2- 172.21.15.121:0/2102459770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02040ae0c0 0x7f02040ce360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:05.012 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.011+0000 7f020b200640 1 -- 172.21.15.121:0/2102459770 >> 172.21.15.121:0/2102459770 conn(0x7f02040d0560 msgr2=0x7f02040ae4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:05.012 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.011+0000 7f020b200640 1 -- 172.21.15.121:0/2102459770 shutdown_connections 2024-04-20T10:22:05.012 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.011+0000 7f020b200640 1 -- 172.21.15.121:0/2102459770 wait complete. 2024-04-20T10:22:05.029 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:22:05.211 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.211+0000 7fd442200640 1 Processor -- start 2024-04-20T10:22:05.212 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.211+0000 7fd442200640 1 -- start start 2024-04-20T10:22:05.213 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.213+0000 7fd442200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd43c161d50 0x7fd43c0d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:05.213 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.213+0000 7fd442200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd43c0b3330 0x7fd43c0b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:05.213 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.213+0000 7fd442200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd43c06c190 con 0x7fd43c06c6d0 2024-04-20T10:22:05.213 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.213+0000 7fd442200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fd43c162130 con 0x7fd43c161d50 2024-04-20T10:22:05.213 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.213+0000 7fd442200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fd43c06c470 con 0x7fd43c0b3330 2024-04-20T10:22:05.214 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.213+0000 7fd43be00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd43c0b3330 0x7fd43c0b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:05.214 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.213+0000 7fd440e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd43c161d50 0x7fd43c0d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:05.214 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.214+0000 7fd43be00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd43c0b3330 0x7fd43c0b3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.121:43058/0 (socket says 172.21.15.121:43058) 2024-04-20T10:22:05.214 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.214+0000 7fd440e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd43c161d50 0x7fd43c0d38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.121:34166/0 (socket says 172.21.15.121:34166) 2024-04-20T10:22:05.214 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.214+0000 7fd43be00640 1 -- 172.21.15.121:0/1174763844 learned_addr learned my addr 172.21.15.121:0/1174763844 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:05.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.214+0000 7fd43be00640 1 -- 172.21.15.121:0/1174763844 >> v2:172.21.15.123:3300/0 conn(0x7fd43c161d50 msgr2=0x7fd43c0d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:05.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.214+0000 7fd43be00640 1 --2- 172.21.15.121:0/1174763844 >> v2:172.21.15.123:3300/0 conn(0x7fd43c161d50 0x7fd43c0d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:05.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.214+0000 7fd43be00640 1 -- 172.21.15.121:0/1174763844 >> v1:172.21.15.123:6789/0 conn(0x7fd43c06c6d0 legacy=0x7fd43c0cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:05.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.214+0000 7fd43be00640 1 -- 172.21.15.121:0/1174763844 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd43c0d2e00 con 0x7fd43c0b3330 2024-04-20T10:22:05.216 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.216+0000 7fd43be00640 1 --2- 172.21.15.121:0/1174763844 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd43c0b3330 0x7fd43c0b3880 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd428009870 tx=0x7fd42802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25cc3b7c6f7a918f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:05.216 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.216+0000 7fd43b400640 1 -- 172.21.15.121:0/1174763844 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd428034070 con 0x7fd43c0b3330 2024-04-20T10:22:05.216 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.216+0000 7fd43b400640 1 -- 172.21.15.121:0/1174763844 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd428030840 con 0x7fd43c0b3330 2024-04-20T10:22:05.216 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.216+0000 7fd43b400640 1 -- 172.21.15.121:0/1174763844 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd428030c10 con 0x7fd43c0b3330 2024-04-20T10:22:05.217 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.217+0000 7fd442200640 1 -- 172.21.15.121:0/1174763844 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd43c0b3330 msgr2=0x7fd43c0b3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:05.217 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.217+0000 7fd442200640 1 --2- 172.21.15.121:0/1174763844 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd43c0b3330 0x7fd43c0b3880 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd428009870 tx=0x7fd42802b980 comp rx=0 tx=0).stop 2024-04-20T10:22:05.217 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.217+0000 7fd442200640 1 -- 172.21.15.121:0/1174763844 shutdown_connections 2024-04-20T10:22:05.217 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.217+0000 7fd442200640 1 --2- 172.21.15.121:0/1174763844 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd43c0b3330 0x7fd43c0b3880 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:05.217 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.217+0000 7fd442200640 1 --2- 172.21.15.121:0/1174763844 >> v2:172.21.15.123:3300/0 conn(0x7fd43c161d50 0x7fd43c0d38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:05.217 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.217+0000 7fd442200640 1 -- 172.21.15.121:0/1174763844 >> 172.21.15.121:0/1174763844 conn(0x7fd43c0d4450 msgr2=0x7fd43c0d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:05.218 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.217+0000 7fd442200640 1 -- 172.21.15.121:0/1174763844 shutdown_connections 2024-04-20T10:22:05.218 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.217+0000 7fd442200640 1 -- 172.21.15.121:0/1174763844 wait complete. 2024-04-20T10:22:05.219 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.218+0000 7fd442200640 1 Processor -- start 2024-04-20T10:22:05.219 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.219+0000 7fd442200640 1 -- start start 2024-04-20T10:22:05.220 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.219+0000 7fd442200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd43c06c6d0 0x7fd43c0cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:05.220 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.219+0000 7fd442200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd43c0b3330 0x7fd43c0cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:05.221 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.219+0000 7fd442200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd43c161d50 0x7fd43c1f4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:05.221 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.219+0000 7fd442200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fd43c06cb60 con 0x7fd43c06c6d0 2024-04-20T10:22:05.221 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.219+0000 7fd442200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd43c0d33c0 con 0x7fd43c161d50 2024-04-20T10:22:05.221 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.219+0000 7fd442200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fd43c06cce0 con 0x7fd43c0b3330 2024-04-20T10:22:05.221 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.220+0000 7fd440e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd43c06c6d0 0x7fd43c0cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:05.221 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.220+0000 7fd440e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd43c06c6d0 0x7fd43c0cdda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43220/0 (socket says 172.21.15.121:43220) 2024-04-20T10:22:05.221 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.220+0000 7fd440e00640 1 -- 172.21.15.121:0/2872227399 learned_addr learned my addr 172.21.15.121:0/2872227399 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:05.221 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.220+0000 7fd43be00640 1 --2- 172.21.15.121:0/2872227399 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd43c0b3330 0x7fd43c0cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:05.221 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.220+0000 7fd440e00640 1 -- 172.21.15.121:0/2872227399 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd43c0b3330 msgr2=0x7fd43c0cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:05.221 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.220+0000 7fd440e00640 1 --2- 172.21.15.121:0/2872227399 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd43c0b3330 0x7fd43c0cc2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:05.221 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.220+0000 7fd440e00640 1 -- 172.21.15.121:0/2872227399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd43c161d50 msgr2=0x7fd43c1f4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:05.221 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.220+0000 7fd441800640 1 --2- 172.21.15.121:0/2872227399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd43c161d50 0x7fd43c1f4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:05.221 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.220+0000 7fd440e00640 1 --2- 172.21.15.121:0/2872227399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd43c161d50 0x7fd43c1f4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:05.222 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.220+0000 7fd440e00640 1 -- 172.21.15.121:0/2872227399 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd43c0ccd20 con 0x7fd43c06c6d0 2024-04-20T10:22:05.222 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.220+0000 7fd440e00640 1 --2- 172.21.15.121:0/2872227399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd43c06c6d0 0x7fd43c0cdda0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd43000e870 tx=0x7fd43000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:05.223 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.221+0000 7fd438c00640 1 -- 172.21.15.121:0/2872227399 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd430017070 con 0x7fd43c06c6d0 2024-04-20T10:22:05.223 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.221+0000 7fd438c00640 1 -- 172.21.15.121:0/2872227399 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd43000c940 con 0x7fd43c06c6d0 2024-04-20T10:22:05.223 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.221+0000 7fd438c00640 1 -- 172.21.15.121:0/2872227399 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43000cd30 con 0x7fd43c06c6d0 2024-04-20T10:22:05.223 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.221+0000 7fd442200640 1 -- 172.21.15.121:0/2872227399 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd43c0cd000 con 0x7fd43c06c6d0 2024-04-20T10:22:05.223 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.221+0000 7fd442200640 1 -- 172.21.15.121:0/2872227399 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd43c0cd8a0 con 0x7fd43c06c6d0 2024-04-20T10:22:05.223 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.222+0000 7fd442200640 1 -- 172.21.15.121:0/2872227399 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd43c075b60 con 0x7fd43c06c6d0 2024-04-20T10:22:05.223 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.223+0000 7fd438c00640 1 -- 172.21.15.121:0/2872227399 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd43001a070 con 0x7fd43c06c6d0 2024-04-20T10:22:05.224 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.224+0000 7fd438c00640 1 --2- 172.21.15.121:0/2872227399 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fd41c06c8f0 0x7fd41c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:05.224 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.224+0000 7fd438c00640 1 -- 172.21.15.121:0/2872227399 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fd43008c1a0 con 0x7fd43c06c6d0 2024-04-20T10:22:05.224 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.224+0000 7fd43be00640 1 --2- 172.21.15.121:0/2872227399 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fd41c06c8f0 0x7fd41c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:05.225 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.225+0000 7fd43be00640 1 --2- 172.21.15.121:0/2872227399 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fd41c06c8f0 0x7fd41c06edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd428036000 tx=0x7fd42803b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:05.229 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.228+0000 7fd438c00640 1 -- 172.21.15.121:0/2872227399 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd43005a1d0 con 0x7fd43c06c6d0 2024-04-20T10:22:05.432 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.431+0000 7fd442200640 1 -- 172.21.15.121:0/2872227399 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd43c15ff80 con 0x7fd43c06c6d0 2024-04-20T10:22:05.433 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.433+0000 7fd438c00640 1 -- 172.21.15.121:0/2872227399 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18940 (secure 0 0 0) 0x7fd430059ad0 con 0x7fd43c06c6d0 2024-04-20T10:22:05.434 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:22:05.434 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":27,"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","created":"2024-04-20T10:20:38.076076+0000","modified":"2024-04-20T10:21:44.715898+0000","last_up_change":"2024-04-20T10:20:50.162700+0000","last_in_change":"2024-04-20T10:20:41.510382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:20:51.141365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:21:00.436937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:21:01.026957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:21:04.862693+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"629a7b8c-a40a-4a2a-a3cc-d9f357774310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6809","nonce":1044588835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6811","nonce":1044588835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6815","nonce":1044588835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6813","nonce":1044588835}]},"public_addr":"172.21.15.121:6809/1044588835","cluster_addr":"172.21.15.121:6811/1044588835","heartbeat_back_addr":"172.21.15.121:6815/1044588835","heartbeat_front_addr":"172.21.15.121:6813/1044588835","state":["exists","up"]},{"osd":1,"uuid":"f2570399-aa50-4ff3-bf5d-3d09207290db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6801","nonce":3505199624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6803","nonce":3505199624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6807","nonce":3505199624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6805","nonce":3505199624}]},"public_addr":"172.21.15.121:6801/3505199624","cluster_addr":"172.21.15.121:6803/3505199624","heartbeat_back_addr":"172.21.15.121:6807/3505199624","heartbeat_front_addr":"172.21.15.121:6805/3505199624","state":["exists","up"]},{"osd":2,"uuid":"50893807-dc30-49f4-b41f-739366626658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6817","nonce":3656724207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6819","nonce":3656724207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6827","nonce":3656724207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6822","nonce":3656724207}]},"public_addr":"172.21.15.121:6817/3656724207","cluster_addr":"172.21.15.121:6819/3656724207","heartbeat_back_addr":"172.21.15.121:6827/3656724207","heartbeat_front_addr":"172.21.15.121:6822/3656724207","state":["exists","up"]},{"osd":3,"uuid":"dadb01bc-5491-4ffc-8824-d48a057240b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6823","nonce":2232342832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6826","nonce":2232342832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6831","nonce":2232342832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6829","nonce":2232342832}]},"public_addr":"172.21.15.121:6823/2232342832","cluster_addr":"172.21.15.121:6826/2232342832","heartbeat_back_addr":"172.21.15.121:6831/2232342832","heartbeat_front_addr":"172.21.15.121:6829/2232342832","state":["exists","up"]},{"osd":4,"uuid":"b3f3b4a1-b375-4526-b69e-344ab24c2da1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6801","nonce":99353111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6803","nonce":99353111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6807","nonce":99353111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6805","nonce":99353111}]},"public_addr":"172.21.15.123:6801/99353111","cluster_addr":"172.21.15.123:6803/99353111","heartbeat_back_addr":"172.21.15.123:6807/99353111","heartbeat_front_addr":"172.21.15.123:6805/99353111","state":["exists","up"]},{"osd":5,"uuid":"2ee24a08-1624-4961-9642-d7b510282484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6818","nonce":2504374688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6822","nonce":2504374688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6830","nonce":2504374688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6826","nonce":2504374688}]},"public_addr":"172.21.15.123:6818/2504374688","cluster_addr":"172.21.15.123:6822/2504374688","heartbeat_back_addr":"172.21.15.123:6830/2504374688","heartbeat_front_addr":"172.21.15.123:6826/2504374688","state":["exists","up"]},{"osd":6,"uuid":"186118cc-6128-49de-acca-55260945c1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6819","nonce":4139299599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6823","nonce":4139299599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6829","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6831","nonce":4139299599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6827","nonce":4139299599}]},"public_addr":"172.21.15.123:6819/4139299599","cluster_addr":"172.21.15.123:6823/4139299599","heartbeat_back_addr":"172.21.15.123:6831/4139299599","heartbeat_front_addr":"172.21.15.123:6827/4139299599","state":["exists","up"]},{"osd":7,"uuid":"a00aac6e-4bd9-415a-a93d-ae85210865ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6809","nonce":4043981069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6811","nonce":4043981069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6815","nonce":4043981069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6813","nonce":4043981069}]},"public_addr":"172.21.15.123:6809/4043981069","cluster_addr":"172.21.15.123:6811/4043981069","heartbeat_back_addr":"172.21.15.123:6815/4043981069","heartbeat_front_addr":"172.21.15.123:6813/4043981069","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.904540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.958585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.616367+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.885244+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.103127+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.167248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.884348+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.862356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.23","mappings":[{"from":3,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:22:05.436 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.436+0000 7fd442200640 1 -- 172.21.15.121:0/2872227399 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fd41c06c8f0 msgr2=0x7fd41c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:05.436 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.436+0000 7fd442200640 1 --2- 172.21.15.121:0/2872227399 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fd41c06c8f0 0x7fd41c06edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd428036000 tx=0x7fd42803b040 comp rx=0 tx=0).stop 2024-04-20T10:22:05.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.436+0000 7fd442200640 1 -- 172.21.15.121:0/2872227399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd43c06c6d0 msgr2=0x7fd43c0cdda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:05.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.436+0000 7fd442200640 1 --2- 172.21.15.121:0/2872227399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd43c06c6d0 0x7fd43c0cdda0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd43000e870 tx=0x7fd43000ed30 comp rx=0 tx=0).stop 2024-04-20T10:22:05.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.436+0000 7fd442200640 1 -- 172.21.15.121:0/2872227399 shutdown_connections 2024-04-20T10:22:05.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.436+0000 7fd442200640 1 --2- 172.21.15.121:0/2872227399 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fd41c06c8f0 0x7fd41c06edb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:05.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.436+0000 7fd442200640 1 --2- 172.21.15.121:0/2872227399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd43c161d50 0x7fd43c1f4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:05.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.436+0000 7fd442200640 1 --2- 172.21.15.121:0/2872227399 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fd43c0b3330 0x7fd43c0cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:05.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.436+0000 7fd442200640 1 --2- 172.21.15.121:0/2872227399 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fd43c06c6d0 0x7fd43c0cdda0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:05.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.436+0000 7fd442200640 1 -- 172.21.15.121:0/2872227399 >> 172.21.15.121:0/2872227399 conn(0x7fd43c0d4450 msgr2=0x7fd43c0b7740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:05.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.436+0000 7fd442200640 1 -- 172.21.15.121:0/2872227399 shutdown_connections 2024-04-20T10:22:05.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.436+0000 7fd442200640 1 -- 172.21.15.121:0/2872227399 wait complete. 2024-04-20T10:22:05.455 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:20:51.141365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:21:00.436937+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:21:01.026957+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T10:21:04.862693+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T10:22:05.456 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:22:05.660 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.655+0000 7f45ff200640 1 Processor -- start 2024-04-20T10:22:05.660 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.655+0000 7f45ff200640 1 -- start start 2024-04-20T10:22:05.660 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.657+0000 7f45ff200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46000ceeb0 0x7f460016d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:05.661 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.657+0000 7f45ff200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f460006b9a0 con 0x7f46000ca740 2024-04-20T10:22:05.661 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.657+0000 7f45ff200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f460006bc10 con 0x7f46001629b0 2024-04-20T10:22:05.661 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.657+0000 7f45ff200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f460006c540 con 0x7f46000ceeb0 2024-04-20T10:22:05.661 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.657+0000 7f45f7e00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f46000ca740 0x7f46000acf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:40546/0 (socket says 172.21.15.121:40546) 2024-04-20T10:22:05.661 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.657+0000 7f45f7e00640 1 -- 172.21.15.121:0/741224673 learned_addr learned my addr 172.21.15.121:0/741224673 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:05.661 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.658+0000 7f45fe800640 1 --2- 172.21.15.121:0/741224673 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46000ceeb0 0x7f460016d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:05.661 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.658+0000 7f45fd400640 1 -- 172.21.15.121:0/741224673 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 283219226 0 0) 0x7f460006b9a0 con 0x7f46000ca740 2024-04-20T10:22:05.661 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.658+0000 7f45fd400640 1 -- 172.21.15.121:0/741224673 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45e4003520 con 0x7f46000ca740 2024-04-20T10:22:05.661 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.658+0000 7f45fd400640 1 -- 172.21.15.121:0/741224673 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 71477891 0 0) 0x7f45e4003520 con 0x7f46000ca740 2024-04-20T10:22:05.661 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.658+0000 7f45fd400640 1 -- 172.21.15.121:0/741224673 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46000ceeb0 msgr2=0x7f460016d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:05.661 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.658+0000 7f45fd400640 1 --2- 172.21.15.121:0/741224673 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46000ceeb0 0x7f460016d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:05.661 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.658+0000 7f45fd400640 1 -- 172.21.15.121:0/741224673 >> v1:172.21.15.123:6789/0 conn(0x7f46001629b0 legacy=0x7f46000cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:05.661 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.658+0000 7f45fd400640 1 -- 172.21.15.121:0/741224673 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f460006bef0 con 0x7f46000ca740 2024-04-20T10:22:05.661 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.658+0000 7f45fd400640 1 -- 172.21.15.121:0/741224673 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f45e8001670 con 0x7f46000ca740 2024-04-20T10:22:05.662 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.658+0000 7f45fd400640 1 -- 172.21.15.121:0/741224673 >> v1:172.21.15.121:6789/0 conn(0x7f46000ca740 legacy=0x7f46000acf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:05.662 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.658+0000 7f45fd400640 1 --2- 172.21.15.121:0/741224673 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46000ceeb0 0x7f45e40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:05.662 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.658+0000 7f45fd400640 1 --2- 172.21.15.121:0/741224673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45e4003fb0 0x7f45e4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:05.662 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.658+0000 7f45fd400640 1 --2- 172.21.15.121:0/741224673 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45e4004d40 0x7f45e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:05.662 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.658+0000 7f45fd400640 1 -- 172.21.15.121:0/741224673 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f460006c540 con 0x7f45e4004d40 2024-04-20T10:22:05.662 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.658+0000 7f45fd400640 1 -- 172.21.15.121:0/741224673 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f45e400bb00 con 0x7f45e4003fb0 2024-04-20T10:22:05.662 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.658+0000 7f45fd400640 1 -- 172.21.15.121:0/741224673 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f45e400bd00 con 0x7f46000ceeb0 2024-04-20T10:22:05.662 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.659+0000 7f45fe800640 1 --2- 172.21.15.121:0/741224673 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45e4004d40 0x7f45e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:05.662 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.659+0000 7f45fde00640 1 --2- 172.21.15.121:0/741224673 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46000ceeb0 0x7f45e40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:05.662 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.659+0000 7f45f7e00640 1 --2- 172.21.15.121:0/741224673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45e4003fb0 0x7f45e4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:05.662 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.659+0000 7f45fe800640 1 -- 172.21.15.121:0/741224673 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46000ceeb0 msgr2=0x7f45e40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:05.662 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.659+0000 7f45fe800640 1 --2- 172.21.15.121:0/741224673 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46000ceeb0 0x7f45e40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:05.662 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.659+0000 7f45fe800640 1 -- 172.21.15.121:0/741224673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45e4003fb0 msgr2=0x7f45e4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:05.663 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.659+0000 7f45fe800640 1 --2- 172.21.15.121:0/741224673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45e4003fb0 0x7f45e4004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:05.663 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.659+0000 7f45fe800640 1 -- 172.21.15.121:0/741224673 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45e4003520 con 0x7f45e4004d40 2024-04-20T10:22:05.663 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.660+0000 7f45fe800640 1 --2- 172.21.15.121:0/741224673 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45e4004d40 0x7f45e40051c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f45f8009fc0 tx=0x7f45e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa0eae704b85d0a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:05.664 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.660+0000 7f45fd400640 1 -- 172.21.15.121:0/741224673 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f8039070 con 0x7f45e4004d40 2024-04-20T10:22:05.664 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.660+0000 7f45fd400640 1 -- 172.21.15.121:0/741224673 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45f802cc20 con 0x7f45e4004d40 2024-04-20T10:22:05.664 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.660+0000 7f45fd400640 1 -- 172.21.15.121:0/741224673 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f8009c10 con 0x7f45e4004d40 2024-04-20T10:22:05.664 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.661+0000 7f45ff200640 1 -- 172.21.15.121:0/741224673 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45e4004d40 msgr2=0x7f45e40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:05.664 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.661+0000 7f45ff200640 1 --2- 172.21.15.121:0/741224673 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45e4004d40 0x7f45e40051c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f45f8009fc0 tx=0x7f45e4005830 comp rx=0 tx=0).stop 2024-04-20T10:22:05.664 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.661+0000 7f45fe800640 1 -- 172.21.15.121:0/741224673 reap_dead start 2024-04-20T10:22:05.664 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.661+0000 7f45ff200640 1 -- 172.21.15.121:0/741224673 shutdown_connections 2024-04-20T10:22:05.664 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.661+0000 7f45ff200640 1 -- 172.21.15.121:0/741224673 >> 172.21.15.121:0/741224673 conn(0x7f46000cffa0 msgr2=0x7f46000d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:05.664 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.661+0000 7f45ff200640 1 -- 172.21.15.121:0/741224673 shutdown_connections 2024-04-20T10:22:05.664 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.661+0000 7f45ff200640 1 -- 172.21.15.121:0/741224673 wait complete. 2024-04-20T10:22:05.665 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.663+0000 7f45ff200640 1 Processor -- start 2024-04-20T10:22:05.665 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.663+0000 7f45ff200640 1 -- start start 2024-04-20T10:22:05.666 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.664+0000 7f45ff200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45e4004d40 0x7f46001f8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:05.666 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.664+0000 7f45ff200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f460015b4e0 0x7f460009cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:05.666 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.664+0000 7f45ff200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46000b50d0 0x7f46000b5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:05.666 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.664+0000 7f45ff200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f46000cfd00 con 0x7f45e4004d40 2024-04-20T10:22:05.667 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.664+0000 7f45ff200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4600162d90 con 0x7f460015b4e0 2024-04-20T10:22:05.667 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.664+0000 7f45ff200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f46000ad660 con 0x7f46000b50d0 2024-04-20T10:22:05.667 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.664+0000 7f45fde00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45e4004d40 0x7f46001f8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:05.667 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.664+0000 7f45fde00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45e4004d40 0x7f46001f8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43248/0 (socket says 172.21.15.121:43248) 2024-04-20T10:22:05.667 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.664+0000 7f45fde00640 1 -- 172.21.15.121:0/606875962 learned_addr learned my addr 172.21.15.121:0/606875962 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:05.667 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.665+0000 7f45fde00640 1 -- 172.21.15.121:0/606875962 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46000b50d0 msgr2=0x7f46000b5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:05.667 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.665+0000 7f45fde00640 1 --2- 172.21.15.121:0/606875962 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46000b50d0 0x7f46000b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:05.667 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.665+0000 7f45fde00640 1 -- 172.21.15.121:0/606875962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f460015b4e0 msgr2=0x7f460009cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:05.667 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.665+0000 7f45fde00640 1 --2- 172.21.15.121:0/606875962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f460015b4e0 0x7f460009cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:05.667 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.665+0000 7f45fde00640 1 -- 172.21.15.121:0/606875962 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f460009d090 con 0x7f45e4004d40 2024-04-20T10:22:05.667 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.665+0000 7f45fde00640 1 --2- 172.21.15.121:0/606875962 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45e4004d40 0x7f46001f8410 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f45f000e520 tx=0x7f45f000e9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:05.667 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.665+0000 7f45f6000640 1 -- 172.21.15.121:0/606875962 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f00052e0 con 0x7f45e4004d40 2024-04-20T10:22:05.667 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.665+0000 7f45f6000640 1 -- 172.21.15.121:0/606875962 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45f0018070 con 0x7f45e4004d40 2024-04-20T10:22:05.667 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.665+0000 7f45f6000640 1 -- 172.21.15.121:0/606875962 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f0003830 con 0x7f45e4004d40 2024-04-20T10:22:05.668 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.665+0000 7f45ff200640 1 -- 172.21.15.121:0/606875962 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46001f8a60 con 0x7f45e4004d40 2024-04-20T10:22:05.668 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.665+0000 7f45ff200640 1 -- 172.21.15.121:0/606875962 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46001f9020 con 0x7f45e4004d40 2024-04-20T10:22:05.668 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.666+0000 7f45ff200640 1 -- 172.21.15.121:0/606875962 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4600075b30 con 0x7f45e4004d40 2024-04-20T10:22:05.669 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.667+0000 7f45f6000640 1 -- 172.21.15.121:0/606875962 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f45f00039d0 con 0x7f45e4004d40 2024-04-20T10:22:05.669 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.668+0000 7f45f6000640 1 --2- 172.21.15.121:0/606875962 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f45dc06c920 0x7f45dc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:05.669 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.668+0000 7f45f6000640 1 -- 172.21.15.121:0/606875962 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f45f008ba70 con 0x7f45e4004d40 2024-04-20T10:22:05.669 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.668+0000 7f45f7e00640 1 --2- 172.21.15.121:0/606875962 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f45dc06c920 0x7f45dc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:05.669 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.669+0000 7f45f7e00640 1 --2- 172.21.15.121:0/606875962 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f45dc06c920 0x7f45dc06ede0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f46000cc510 tx=0x7f45e8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:05.673 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.673+0000 7f45f6000640 1 -- 172.21.15.121:0/606875962 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f45f0059b30 con 0x7f45e4004d40 2024-04-20T10:22:05.856 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.855+0000 7f45ff200640 1 -- 172.21.15.121:0/606875962 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f46000cc180 con 0x7f45e4004d40 2024-04-20T10:22:05.857 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.856+0000 7f45f6000640 1 -- 172.21.15.121:0/606875962 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f45f0059430 con 0x7f45e4004d40 2024-04-20T10:22:05.857 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2024-04-20T10:22:05.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.859+0000 7f45ff200640 1 -- 172.21.15.121:0/606875962 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f45dc06c920 msgr2=0x7f45dc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:05.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.859+0000 7f45ff200640 1 --2- 172.21.15.121:0/606875962 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f45dc06c920 0x7f45dc06ede0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f46000cc510 tx=0x7f45e8009040 comp rx=0 tx=0).stop 2024-04-20T10:22:05.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.859+0000 7f45ff200640 1 -- 172.21.15.121:0/606875962 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45e4004d40 msgr2=0x7f46001f8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:05.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.859+0000 7f45ff200640 1 --2- 172.21.15.121:0/606875962 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45e4004d40 0x7f46001f8410 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f45f000e520 tx=0x7f45f000e9e0 comp rx=0 tx=0).stop 2024-04-20T10:22:05.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.859+0000 7f45ff200640 1 -- 172.21.15.121:0/606875962 shutdown_connections 2024-04-20T10:22:05.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.859+0000 7f45ff200640 1 --2- 172.21.15.121:0/606875962 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f46000b50d0 0x7f46000b5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:05.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.859+0000 7f45ff200640 1 --2- 172.21.15.121:0/606875962 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f45dc06c920 0x7f45dc06ede0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:05.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.859+0000 7f45ff200640 1 --2- 172.21.15.121:0/606875962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f460015b4e0 0x7f460009cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:05.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.859+0000 7f45ff200640 1 --2- 172.21.15.121:0/606875962 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45e4004d40 0x7f46001f8410 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:05.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.859+0000 7f45ff200640 1 -- 172.21.15.121:0/606875962 >> 172.21.15.121:0/606875962 conn(0x7f46000cffa0 msgr2=0x7f4600072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:05.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.859+0000 7f45ff200640 1 -- 172.21.15.121:0/606875962 shutdown_connections 2024-04-20T10:22:05.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:05.859+0000 7f45ff200640 1 -- 172.21.15.121:0/606875962 wait complete. 2024-04-20T10:22:05.878 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:22:06.075 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.074+0000 7f9b3ee00640 1 Processor -- start 2024-04-20T10:22:06.075 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.074+0000 7f9b3ee00640 1 -- start start 2024-04-20T10:22:06.077 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.076+0000 7f9b3ee00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9b380d1a00 0x7f9b380b1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:06.077 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.076+0000 7f9b3ee00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b380d1e90 con 0x7f9b380d4270 2024-04-20T10:22:06.077 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.076+0000 7f9b3ee00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b380d3820 con 0x7f9b380b17d0 2024-04-20T10:22:06.077 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.076+0000 7f9b3ee00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f9b380d40c0 con 0x7f9b380d1a00 2024-04-20T10:22:06.077 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.076+0000 7f9b37e00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f9b380d4270 0x7f9b380d4650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:40552/0 (socket says 172.21.15.121:40552) 2024-04-20T10:22:06.077 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.076+0000 7f9b37e00640 1 -- 172.21.15.121:0/275326513 learned_addr learned my addr 172.21.15.121:0/275326513 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:06.079 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.077+0000 7f9b36a00640 1 -- 172.21.15.121:0/275326513 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2366917260 0 0) 0x7f9b28001490 con 0x7f9b380d4270 2024-04-20T10:22:06.079 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.077+0000 7f9b36a00640 1 -- 172.21.15.121:0/275326513 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b1c003520 con 0x7f9b380d4270 2024-04-20T10:22:06.079 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.077+0000 7f9b37400640 1 --2- 172.21.15.121:0/275326513 >> v2:172.21.15.123:3300/0 conn(0x7f9b380d1a00 0x7f9b380b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:06.079 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.077+0000 7f9b36a00640 1 -- 172.21.15.121:0/275326513 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2935960366 0 0) 0x7f9b1c003520 con 0x7f9b380d4270 2024-04-20T10:22:06.079 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.077+0000 7f9b36a00640 1 -- 172.21.15.121:0/275326513 >> v2:172.21.15.123:3300/0 conn(0x7f9b380d1a00 msgr2=0x7f9b380b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:06.079 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.077+0000 7f9b36a00640 1 --2- 172.21.15.121:0/275326513 >> v2:172.21.15.123:3300/0 conn(0x7f9b380d1a00 0x7f9b380b1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:06.079 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.077+0000 7f9b36a00640 1 -- 172.21.15.121:0/275326513 >> v1:172.21.15.123:6789/0 conn(0x7f9b380b17d0 legacy=0x7f9b380d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:06.079 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.077+0000 7f9b36a00640 1 -- 172.21.15.121:0/275326513 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b380d3b00 con 0x7f9b380d4270 2024-04-20T10:22:06.079 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.077+0000 7f9b36a00640 1 -- 172.21.15.121:0/275326513 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f9b28001bc0 con 0x7f9b380d4270 2024-04-20T10:22:06.080 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.077+0000 7f9b36a00640 1 -- 172.21.15.121:0/275326513 >> v1:172.21.15.121:6789/0 conn(0x7f9b380d4270 legacy=0x7f9b380d4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:06.080 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.077+0000 7f9b36a00640 1 --2- 172.21.15.121:0/275326513 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b1c003a00 0x7f9b1c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:06.080 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.077+0000 7f9b36a00640 1 --2- 172.21.15.121:0/275326513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b1c004800 0x7f9b1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:06.080 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.077+0000 7f9b36a00640 1 --2- 172.21.15.121:0/275326513 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9b1c005210 0x7f9b1c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:06.080 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.077+0000 7f9b36a00640 1 -- 172.21.15.121:0/275326513 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9b380d40c0 con 0x7f9b1c003a00 2024-04-20T10:22:06.080 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.077+0000 7f9b36a00640 1 -- 172.21.15.121:0/275326513 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1c00c140 con 0x7f9b1c004800 2024-04-20T10:22:06.080 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.077+0000 7f9b36a00640 1 -- 172.21.15.121:0/275326513 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f9b1c00c340 con 0x7f9b1c005210 2024-04-20T10:22:06.080 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.078+0000 7f9b37e00640 1 --2- 172.21.15.121:0/275326513 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b1c003a00 0x7f9b1c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:06.080 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.078+0000 7f9b37400640 1 --2- 172.21.15.121:0/275326513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b1c004800 0x7f9b1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:06.080 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.078+0000 7f9b37e00640 1 -- 172.21.15.121:0/275326513 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9b1c005210 msgr2=0x7f9b1c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:06.080 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.078+0000 7f9b37e00640 1 --2- 172.21.15.121:0/275326513 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9b1c005210 0x7f9b1c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:06.080 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.078+0000 7f9b37e00640 1 -- 172.21.15.121:0/275326513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b1c004800 msgr2=0x7f9b1c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:06.080 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.078+0000 7f9b37e00640 1 --2- 172.21.15.121:0/275326513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b1c004800 0x7f9b1c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:06.081 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.078+0000 7f9b37e00640 1 -- 172.21.15.121:0/275326513 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b1c003520 con 0x7f9b1c003a00 2024-04-20T10:22:06.081 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.078+0000 7f9b3ce00640 1 -- 172.21.15.121:0/275326513 reap_dead start 2024-04-20T10:22:06.082 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.079+0000 7f9b37e00640 1 --2- 172.21.15.121:0/275326513 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b1c003a00 0x7f9b1c003e00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9b2800a970 tx=0x7f9b1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f23b587e637c4f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:06.082 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.079+0000 7f9b36a00640 1 -- 172.21.15.121:0/275326513 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2800a5c0 con 0x7f9b1c003a00 2024-04-20T10:22:06.082 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.079+0000 7f9b36a00640 1 -- 172.21.15.121:0/275326513 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b28034630 con 0x7f9b1c003a00 2024-04-20T10:22:06.082 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.079+0000 7f9b36a00640 1 -- 172.21.15.121:0/275326513 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b28034a60 con 0x7f9b1c003a00 2024-04-20T10:22:06.082 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.080+0000 7f9b3ee00640 1 -- 172.21.15.121:0/275326513 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b1c003a00 msgr2=0x7f9b1c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:06.082 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.080+0000 7f9b3ee00640 1 --2- 172.21.15.121:0/275326513 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b1c003a00 0x7f9b1c003e00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9b2800a970 tx=0x7f9b1c005830 comp rx=0 tx=0).stop 2024-04-20T10:22:06.082 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.080+0000 7f9b3ee00640 1 -- 172.21.15.121:0/275326513 shutdown_connections 2024-04-20T10:22:06.082 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.080+0000 7f9b3ee00640 1 --2- 172.21.15.121:0/275326513 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b1c003a00 0x7f9b1c003e00 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:06.082 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.080+0000 7f9b3ee00640 1 -- 172.21.15.121:0/275326513 >> 172.21.15.121:0/275326513 conn(0x7f9b380b99c0 msgr2=0x7f9b380b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:06.082 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.080+0000 7f9b3ee00640 1 -- 172.21.15.121:0/275326513 shutdown_connections 2024-04-20T10:22:06.082 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.080+0000 7f9b3ee00640 1 -- 172.21.15.121:0/275326513 wait complete. 2024-04-20T10:22:06.083 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.082+0000 7f9b3ee00640 1 Processor -- start 2024-04-20T10:22:06.083 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.082+0000 7f9b3ee00640 1 -- start start 2024-04-20T10:22:06.085 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.083+0000 7f9b3ee00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9b1c003a00 0x7f9b38150b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:06.085 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.083+0000 7f9b3ee00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b3806b900 0x7f9b380cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:06.085 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.083+0000 7f9b3ee00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b380b3120 0x7f9b380b35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:06.085 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.083+0000 7f9b3ee00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9b380d53b0 con 0x7f9b380b3120 2024-04-20T10:22:06.085 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.083+0000 7f9b3ee00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9b380cfe60 con 0x7f9b3806b900 2024-04-20T10:22:06.085 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.083+0000 7f9b3ee00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f9b38161f70 con 0x7f9b1c003a00 2024-04-20T10:22:06.085 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.083+0000 7f9b3ce00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b380b3120 0x7f9b380b35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:06.085 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.083+0000 7f9b3ce00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b380b3120 0x7f9b380b35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:43268/0 (socket says 172.21.15.121:43268) 2024-04-20T10:22:06.085 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.083+0000 7f9b3ce00640 1 -- 172.21.15.121:0/2261277441 learned_addr learned my addr 172.21.15.121:0/2261277441 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:06.085 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.084+0000 7f9b37400640 1 --2- 172.21.15.121:0/2261277441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b3806b900 0x7f9b380cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:06.086 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.084+0000 7f9b37e00640 1 --2- 172.21.15.121:0/2261277441 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9b1c003a00 0x7f9b38150b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:06.086 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.084+0000 7f9b3ce00640 1 -- 172.21.15.121:0/2261277441 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9b1c003a00 msgr2=0x7f9b38150b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:06.086 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.084+0000 7f9b3ce00640 1 --2- 172.21.15.121:0/2261277441 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9b1c003a00 0x7f9b38150b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:06.086 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.084+0000 7f9b3ce00640 1 -- 172.21.15.121:0/2261277441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b3806b900 msgr2=0x7f9b380cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:06.086 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.084+0000 7f9b3ce00640 1 --2- 172.21.15.121:0/2261277441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b3806b900 0x7f9b380cdf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:06.086 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.084+0000 7f9b3ce00640 1 -- 172.21.15.121:0/2261277441 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b3806be50 con 0x7f9b380b3120 2024-04-20T10:22:06.086 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.084+0000 7f9b37400640 1 --2- 172.21.15.121:0/2261277441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b3806b900 0x7f9b380cdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:06.086 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.084+0000 7f9b37e00640 1 --2- 172.21.15.121:0/2261277441 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9b1c003a00 0x7f9b38150b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:06.086 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.084+0000 7f9b3ce00640 1 --2- 172.21.15.121:0/2261277441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b380b3120 0x7f9b380b35a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9b380cebe0 tx=0x7f9b3000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:06.086 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.084+0000 7f9b2fe00640 1 -- 172.21.15.121:0/2261277441 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b30007480 con 0x7f9b380b3120 2024-04-20T10:22:06.086 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.084+0000 7f9b2fe00640 1 -- 172.21.15.121:0/2261277441 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b300057c0 con 0x7f9b380b3120 2024-04-20T10:22:06.086 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.084+0000 7f9b2fe00640 1 -- 172.21.15.121:0/2261277441 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b30007640 con 0x7f9b380b3120 2024-04-20T10:22:06.086 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.084+0000 7f9b3ee00640 1 -- 172.21.15.121:0/2261277441 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b3806c130 con 0x7f9b380b3120 2024-04-20T10:22:06.087 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.085+0000 7f9b3ee00640 1 -- 172.21.15.121:0/2261277441 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b3806c6f0 con 0x7f9b380b3120 2024-04-20T10:22:06.087 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.086+0000 7f9b3ee00640 1 -- 172.21.15.121:0/2261277441 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b38068980 con 0x7f9b380b3120 2024-04-20T10:22:06.087 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.087+0000 7f9b2fe00640 1 -- 172.21.15.121:0/2261277441 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b30011070 con 0x7f9b380b3120 2024-04-20T10:22:06.088 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.088+0000 7f9b2fe00640 1 --2- 172.21.15.121:0/2261277441 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9b1406c8f0 0x7f9b1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:06.088 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.088+0000 7f9b2fe00640 1 -- 172.21.15.121:0/2261277441 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f9b3008ad20 con 0x7f9b380b3120 2024-04-20T10:22:06.088 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.088+0000 7f9b37e00640 1 --2- 172.21.15.121:0/2261277441 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9b1406c8f0 0x7f9b1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:06.089 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.089+0000 7f9b37e00640 1 --2- 172.21.15.121:0/2261277441 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9b1406c8f0 0x7f9b1406edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c0057e0 tx=0x7f9b2802f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:06.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.092+0000 7f9b2fe00640 1 -- 172.21.15.121:0/2261277441 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9b30058cb0 con 0x7f9b380b3120 2024-04-20T10:22:06.291 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.291+0000 7f9b3ee00640 1 -- 172.21.15.121:0/2261277441 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9b380d0420 con 0x7f9b380b3120 2024-04-20T10:22:06.292 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.292+0000 7f9b2fe00640 1 -- 172.21.15.121:0/2261277441 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f9b300585b0 con 0x7f9b380b3120 2024-04-20T10:22:06.292 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 32 2024-04-20T10:22:06.295 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.294+0000 7f9b3ee00640 1 -- 172.21.15.121:0/2261277441 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9b1406c8f0 msgr2=0x7f9b1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:06.295 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.294+0000 7f9b3ee00640 1 --2- 172.21.15.121:0/2261277441 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9b1406c8f0 0x7f9b1406edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c0057e0 tx=0x7f9b2802f000 comp rx=0 tx=0).stop 2024-04-20T10:22:06.295 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.294+0000 7f9b3ee00640 1 -- 172.21.15.121:0/2261277441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b380b3120 msgr2=0x7f9b380b35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:06.295 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.294+0000 7f9b3ee00640 1 --2- 172.21.15.121:0/2261277441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b380b3120 0x7f9b380b35a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9b380cebe0 tx=0x7f9b3000c820 comp rx=0 tx=0).stop 2024-04-20T10:22:06.296 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.294+0000 7f9b3ee00640 1 -- 172.21.15.121:0/2261277441 shutdown_connections 2024-04-20T10:22:06.296 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.294+0000 7f9b3ee00640 1 --2- 172.21.15.121:0/2261277441 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9b380b3120 0x7f9b380b35a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:06.296 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.294+0000 7f9b3ee00640 1 --2- 172.21.15.121:0/2261277441 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9b1406c8f0 0x7f9b1406edb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:06.296 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.294+0000 7f9b3ee00640 1 --2- 172.21.15.121:0/2261277441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9b3806b900 0x7f9b380cdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:06.296 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.294+0000 7f9b3ee00640 1 --2- 172.21.15.121:0/2261277441 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9b1c003a00 0x7f9b38150b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:06.296 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.294+0000 7f9b3ee00640 1 -- 172.21.15.121:0/2261277441 >> 172.21.15.121:0/2261277441 conn(0x7f9b380b99c0 msgr2=0x7f9b38160670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:06.296 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.295+0000 7f9b3ee00640 1 -- 172.21.15.121:0/2261277441 shutdown_connections 2024-04-20T10:22:06.296 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.295+0000 7f9b3ee00640 1 -- 172.21.15.121:0/2261277441 wait complete. 2024-04-20T10:22:06.314 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:22:06.512 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.511+0000 7f7257200640 1 Processor -- start 2024-04-20T10:22:06.512 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.512+0000 7f7257200640 1 -- start start 2024-04-20T10:22:06.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.513+0000 7f7257200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f725815a130 0x7f7258073470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:06.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.513+0000 7f7257200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7258071a60 0x7f72580739b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:06.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.513+0000 7f7257200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f725806f160 0x7f725806f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:06.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.513+0000 7f7257200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f725815d7f0 con 0x7f725815a130 2024-04-20T10:22:06.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.513+0000 7f7257200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f725815cc20 con 0x7f7258071a60 2024-04-20T10:22:06.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.513+0000 7f7257200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f725815cec0 con 0x7f725806f160 2024-04-20T10:22:06.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.514+0000 7f7255e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f725815a130 0x7f7258073470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:06.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.514+0000 7f7255e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f725815a130 0x7f7258073470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42364/0 (socket says 172.21.15.121:42364) 2024-04-20T10:22:06.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.514+0000 7f7255e00640 1 -- 172.21.15.121:0/1720413924 learned_addr learned my addr 172.21.15.121:0/1720413924 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:06.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.514+0000 7f724d400640 1 --2- 172.21.15.121:0/1720413924 >> v2:172.21.15.123:3300/0 conn(0x7f7258071a60 0x7f72580739b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:06.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.514+0000 7f7256800640 1 --2- 172.21.15.121:0/1720413924 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f725806f160 0x7f725806f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:06.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.514+0000 7f7255e00640 1 -- 172.21.15.121:0/1720413924 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f725806f160 msgr2=0x7f725806f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:06.516 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.514+0000 7f7255e00640 1 --2- 172.21.15.121:0/1720413924 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f725806f160 0x7f725806f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:06.516 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.514+0000 7f7255e00640 1 -- 172.21.15.121:0/1720413924 >> v2:172.21.15.123:3300/0 conn(0x7f7258071a60 msgr2=0x7f72580739b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:06.516 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.514+0000 7f7255e00640 1 --2- 172.21.15.121:0/1720413924 >> v2:172.21.15.123:3300/0 conn(0x7f7258071a60 0x7f72580739b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:06.516 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.514+0000 7f7255e00640 1 -- 172.21.15.121:0/1720413924 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f725815d1a0 con 0x7f725815a130 2024-04-20T10:22:06.516 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.514+0000 7f7256800640 1 --2- 172.21.15.121:0/1720413924 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f725806f160 0x7f725806f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:06.516 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.515+0000 7f7255e00640 1 --2- 172.21.15.121:0/1720413924 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f725815a130 0x7f7258073470 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f725806cfe0 tx=0x7f724802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46e7bfd11626b4be server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:06.516 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.516+0000 7f7255400640 1 -- 172.21.15.121:0/1720413924 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724802e070 con 0x7f725815a130 2024-04-20T10:22:06.517 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.516+0000 7f7255400640 1 -- 172.21.15.121:0/1720413924 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7248037070 con 0x7f725815a130 2024-04-20T10:22:06.517 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.516+0000 7f7255400640 1 -- 172.21.15.121:0/1720413924 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7248031a00 con 0x7f725815a130 2024-04-20T10:22:06.517 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.517+0000 7f7257200640 1 -- 172.21.15.121:0/1720413924 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f725815a130 msgr2=0x7f7258073470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:06.518 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.517+0000 7f7257200640 1 --2- 172.21.15.121:0/1720413924 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f725815a130 0x7f7258073470 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f725806cfe0 tx=0x7f724802b880 comp rx=0 tx=0).stop 2024-04-20T10:22:06.518 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.517+0000 7f7257200640 1 -- 172.21.15.121:0/1720413924 shutdown_connections 2024-04-20T10:22:06.518 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.517+0000 7f7257200640 1 --2- 172.21.15.121:0/1720413924 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f725806f160 0x7f725806f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:06.518 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.517+0000 7f7257200640 1 --2- 172.21.15.121:0/1720413924 >> v2:172.21.15.123:3300/0 conn(0x7f7258071a60 0x7f72580739b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:06.518 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.517+0000 7f7257200640 1 --2- 172.21.15.121:0/1720413924 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f725815a130 0x7f7258073470 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:06.518 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.517+0000 7f7257200640 1 -- 172.21.15.121:0/1720413924 >> 172.21.15.121:0/1720413924 conn(0x7f72580cffa0 msgr2=0x7f72580d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:06.518 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.517+0000 7f7257200640 1 -- 172.21.15.121:0/1720413924 shutdown_connections 2024-04-20T10:22:06.518 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.517+0000 7f7257200640 1 -- 172.21.15.121:0/1720413924 wait complete. 2024-04-20T10:22:06.519 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.519+0000 7f7257200640 1 Processor -- start 2024-04-20T10:22:06.519 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.519+0000 7f7257200640 1 -- start start 2024-04-20T10:22:06.522 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.520+0000 7f7257200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f725806f160 0x7f725815ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:06.522 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.520+0000 7f7257200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7258071a60 0x7f72581f7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:06.522 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.520+0000 7f7257200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f725815a130 0x7f72581f8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:06.522 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.520+0000 7f7257200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7258160100 con 0x7f725815a130 2024-04-20T10:22:06.522 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.520+0000 7f7257200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f725806fb30 con 0x7f7258071a60 2024-04-20T10:22:06.522 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.520+0000 7f7257200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f7258160280 con 0x7f725806f160 2024-04-20T10:22:06.522 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.520+0000 7f7256800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f725815a130 0x7f72581f8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:06.522 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.520+0000 7f7256800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f725815a130 0x7f72581f8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42366/0 (socket says 172.21.15.121:42366) 2024-04-20T10:22:06.522 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.520+0000 7f7256800640 1 -- 172.21.15.121:0/3810423906 learned_addr learned my addr 172.21.15.121:0/3810423906 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:06.522 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.520+0000 7f7255e00640 1 --2- 172.21.15.121:0/3810423906 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f725806f160 0x7f725815ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:06.522 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.520+0000 7f724d400640 1 --2- 172.21.15.121:0/3810423906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7258071a60 0x7f72581f7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:06.523 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.521+0000 7f7256800640 1 -- 172.21.15.121:0/3810423906 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f725806f160 msgr2=0x7f725815ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:06.523 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.521+0000 7f7256800640 1 --2- 172.21.15.121:0/3810423906 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f725806f160 0x7f725815ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:06.523 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.521+0000 7f7256800640 1 -- 172.21.15.121:0/3810423906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7258071a60 msgr2=0x7f72581f7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:06.523 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.521+0000 7f7256800640 1 --2- 172.21.15.121:0/3810423906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7258071a60 0x7f72581f7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:06.523 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.521+0000 7f7256800640 1 -- 172.21.15.121:0/3810423906 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72581f45b0 con 0x7f725815a130 2024-04-20T10:22:06.523 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.521+0000 7f724d400640 1 --2- 172.21.15.121:0/3810423906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7258071a60 0x7f72581f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:06.523 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.521+0000 7f7256800640 1 --2- 172.21.15.121:0/3810423906 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f725815a130 0x7f72581f8280 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f725000cef0 tx=0x7f725000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:06.523 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.521+0000 7f724ea00640 1 -- 172.21.15.121:0/3810423906 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725001a070 con 0x7f725815a130 2024-04-20T10:22:06.523 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.521+0000 7f724ea00640 1 -- 172.21.15.121:0/3810423906 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f725000d860 con 0x7f725815a130 2024-04-20T10:22:06.523 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.521+0000 7f724ea00640 1 -- 172.21.15.121:0/3810423906 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725000dc50 con 0x7f725815a130 2024-04-20T10:22:06.523 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.521+0000 7f7257200640 1 -- 172.21.15.121:0/3810423906 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f725815dbe0 con 0x7f725815a130 2024-04-20T10:22:06.523 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.521+0000 7f7257200640 1 -- 172.21.15.121:0/3810423906 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f725815e1a0 con 0x7f725815a130 2024-04-20T10:22:06.524 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.523+0000 7f7257200640 1 -- 172.21.15.121:0/3810423906 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7258075b30 con 0x7f725815a130 2024-04-20T10:22:06.524 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.524+0000 7f724ea00640 1 -- 172.21.15.121:0/3810423906 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f725001d070 con 0x7f725815a130 2024-04-20T10:22:06.525 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.524+0000 7f724ea00640 1 --2- 172.21.15.121:0/3810423906 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f723406c8f0 0x7f723406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:06.525 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.525+0000 7f724ea00640 1 -- 172.21.15.121:0/3810423906 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f725008c750 con 0x7f725815a130 2024-04-20T10:22:06.525 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.525+0000 7f7255e00640 1 --2- 172.21.15.121:0/3810423906 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f723406c8f0 0x7f723406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:06.526 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.525+0000 7f7255e00640 1 --2- 172.21.15.121:0/3810423906 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f723406c8f0 0x7f723406edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7248036000 tx=0x7f724803d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:06.529 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.529+0000 7f724ea00640 1 -- 172.21.15.121:0/3810423906 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f725005a8d0 con 0x7f725815a130 2024-04-20T10:22:06.734 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.734+0000 7f7257200640 1 -- 172.21.15.121:0/3810423906 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f72581e7cc0 con 0x7f725815a130 2024-04-20T10:22:06.735 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.734+0000 7f724ea00640 1 -- 172.21.15.121:0/3810423906 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f725005a1d0 con 0x7f725815a130 2024-04-20T10:22:06.735 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2024-04-20T10:22:06.738 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.737+0000 7f7257200640 1 -- 172.21.15.121:0/3810423906 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f723406c8f0 msgr2=0x7f723406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:06.738 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.737+0000 7f7257200640 1 --2- 172.21.15.121:0/3810423906 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f723406c8f0 0x7f723406edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7248036000 tx=0x7f724803d040 comp rx=0 tx=0).stop 2024-04-20T10:22:06.739 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.737+0000 7f7257200640 1 -- 172.21.15.121:0/3810423906 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f725815a130 msgr2=0x7f72581f8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:06.739 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.737+0000 7f7257200640 1 --2- 172.21.15.121:0/3810423906 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f725815a130 0x7f72581f8280 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f725000cef0 tx=0x7f725000a710 comp rx=0 tx=0).stop 2024-04-20T10:22:06.739 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.738+0000 7f7257200640 1 -- 172.21.15.121:0/3810423906 shutdown_connections 2024-04-20T10:22:06.739 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.738+0000 7f7257200640 1 --2- 172.21.15.121:0/3810423906 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f725815a130 0x7f72581f8280 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:06.739 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.738+0000 7f7257200640 1 --2- 172.21.15.121:0/3810423906 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f723406c8f0 0x7f723406edb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:06.739 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.738+0000 7f7257200640 1 --2- 172.21.15.121:0/3810423906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7258071a60 0x7f72581f7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:06.739 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.738+0000 7f7257200640 1 --2- 172.21.15.121:0/3810423906 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f725806f160 0x7f725815ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:06.739 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.738+0000 7f7257200640 1 -- 172.21.15.121:0/3810423906 >> 172.21.15.121:0/3810423906 conn(0x7f72580cffa0 msgr2=0x7f7258154980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:06.739 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.738+0000 7f7257200640 1 -- 172.21.15.121:0/3810423906 shutdown_connections 2024-04-20T10:22:06.739 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.738+0000 7f7257200640 1 -- 172.21.15.121:0/3810423906 wait complete. 2024-04-20T10:22:06.757 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T10:22:06.960 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.959+0000 7f82d6000640 1 Processor -- start 2024-04-20T10:22:06.960 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.960+0000 7f82d6000640 1 -- start start 2024-04-20T10:22:06.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.961+0000 7f82d6000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82d0156c20 0x7f82d006b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:06.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.961+0000 7f82d6000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f82d015ca40 0x7f82d00ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:06.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.961+0000 7f82d6000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f82d00ad8e0 0x7f82d00d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:06.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.961+0000 7f82d6000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f82d0141c60 con 0x7f82d00ad8e0 2024-04-20T10:22:06.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.961+0000 7f82d6000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f82d015cf90 con 0x7f82d015ca40 2024-04-20T10:22:06.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.961+0000 7f82d6000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f82d015d270 con 0x7f82d0156c20 2024-04-20T10:22:06.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.962+0000 7f82cfe00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f82d00ad8e0 0x7f82d00d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:06.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.962+0000 7f82cfe00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f82d00ad8e0 0x7f82d00d0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42370/0 (socket says 172.21.15.121:42370) 2024-04-20T10:22:06.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.962+0000 7f82cfe00640 1 -- 172.21.15.121:0/2852308210 learned_addr learned my addr 172.21.15.121:0/2852308210 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:06.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.962+0000 7f82c6a00640 1 --2- 172.21.15.121:0/2852308210 >> v2:172.21.15.123:3300/0 conn(0x7f82d015ca40 0x7f82d00ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:06.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.962+0000 7f82cf400640 1 --2- 172.21.15.121:0/2852308210 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82d0156c20 0x7f82d006b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:06.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.962+0000 7f82cfe00640 1 -- 172.21.15.121:0/2852308210 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82d0156c20 msgr2=0x7f82d006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:06.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.962+0000 7f82cfe00640 1 --2- 172.21.15.121:0/2852308210 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82d0156c20 0x7f82d006b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:06.964 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.962+0000 7f82cfe00640 1 -- 172.21.15.121:0/2852308210 >> v2:172.21.15.123:3300/0 conn(0x7f82d015ca40 msgr2=0x7f82d00ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:06.964 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.962+0000 7f82cfe00640 1 --2- 172.21.15.121:0/2852308210 >> v2:172.21.15.123:3300/0 conn(0x7f82d015ca40 0x7f82d00ad3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:06.964 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.962+0000 7f82cfe00640 1 -- 172.21.15.121:0/2852308210 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d015d550 con 0x7f82d00ad8e0 2024-04-20T10:22:06.964 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.962+0000 7f82cf400640 1 --2- 172.21.15.121:0/2852308210 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82d0156c20 0x7f82d006b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:22:06.964 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.963+0000 7f82cfe00640 1 --2- 172.21.15.121:0/2852308210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82d00ad8e0 0x7f82d00d0360 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f82d006fe70 tx=0x7f82c802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84719fc6ed9d1646 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:06.964 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.964+0000 7f82cea00640 1 -- 172.21.15.121:0/2852308210 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c802ca70 con 0x7f82d00ad8e0 2024-04-20T10:22:06.964 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.964+0000 7f82cea00640 1 -- 172.21.15.121:0/2852308210 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82c802cc10 con 0x7f82d00ad8e0 2024-04-20T10:22:06.965 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.964+0000 7f82cea00640 1 -- 172.21.15.121:0/2852308210 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c80343f0 con 0x7f82d00ad8e0 2024-04-20T10:22:06.965 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.965+0000 7f82d6000640 1 -- 172.21.15.121:0/2852308210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82d00ad8e0 msgr2=0x7f82d00d0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:06.965 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.965+0000 7f82d6000640 1 --2- 172.21.15.121:0/2852308210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82d00ad8e0 0x7f82d00d0360 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f82d006fe70 tx=0x7f82c802bed0 comp rx=0 tx=0).stop 2024-04-20T10:22:06.965 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.965+0000 7f82d6000640 1 -- 172.21.15.121:0/2852308210 shutdown_connections 2024-04-20T10:22:06.966 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.965+0000 7f82d6000640 1 --2- 172.21.15.121:0/2852308210 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82d00ad8e0 0x7f82d00d0360 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:06.966 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.965+0000 7f82d6000640 1 --2- 172.21.15.121:0/2852308210 >> v2:172.21.15.123:3300/0 conn(0x7f82d015ca40 0x7f82d00ad3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:06.966 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.965+0000 7f82d6000640 1 --2- 172.21.15.121:0/2852308210 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82d0156c20 0x7f82d006b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:06.966 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.965+0000 7f82d6000640 1 -- 172.21.15.121:0/2852308210 >> 172.21.15.121:0/2852308210 conn(0x7f82d00b3350 msgr2=0x7f82d00b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:06.966 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.965+0000 7f82d6000640 1 -- 172.21.15.121:0/2852308210 shutdown_connections 2024-04-20T10:22:06.966 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.965+0000 7f82d6000640 1 -- 172.21.15.121:0/2852308210 wait complete. 2024-04-20T10:22:06.967 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.967+0000 7f82d6000640 1 Processor -- start 2024-04-20T10:22:06.967 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.967+0000 7f82d6000640 1 -- start start 2024-04-20T10:22:06.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.968+0000 7f82d6000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82d00ad8e0 0x7f82d00999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:06.968 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.968+0000 7f82d6000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82d0156c20 0x7f82d006f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:06.969 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.968+0000 7f82d6000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82d015ca40 0x7f82d0155180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:06.969 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.968+0000 7f82d6000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f82d006bfc0 con 0x7f82d015ca40 2024-04-20T10:22:06.969 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.968+0000 7f82d6000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f82d006be40 con 0x7f82d0156c20 2024-04-20T10:22:06.969 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.968+0000 7f82d6000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f82d00d08a0 con 0x7f82d00ad8e0 2024-04-20T10:22:06.969 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.968+0000 7f82cfe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82d015ca40 0x7f82d0155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:06.969 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.968+0000 7f82cfe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82d015ca40 0x7f82d0155180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42378/0 (socket says 172.21.15.121:42378) 2024-04-20T10:22:06.969 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.968+0000 7f82cfe00640 1 -- 172.21.15.121:0/3232774331 learned_addr learned my addr 172.21.15.121:0/3232774331 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:06.970 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.968+0000 7f82cfe00640 1 -- 172.21.15.121:0/3232774331 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82d00ad8e0 msgr2=0x7f82d00999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:06.970 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.968+0000 7f82cfe00640 1 --2- 172.21.15.121:0/3232774331 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82d00ad8e0 0x7f82d00999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:06.970 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.968+0000 7f82cfe00640 1 -- 172.21.15.121:0/3232774331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82d0156c20 msgr2=0x7f82d006f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:06.970 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.968+0000 7f82cfe00640 1 --2- 172.21.15.121:0/3232774331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82d0156c20 0x7f82d006f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:06.971 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.968+0000 7f82cfe00640 1 -- 172.21.15.121:0/3232774331 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d006dc50 con 0x7f82d015ca40 2024-04-20T10:22:06.971 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.969+0000 7f82cfe00640 1 --2- 172.21.15.121:0/3232774331 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82d015ca40 0x7f82d0155180 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f82c8036040 tx=0x7f82c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:06.971 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.969+0000 7f82c7e00640 1 -- 172.21.15.121:0/3232774331 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c8043020 con 0x7f82d015ca40 2024-04-20T10:22:06.971 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.969+0000 7f82c7e00640 1 -- 172.21.15.121:0/3232774331 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82c803e450 con 0x7f82d015ca40 2024-04-20T10:22:06.971 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.969+0000 7f82d6000640 1 -- 172.21.15.121:0/3232774331 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82d006def0 con 0x7f82d015ca40 2024-04-20T10:22:06.971 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.969+0000 7f82c7e00640 1 -- 172.21.15.121:0/3232774331 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c8009510 con 0x7f82d015ca40 2024-04-20T10:22:06.971 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.969+0000 7f82d6000640 1 -- 172.21.15.121:0/3232774331 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82d006e4b0 con 0x7f82d015ca40 2024-04-20T10:22:06.971 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.970+0000 7f82d6000640 1 -- 172.21.15.121:0/3232774331 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82d0068980 con 0x7f82d015ca40 2024-04-20T10:22:06.972 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.971+0000 7f82c7e00640 1 -- 172.21.15.121:0/3232774331 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f82c8046020 con 0x7f82d015ca40 2024-04-20T10:22:06.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.973+0000 7f82c7e00640 1 --2- 172.21.15.121:0/3232774331 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f82ac06c920 0x7f82ac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:06.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.973+0000 7f82c7e00640 1 -- 172.21.15.121:0/3232774331 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f82c80ae450 con 0x7f82d015ca40 2024-04-20T10:22:06.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.973+0000 7f82cf400640 1 --2- 172.21.15.121:0/3232774331 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f82ac06c920 0x7f82ac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:06.974 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.974+0000 7f82cf400640 1 --2- 172.21.15.121:0/3232774331 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f82ac06c920 0x7f82ac06ede0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f82c00096f0 tx=0x7f82c0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:06.978 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:06.977+0000 7f82c7e00640 1 -- 172.21.15.121:0/3232774331 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f82c804d070 con 0x7f82d015ca40 2024-04-20T10:22:07.174 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.173+0000 7f82d6000640 1 -- 172.21.15.121:0/3232774331 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f82d00b3830 con 0x7f82d015ca40 2024-04-20T10:22:07.174 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.174+0000 7f82c7e00640 1 -- 172.21.15.121:0/3232774331 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f82c807c470 con 0x7f82d015ca40 2024-04-20T10:22:07.175 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 32 2024-04-20T10:22:07.177 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.176+0000 7f82d6000640 1 -- 172.21.15.121:0/3232774331 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f82ac06c920 msgr2=0x7f82ac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:07.177 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.176+0000 7f82d6000640 1 --2- 172.21.15.121:0/3232774331 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f82ac06c920 0x7f82ac06ede0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f82c00096f0 tx=0x7f82c0008040 comp rx=0 tx=0).stop 2024-04-20T10:22:07.177 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.176+0000 7f82d6000640 1 -- 172.21.15.121:0/3232774331 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82d015ca40 msgr2=0x7f82d0155180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:07.177 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.176+0000 7f82d6000640 1 --2- 172.21.15.121:0/3232774331 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82d015ca40 0x7f82d0155180 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f82c8036040 tx=0x7f82c8004a40 comp rx=0 tx=0).stop 2024-04-20T10:22:07.177 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.176+0000 7f82d6000640 1 -- 172.21.15.121:0/3232774331 shutdown_connections 2024-04-20T10:22:07.177 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.176+0000 7f82d6000640 1 --2- 172.21.15.121:0/3232774331 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82d015ca40 0x7f82d0155180 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:07.177 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.176+0000 7f82d6000640 1 --2- 172.21.15.121:0/3232774331 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f82ac06c920 0x7f82ac06ede0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:07.177 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.176+0000 7f82d6000640 1 --2- 172.21.15.121:0/3232774331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82d0156c20 0x7f82d006f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:07.178 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.176+0000 7f82d6000640 1 --2- 172.21.15.121:0/3232774331 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82d00ad8e0 0x7f82d00999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:07.178 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.176+0000 7f82d6000640 1 -- 172.21.15.121:0/3232774331 >> 172.21.15.121:0/3232774331 conn(0x7f82d00b3350 msgr2=0x7f82d006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:07.178 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.177+0000 7f82d6000640 1 -- 172.21.15.121:0/3232774331 shutdown_connections 2024-04-20T10:22:07.178 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.177+0000 7f82d6000640 1 -- 172.21.15.121:0/3232774331 wait complete. 2024-04-20T10:22:07.206 INFO:tasks.cephfs_test_runner:Starting test: test_generic (tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev) 2024-04-20T10:22:07.206 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:22:07.403 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.402+0000 7f3c91c00640 1 Processor -- start 2024-04-20T10:22:07.403 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.403+0000 7f3c91c00640 1 -- start start 2024-04-20T10:22:07.405 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.404+0000 7f3c91c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3c8c161d50 0x7f3c8c0d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:07.405 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.404+0000 7f3c91c00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3c8c0b3330 0x7f3c8c0b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:07.406 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.404+0000 7f3c91c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3c8c06c6d0 0x7f3c8c06cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:07.406 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.404+0000 7f3c91c00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f3c8c162130 con 0x7f3c8c06c6d0 2024-04-20T10:22:07.406 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.404+0000 7f3c91c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f3c8c06c190 con 0x7f3c8c161d50 2024-04-20T10:22:07.406 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.404+0000 7f3c91c00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f3c8c06c470 con 0x7f3c8c0b3330 2024-04-20T10:22:07.406 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.404+0000 7f3c91200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3c8c06c6d0 0x7f3c8c06cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:07.406 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.405+0000 7f3c91200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3c8c06c6d0 0x7f3c8c06cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42392/0 (socket says 172.21.15.121:42392) 2024-04-20T10:22:07.406 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.405+0000 7f3c91200640 1 -- 172.21.15.121:0/4040200694 learned_addr learned my addr 172.21.15.121:0/4040200694 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:07.406 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.405+0000 7f3c8be00640 1 --2- 172.21.15.121:0/4040200694 >> v2:172.21.15.123:3300/0 conn(0x7f3c8c161d50 0x7f3c8c0d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:07.406 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.405+0000 7f3c8b400640 1 --2- 172.21.15.121:0/4040200694 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3c8c0b3330 0x7f3c8c0b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:07.406 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.405+0000 7f3c91200640 1 -- 172.21.15.121:0/4040200694 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3c8c0b3330 msgr2=0x7f3c8c0b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:07.406 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.405+0000 7f3c91200640 1 --2- 172.21.15.121:0/4040200694 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3c8c0b3330 0x7f3c8c0b3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:07.406 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.405+0000 7f3c91200640 1 -- 172.21.15.121:0/4040200694 >> v2:172.21.15.123:3300/0 conn(0x7f3c8c161d50 msgr2=0x7f3c8c0d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:07.407 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.405+0000 7f3c91200640 1 --2- 172.21.15.121:0/4040200694 >> v2:172.21.15.123:3300/0 conn(0x7f3c8c161d50 0x7f3c8c0d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:07.407 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.405+0000 7f3c91200640 1 -- 172.21.15.121:0/4040200694 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c8c0d2e00 con 0x7f3c8c06c6d0 2024-04-20T10:22:07.409 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.406+0000 7f3c91200640 1 --2- 172.21.15.121:0/4040200694 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c8c06c6d0 0x7f3c8c06cb60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c0b6f50 tx=0x7f3c8402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14e145394f8ac631 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:07.409 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.406+0000 7f3c8aa00640 1 -- 172.21.15.121:0/4040200694 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8402ca70 con 0x7f3c8c06c6d0 2024-04-20T10:22:07.409 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.406+0000 7f3c8aa00640 1 -- 172.21.15.121:0/4040200694 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c8402cc10 con 0x7f3c8c06c6d0 2024-04-20T10:22:07.409 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.406+0000 7f3c8aa00640 1 -- 172.21.15.121:0/4040200694 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c840343f0 con 0x7f3c8c06c6d0 2024-04-20T10:22:07.409 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.407+0000 7f3c91c00640 1 -- 172.21.15.121:0/4040200694 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c8c06c6d0 msgr2=0x7f3c8c06cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:07.410 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.407+0000 7f3c91c00640 1 --2- 172.21.15.121:0/4040200694 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c8c06c6d0 0x7f3c8c06cb60 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c0b6f50 tx=0x7f3c8402bed0 comp rx=0 tx=0).stop 2024-04-20T10:22:07.410 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.407+0000 7f3c91c00640 1 -- 172.21.15.121:0/4040200694 shutdown_connections 2024-04-20T10:22:07.410 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.407+0000 7f3c91c00640 1 --2- 172.21.15.121:0/4040200694 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c8c06c6d0 0x7f3c8c06cb60 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:07.410 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.407+0000 7f3c91c00640 1 --2- 172.21.15.121:0/4040200694 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3c8c0b3330 0x7f3c8c0b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:07.410 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.407+0000 7f3c91c00640 1 --2- 172.21.15.121:0/4040200694 >> v2:172.21.15.123:3300/0 conn(0x7f3c8c161d50 0x7f3c8c0d38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:07.411 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.407+0000 7f3c91c00640 1 -- 172.21.15.121:0/4040200694 >> 172.21.15.121:0/4040200694 conn(0x7f3c8c0d4450 msgr2=0x7f3c8c0d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:07.411 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.407+0000 7f3c91c00640 1 -- 172.21.15.121:0/4040200694 shutdown_connections 2024-04-20T10:22:07.411 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.407+0000 7f3c91c00640 1 -- 172.21.15.121:0/4040200694 wait complete. 2024-04-20T10:22:07.416 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.409+0000 7f3c91c00640 1 Processor -- start 2024-04-20T10:22:07.417 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.409+0000 7f3c91c00640 1 -- start start 2024-04-20T10:22:07.417 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.410+0000 7f3c91c00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3c8c06c6d0 0x7f3c8c0ca750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:07.417 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.410+0000 7f3c91c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c8c0b3330 0x7f3c8c0cd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:07.417 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.410+0000 7f3c91c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c8c161d50 0x7f3c8c14f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:07.417 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.410+0000 7f3c91c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8c0cfa00 con 0x7f3c8c0b3330 2024-04-20T10:22:07.418 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.410+0000 7f3c91c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8c0d33c0 con 0x7f3c8c161d50 2024-04-20T10:22:07.418 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.410+0000 7f3c91c00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f3c8c0cfb80 con 0x7f3c8c06c6d0 2024-04-20T10:22:07.418 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.410+0000 7f3c8b400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c8c0b3330 0x7f3c8c0cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:07.418 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.410+0000 7f3c8b400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c8c0b3330 0x7f3c8c0cd850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42396/0 (socket says 172.21.15.121:42396) 2024-04-20T10:22:07.418 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.410+0000 7f3c8b400640 1 -- 172.21.15.121:0/1789346931 learned_addr learned my addr 172.21.15.121:0/1789346931 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:07.418 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.411+0000 7f3c8b400640 1 -- 172.21.15.121:0/1789346931 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3c8c06c6d0 msgr2=0x7f3c8c0ca750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:07.419 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.411+0000 7f3c8b400640 1 --2- 172.21.15.121:0/1789346931 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3c8c06c6d0 0x7f3c8c0ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:07.419 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.411+0000 7f3c91200640 1 --2- 172.21.15.121:0/1789346931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c8c161d50 0x7f3c8c14f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:07.419 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.411+0000 7f3c8b400640 1 -- 172.21.15.121:0/1789346931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c8c161d50 msgr2=0x7f3c8c14f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:07.419 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.411+0000 7f3c8b400640 1 --2- 172.21.15.121:0/1789346931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c8c161d50 0x7f3c8c14f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:07.419 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.411+0000 7f3c8b400640 1 -- 172.21.15.121:0/1789346931 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c8c0c98b0 con 0x7f3c8c0b3330 2024-04-20T10:22:07.420 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.411+0000 7f3c8b400640 1 --2- 172.21.15.121:0/1789346931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c8c0b3330 0x7f3c8c0cd850 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3c7400d830 tx=0x7f3c7400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:07.420 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.411+0000 7f3c83e00640 1 -- 172.21.15.121:0/1789346931 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c74018020 con 0x7f3c8c0b3330 2024-04-20T10:22:07.420 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.411+0000 7f3c83e00640 1 -- 172.21.15.121:0/1789346931 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c740096a0 con 0x7f3c8c0b3330 2024-04-20T10:22:07.420 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.411+0000 7f3c83e00640 1 -- 172.21.15.121:0/1789346931 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c74009a70 con 0x7f3c8c0b3330 2024-04-20T10:22:07.420 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.411+0000 7f3c91c00640 1 -- 172.21.15.121:0/1789346931 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c8c0cac90 con 0x7f3c8c0b3330 2024-04-20T10:22:07.421 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.411+0000 7f3c91c00640 1 -- 172.21.15.121:0/1789346931 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c8c0c9010 con 0x7f3c8c0b3330 2024-04-20T10:22:07.421 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.413+0000 7f3c91c00640 1 -- 172.21.15.121:0/1789346931 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c8c075b60 con 0x7f3c8c0b3330 2024-04-20T10:22:07.421 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.414+0000 7f3c83e00640 1 -- 172.21.15.121:0/1789346931 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3c7401b020 con 0x7f3c8c0b3330 2024-04-20T10:22:07.421 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.414+0000 7f3c83e00640 1 --2- 172.21.15.121:0/1789346931 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f3c6806c8f0 0x7f3c6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:07.421 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.415+0000 7f3c83e00640 1 -- 172.21.15.121:0/1789346931 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f3c7408c310 con 0x7f3c8c0b3330 2024-04-20T10:22:07.421 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.415+0000 7f3c8be00640 1 --2- 172.21.15.121:0/1789346931 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f3c6806c8f0 0x7f3c6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:07.422 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.416+0000 7f3c8be00640 1 --2- 172.21.15.121:0/1789346931 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f3c6806c8f0 0x7f3c6806edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c006fd0 tx=0x7f3c7c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:07.422 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.419+0000 7f3c83e00640 1 -- 172.21.15.121:0/1789346931 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3c74022030 con 0x7f3c8c0b3330 2024-04-20T10:22:07.627 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.627+0000 7f3c91c00640 1 -- 172.21.15.121:0/1789346931 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3c8c0d49d0 con 0x7f3c8c0b3330 2024-04-20T10:22:07.629 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.628+0000 7f3c83e00640 1 -- 172.21.15.121:0/1789346931 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18940 (secure 0 0 0) 0x7f3c740102e0 con 0x7f3c8c0b3330 2024-04-20T10:22:07.629 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:22:07.630 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":27,"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","created":"2024-04-20T10:20:38.076076+0000","modified":"2024-04-20T10:21:44.715898+0000","last_up_change":"2024-04-20T10:20:50.162700+0000","last_in_change":"2024-04-20T10:20:41.510382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:20:51.141365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:21:00.436937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:21:01.026957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:21:04.862693+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"629a7b8c-a40a-4a2a-a3cc-d9f357774310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6809","nonce":1044588835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6811","nonce":1044588835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6815","nonce":1044588835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6813","nonce":1044588835}]},"public_addr":"172.21.15.121:6809/1044588835","cluster_addr":"172.21.15.121:6811/1044588835","heartbeat_back_addr":"172.21.15.121:6815/1044588835","heartbeat_front_addr":"172.21.15.121:6813/1044588835","state":["exists","up"]},{"osd":1,"uuid":"f2570399-aa50-4ff3-bf5d-3d09207290db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6801","nonce":3505199624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6803","nonce":3505199624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6807","nonce":3505199624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6805","nonce":3505199624}]},"public_addr":"172.21.15.121:6801/3505199624","cluster_addr":"172.21.15.121:6803/3505199624","heartbeat_back_addr":"172.21.15.121:6807/3505199624","heartbeat_front_addr":"172.21.15.121:6805/3505199624","state":["exists","up"]},{"osd":2,"uuid":"50893807-dc30-49f4-b41f-739366626658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6817","nonce":3656724207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6819","nonce":3656724207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6827","nonce":3656724207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6822","nonce":3656724207}]},"public_addr":"172.21.15.121:6817/3656724207","cluster_addr":"172.21.15.121:6819/3656724207","heartbeat_back_addr":"172.21.15.121:6827/3656724207","heartbeat_front_addr":"172.21.15.121:6822/3656724207","state":["exists","up"]},{"osd":3,"uuid":"dadb01bc-5491-4ffc-8824-d48a057240b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6823","nonce":2232342832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6826","nonce":2232342832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6831","nonce":2232342832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6829","nonce":2232342832}]},"public_addr":"172.21.15.121:6823/2232342832","cluster_addr":"172.21.15.121:6826/2232342832","heartbeat_back_addr":"172.21.15.121:6831/2232342832","heartbeat_front_addr":"172.21.15.121:6829/2232342832","state":["exists","up"]},{"osd":4,"uuid":"b3f3b4a1-b375-4526-b69e-344ab24c2da1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6801","nonce":99353111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6803","nonce":99353111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6807","nonce":99353111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6805","nonce":99353111}]},"public_addr":"172.21.15.123:6801/99353111","cluster_addr":"172.21.15.123:6803/99353111","heartbeat_back_addr":"172.21.15.123:6807/99353111","heartbeat_front_addr":"172.21.15.123:6805/99353111","state":["exists","up"]},{"osd":5,"uuid":"2ee24a08-1624-4961-9642-d7b510282484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6818","nonce":2504374688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6822","nonce":2504374688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6830","nonce":2504374688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6826","nonce":2504374688}]},"public_addr":"172.21.15.123:6818/2504374688","cluster_addr":"172.21.15.123:6822/2504374688","heartbeat_back_addr":"172.21.15.123:6830/2504374688","heartbeat_front_addr":"172.21.15.123:6826/2504374688","state":["exists","up"]},{"osd":6,"uuid":"186118cc-6128-49de-acca-55260945c1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6819","nonce":4139299599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6823","nonce":4139299599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6829","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6831","nonce":4139299599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6827","nonce":4139299599}]},"public_addr":"172.21.15.123:6819/4139299599","cluster_addr":"172.21.15.123:6823/4139299599","heartbeat_back_addr":"172.21.15.123:6831/4139299599","heartbeat_front_addr":"172.21.15.123:6827/4139299599","state":["exists","up"]},{"osd":7,"uuid":"a00aac6e-4bd9-415a-a93d-ae85210865ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6809","nonce":4043981069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6811","nonce":4043981069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6815","nonce":4043981069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6813","nonce":4043981069}]},"public_addr":"172.21.15.123:6809/4043981069","cluster_addr":"172.21.15.123:6811/4043981069","heartbeat_back_addr":"172.21.15.123:6815/4043981069","heartbeat_front_addr":"172.21.15.123:6813/4043981069","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.904540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.958585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.616367+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.885244+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.103127+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.167248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.884348+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.862356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.23","mappings":[{"from":3,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:22:07.632 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.631+0000 7f3c91c00640 1 -- 172.21.15.121:0/1789346931 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f3c6806c8f0 msgr2=0x7f3c6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:07.632 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.631+0000 7f3c91c00640 1 --2- 172.21.15.121:0/1789346931 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f3c6806c8f0 0x7f3c6806edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c006fd0 tx=0x7f3c7c008040 comp rx=0 tx=0).stop 2024-04-20T10:22:07.632 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.631+0000 7f3c91c00640 1 -- 172.21.15.121:0/1789346931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c8c0b3330 msgr2=0x7f3c8c0cd850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:07.632 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.631+0000 7f3c91c00640 1 --2- 172.21.15.121:0/1789346931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c8c0b3330 0x7f3c8c0cd850 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3c7400d830 tx=0x7f3c7400dcf0 comp rx=0 tx=0).stop 2024-04-20T10:22:07.632 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.631+0000 7f3c91c00640 1 -- 172.21.15.121:0/1789346931 shutdown_connections 2024-04-20T10:22:07.632 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.631+0000 7f3c91c00640 1 --2- 172.21.15.121:0/1789346931 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f3c6806c8f0 0x7f3c6806edb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:07.632 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.631+0000 7f3c91c00640 1 --2- 172.21.15.121:0/1789346931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c8c161d50 0x7f3c8c14f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:07.632 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.631+0000 7f3c91c00640 1 --2- 172.21.15.121:0/1789346931 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3c8c0b3330 0x7f3c8c0cd850 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:07.632 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.631+0000 7f3c91c00640 1 --2- 172.21.15.121:0/1789346931 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3c8c06c6d0 0x7f3c8c0ca750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:07.633 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.631+0000 7f3c91c00640 1 -- 172.21.15.121:0/1789346931 >> 172.21.15.121:0/1789346931 conn(0x7f3c8c0d4450 msgr2=0x7f3c8c0b76f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:07.633 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.631+0000 7f3c91c00640 1 -- 172.21.15.121:0/1789346931 shutdown_connections 2024-04-20T10:22:07.633 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.631+0000 7f3c91c00640 1 -- 172.21.15.121:0/1789346931 wait complete. 2024-04-20T10:22:07.651 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:20:51.141365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:21:00.436937+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:21:01.026957+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T10:21:04.862693+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T10:22:07.652 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:22:07.854 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.853+0000 7fac93000640 1 Processor -- start 2024-04-20T10:22:07.854 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.854+0000 7fac93000640 1 -- start start 2024-04-20T10:22:07.855 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.855+0000 7fac93000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fac8c156c20 0x7fac8c06b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:07.855 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.855+0000 7fac93000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac8c15ca40 0x7fac8c0ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:07.855 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.855+0000 7fac93000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac8c141c60 con 0x7fac8c0ad8e0 2024-04-20T10:22:07.856 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.855+0000 7fac93000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fac8c15cf90 con 0x7fac8c156c20 2024-04-20T10:22:07.856 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.855+0000 7fac93000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fac8c15d270 con 0x7fac8c15ca40 2024-04-20T10:22:07.856 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.855+0000 7fac91000640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fac8c0ad8e0 0x7fac8c0d0360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:34820/0 (socket says 172.21.15.121:34820) 2024-04-20T10:22:07.856 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.855+0000 7fac91000640 1 -- 172.21.15.121:0/5949459 learned_addr learned my addr 172.21.15.121:0/5949459 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:07.857 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.855+0000 7fac83400640 1 --2- 172.21.15.121:0/5949459 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac8c15ca40 0x7fac8c0ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:07.858 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.855+0000 7fac8be00640 1 --2- 172.21.15.121:0/5949459 >> v2:172.21.15.123:3300/0 conn(0x7fac8c156c20 0x7fac8c06b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:07.858 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.856+0000 7fac8b400640 1 -- 172.21.15.121:0/5949459 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2704259065 0 0) 0x7fac8c141c60 con 0x7fac8c0ad8e0 2024-04-20T10:22:07.858 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.856+0000 7fac8b400640 1 -- 172.21.15.121:0/5949459 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac70003520 con 0x7fac8c0ad8e0 2024-04-20T10:22:07.858 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.856+0000 7fac8b400640 1 -- 172.21.15.121:0/5949459 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 853836149 0 0) 0x7fac70003520 con 0x7fac8c0ad8e0 2024-04-20T10:22:07.858 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.856+0000 7fac8b400640 1 -- 172.21.15.121:0/5949459 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac8c15ca40 msgr2=0x7fac8c0ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:07.858 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.856+0000 7fac8b400640 1 --2- 172.21.15.121:0/5949459 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac8c15ca40 0x7fac8c0ad3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:07.858 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.856+0000 7fac8b400640 1 -- 172.21.15.121:0/5949459 >> v2:172.21.15.123:3300/0 conn(0x7fac8c156c20 msgr2=0x7fac8c06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:07.858 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.856+0000 7fac8b400640 1 --2- 172.21.15.121:0/5949459 >> v2:172.21.15.123:3300/0 conn(0x7fac8c156c20 0x7fac8c06b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:07.858 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.856+0000 7fac8b400640 1 -- 172.21.15.121:0/5949459 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac8c15d550 con 0x7fac8c0ad8e0 2024-04-20T10:22:07.858 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.856+0000 7fac8b400640 1 -- 172.21.15.121:0/5949459 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7fac840019f0 con 0x7fac8c0ad8e0 2024-04-20T10:22:07.858 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.856+0000 7fac8b400640 1 -- 172.21.15.121:0/5949459 >> v1:172.21.15.121:6789/0 conn(0x7fac8c0ad8e0 legacy=0x7fac8c0d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:07.858 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.856+0000 7fac8b400640 1 --2- 172.21.15.121:0/5949459 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac8c15ca40 0x7fac700039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:07.858 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.856+0000 7fac8b400640 1 --2- 172.21.15.121:0/5949459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac70003f30 0x7fac70004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:07.859 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.856+0000 7fac8b400640 1 --2- 172.21.15.121:0/5949459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac70004d40 0x7fac700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:07.859 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.856+0000 7fac8b400640 1 -- 172.21.15.121:0/5949459 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fac8c15cf90 con 0x7fac70003f30 2024-04-20T10:22:07.859 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.856+0000 7fac8b400640 1 -- 172.21.15.121:0/5949459 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fac8c15d270 con 0x7fac70004d40 2024-04-20T10:22:07.859 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.856+0000 7fac8b400640 1 -- 172.21.15.121:0/5949459 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fac7000b9c0 con 0x7fac8c15ca40 2024-04-20T10:22:07.859 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.856+0000 7fac83400640 1 --2- 172.21.15.121:0/5949459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac70003f30 0x7fac70004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:07.859 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.857+0000 7fac8be00640 1 --2- 172.21.15.121:0/5949459 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac8c15ca40 0x7fac700039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:07.859 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.857+0000 7fac83400640 1 -- 172.21.15.121:0/5949459 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac8c15ca40 msgr2=0x7fac700039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:07.859 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.857+0000 7fac83400640 1 --2- 172.21.15.121:0/5949459 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac8c15ca40 0x7fac700039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:07.859 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.857+0000 7fac83400640 1 -- 172.21.15.121:0/5949459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac70004d40 msgr2=0x7fac700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:07.859 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.857+0000 7fac83400640 1 --2- 172.21.15.121:0/5949459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac70004d40 0x7fac700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:07.859 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.857+0000 7fac83400640 1 -- 172.21.15.121:0/5949459 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac70003520 con 0x7fac70003f30 2024-04-20T10:22:07.860 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.858+0000 7fac83400640 1 --2- 172.21.15.121:0/5949459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac70003f30 0x7fac70004800 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fac74009ac0 tx=0x7fac70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbd4c2c4cb2e3bde server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:07.860 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.858+0000 7fac8b400640 1 -- 172.21.15.121:0/5949459 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac74039070 con 0x7fac70003f30 2024-04-20T10:22:07.860 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.858+0000 7fac8b400640 1 -- 172.21.15.121:0/5949459 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac7402c6b0 con 0x7fac70003f30 2024-04-20T10:22:07.861 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.858+0000 7fac8b400640 1 -- 172.21.15.121:0/5949459 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac74009690 con 0x7fac70003f30 2024-04-20T10:22:07.861 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.859+0000 7fac93000640 1 -- 172.21.15.121:0/5949459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac70003f30 msgr2=0x7fac70004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:07.861 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.859+0000 7fac93000640 1 --2- 172.21.15.121:0/5949459 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac70003f30 0x7fac70004800 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fac74009ac0 tx=0x7fac70005830 comp rx=0 tx=0).stop 2024-04-20T10:22:07.861 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.859+0000 7fac91000640 1 -- 172.21.15.121:0/5949459 reap_dead start 2024-04-20T10:22:07.861 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.859+0000 7fac93000640 1 -- 172.21.15.121:0/5949459 shutdown_connections 2024-04-20T10:22:07.861 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.859+0000 7fac93000640 1 -- 172.21.15.121:0/5949459 >> 172.21.15.121:0/5949459 conn(0x7fac8c0b3350 msgr2=0x7fac8c0b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:07.861 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.859+0000 7fac93000640 1 -- 172.21.15.121:0/5949459 shutdown_connections 2024-04-20T10:22:07.861 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.859+0000 7fac93000640 1 -- 172.21.15.121:0/5949459 wait complete. 2024-04-20T10:22:07.862 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.861+0000 7fac93000640 1 Processor -- start 2024-04-20T10:22:07.862 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.861+0000 7fac93000640 1 -- start start 2024-04-20T10:22:07.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.862+0000 7fac93000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac70003f30 0x7fac8c099760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:07.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.862+0000 7fac93000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac8c06d150 0x7fac8c15b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:07.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.862+0000 7fac93000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac8c0cc4d0 0x7fac8c15b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:07.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.862+0000 7fac93000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fac8c06bfc0 con 0x7fac70003f30 2024-04-20T10:22:07.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.862+0000 7fac93000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fac8c06be40 con 0x7fac8c06d150 2024-04-20T10:22:07.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.862+0000 7fac93000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fac8c0d0a70 con 0x7fac8c0cc4d0 2024-04-20T10:22:07.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.862+0000 7fac8be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac70003f30 0x7fac8c099760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:07.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.862+0000 7fac8be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac70003f30 0x7fac8c099760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42420/0 (socket says 172.21.15.121:42420) 2024-04-20T10:22:07.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.862+0000 7fac8be00640 1 -- 172.21.15.121:0/2901141337 learned_addr learned my addr 172.21.15.121:0/2901141337 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:07.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.862+0000 7fac8be00640 1 -- 172.21.15.121:0/2901141337 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac8c0cc4d0 msgr2=0x7fac8c15b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:07.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.862+0000 7fac8be00640 1 --2- 172.21.15.121:0/2901141337 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac8c0cc4d0 0x7fac8c15b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:07.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.862+0000 7fac8be00640 1 -- 172.21.15.121:0/2901141337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac8c06d150 msgr2=0x7fac8c15b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:07.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.862+0000 7fac8be00640 1 --2- 172.21.15.121:0/2901141337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac8c06d150 0x7fac8c15b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:07.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.862+0000 7fac8be00640 1 -- 172.21.15.121:0/2901141337 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac8c151cd0 con 0x7fac70003f30 2024-04-20T10:22:07.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.863+0000 7fac8be00640 1 --2- 172.21.15.121:0/2901141337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac70003f30 0x7fac8c099760 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fac7c0116d0 tx=0x7fac7c011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:07.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.863+0000 7fac88c00640 1 -- 172.21.15.121:0/2901141337 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c00f9c0 con 0x7fac70003f30 2024-04-20T10:22:07.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.863+0000 7fac88c00640 1 -- 172.21.15.121:0/2901141337 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac7c00fb60 con 0x7fac70003f30 2024-04-20T10:22:07.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.863+0000 7fac93000640 1 -- 172.21.15.121:0/2901141337 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac8c151fb0 con 0x7fac70003f30 2024-04-20T10:22:07.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.863+0000 7fac88c00640 1 -- 172.21.15.121:0/2901141337 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c015460 con 0x7fac70003f30 2024-04-20T10:22:07.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.863+0000 7fac93000640 1 -- 172.21.15.121:0/2901141337 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac8c14bc00 con 0x7fac70003f30 2024-04-20T10:22:07.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.864+0000 7fac93000640 1 -- 172.21.15.121:0/2901141337 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac8c068980 con 0x7fac70003f30 2024-04-20T10:22:07.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.865+0000 7fac88c00640 1 -- 172.21.15.121:0/2901141337 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fac7c015600 con 0x7fac70003f30 2024-04-20T10:22:07.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.866+0000 7fac88c00640 1 --2- 172.21.15.121:0/2901141337 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fac6806c920 0x7fac6806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:07.867 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.866+0000 7fac88c00640 1 -- 172.21.15.121:0/2901141337 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fac7c08c8c0 con 0x7fac70003f30 2024-04-20T10:22:07.867 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.866+0000 7fac83400640 1 --2- 172.21.15.121:0/2901141337 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fac6806c920 0x7fac6806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:07.867 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.867+0000 7fac83400640 1 --2- 172.21.15.121:0/2901141337 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fac6806c920 0x7fac6806ede0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fac74036000 tx=0x7fac7402e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:07.871 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:07.871+0000 7fac88c00640 1 -- 172.21.15.121:0/2901141337 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fac7c019030 con 0x7fac70003f30 2024-04-20T10:22:08.077 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.077+0000 7fac93000640 1 -- 172.21.15.121:0/2901141337 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fac8c0cfd90 con 0x7fac70003f30 2024-04-20T10:22:08.078 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.078+0000 7fac88c00640 1 -- 172.21.15.121:0/2901141337 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fac7c05a980 con 0x7fac70003f30 2024-04-20T10:22:08.079 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2024-04-20T10:22:08.081 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.081+0000 7fac93000640 1 -- 172.21.15.121:0/2901141337 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fac6806c920 msgr2=0x7fac6806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:08.081 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.081+0000 7fac93000640 1 --2- 172.21.15.121:0/2901141337 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fac6806c920 0x7fac6806ede0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fac74036000 tx=0x7fac7402e000 comp rx=0 tx=0).stop 2024-04-20T10:22:08.081 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.081+0000 7fac93000640 1 -- 172.21.15.121:0/2901141337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac70003f30 msgr2=0x7fac8c099760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:08.081 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.081+0000 7fac93000640 1 --2- 172.21.15.121:0/2901141337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac70003f30 0x7fac8c099760 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fac7c0116d0 tx=0x7fac7c011b90 comp rx=0 tx=0).stop 2024-04-20T10:22:08.081 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.081+0000 7fac93000640 1 -- 172.21.15.121:0/2901141337 shutdown_connections 2024-04-20T10:22:08.082 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.081+0000 7fac93000640 1 --2- 172.21.15.121:0/2901141337 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fac8c0cc4d0 0x7fac8c15b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:08.082 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.081+0000 7fac93000640 1 --2- 172.21.15.121:0/2901141337 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fac6806c920 0x7fac6806ede0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:08.082 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.081+0000 7fac93000640 1 --2- 172.21.15.121:0/2901141337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac8c06d150 0x7fac8c15b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:08.082 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.081+0000 7fac93000640 1 --2- 172.21.15.121:0/2901141337 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fac70003f30 0x7fac8c099760 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:08.082 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.081+0000 7fac93000640 1 -- 172.21.15.121:0/2901141337 >> 172.21.15.121:0/2901141337 conn(0x7fac8c0b3350 msgr2=0x7fac8c14b360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:08.082 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.081+0000 7fac93000640 1 -- 172.21.15.121:0/2901141337 shutdown_connections 2024-04-20T10:22:08.082 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.081+0000 7fac93000640 1 -- 172.21.15.121:0/2901141337 wait complete. 2024-04-20T10:22:08.100 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:22:08.296 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.295+0000 7f2d89200640 1 Processor -- start 2024-04-20T10:22:08.296 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.296+0000 7f2d89200640 1 -- start start 2024-04-20T10:22:08.298 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.297+0000 7f2d89200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2d840aee60 0x7f2d840af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:08.298 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.297+0000 7f2d89200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2d8406e060 0x7f2d8406e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:08.298 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.297+0000 7f2d89200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d840af780 con 0x7f2d8415d4f0 2024-04-20T10:22:08.298 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.297+0000 7f2d89200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f2d8406c510 con 0x7f2d840aee60 2024-04-20T10:22:08.298 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.297+0000 7f2d89200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f2d8406b900 con 0x7f2d8406e060 2024-04-20T10:22:08.298 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.298+0000 7f2d82a00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f2d8415d4f0 0x7f2d840ad0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:34834/0 (socket says 172.21.15.121:34834) 2024-04-20T10:22:08.298 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.298+0000 7f2d82a00640 1 -- 172.21.15.121:0/1344232723 learned_addr learned my addr 172.21.15.121:0/1344232723 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:08.300 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.298+0000 7f2d83400640 1 --2- 172.21.15.121:0/1344232723 >> v2:172.21.15.123:3300/0 conn(0x7f2d840aee60 0x7f2d840af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:08.300 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.298+0000 7f2d83e00640 1 --2- 172.21.15.121:0/1344232723 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2d8406e060 0x7f2d8406e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:08.300 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.298+0000 7f2d82000640 1 -- 172.21.15.121:0/1344232723 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 19263377 0 0) 0x7f2d840af780 con 0x7f2d8415d4f0 2024-04-20T10:22:08.300 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.298+0000 7f2d82000640 1 -- 172.21.15.121:0/1344232723 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d68003520 con 0x7f2d8415d4f0 2024-04-20T10:22:08.300 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.299+0000 7f2d82000640 1 -- 172.21.15.121:0/1344232723 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3136460904 0 0) 0x7f2d68003520 con 0x7f2d8415d4f0 2024-04-20T10:22:08.300 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.299+0000 7f2d82000640 1 -- 172.21.15.121:0/1344232723 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2d8406e060 msgr2=0x7f2d8406e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:08.300 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.299+0000 7f2d82000640 1 --2- 172.21.15.121:0/1344232723 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2d8406e060 0x7f2d8406e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:08.301 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.299+0000 7f2d82000640 1 -- 172.21.15.121:0/1344232723 >> v2:172.21.15.123:3300/0 conn(0x7f2d840aee60 msgr2=0x7f2d840af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:08.301 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.299+0000 7f2d82000640 1 --2- 172.21.15.121:0/1344232723 >> v2:172.21.15.123:3300/0 conn(0x7f2d840aee60 0x7f2d840af240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:08.301 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.299+0000 7f2d82000640 1 -- 172.21.15.121:0/1344232723 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d8406bbe0 con 0x7f2d8415d4f0 2024-04-20T10:22:08.301 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.299+0000 7f2d82000640 1 -- 172.21.15.121:0/1344232723 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f2d70001670 con 0x7f2d8415d4f0 2024-04-20T10:22:08.301 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.299+0000 7f2d82000640 1 -- 172.21.15.121:0/1344232723 >> v1:172.21.15.121:6789/0 conn(0x7f2d8415d4f0 legacy=0x7f2d840ad0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:08.301 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.299+0000 7f2d82000640 1 --2- 172.21.15.121:0/1344232723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d680039b0 0x7f2d68003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:08.301 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.299+0000 7f2d82000640 1 --2- 172.21.15.121:0/1344232723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d68004800 0x7f2d68004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:08.301 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.299+0000 7f2d82000640 1 --2- 172.21.15.121:0/1344232723 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2d8406e060 0x7f2d680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:08.301 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.299+0000 7f2d82000640 1 -- 172.21.15.121:0/1344232723 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8406c510 con 0x7f2d680039b0 2024-04-20T10:22:08.301 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.299+0000 7f2d82000640 1 -- 172.21.15.121:0/1344232723 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8406b900 con 0x7f2d68004800 2024-04-20T10:22:08.301 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.299+0000 7f2d82000640 1 -- 172.21.15.121:0/1344232723 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f2d6800ba40 con 0x7f2d8406e060 2024-04-20T10:22:08.301 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.299+0000 7f2d83400640 1 --2- 172.21.15.121:0/1344232723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d680039b0 0x7f2d68003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:08.301 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.299+0000 7f2d83400640 1 -- 172.21.15.121:0/1344232723 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2d8406e060 msgr2=0x7f2d680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:08.301 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.299+0000 7f2d83400640 1 --2- 172.21.15.121:0/1344232723 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2d8406e060 0x7f2d680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:08.302 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.299+0000 7f2d83400640 1 -- 172.21.15.121:0/1344232723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d68004800 msgr2=0x7f2d68004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:08.302 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.299+0000 7f2d83400640 1 --2- 172.21.15.121:0/1344232723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d68004800 0x7f2d68004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:08.302 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.299+0000 7f2d83400640 1 -- 172.21.15.121:0/1344232723 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d68003520 con 0x7f2d680039b0 2024-04-20T10:22:08.303 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.301+0000 7f2d83400640 1 --2- 172.21.15.121:0/1344232723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d680039b0 0x7f2d68003dd0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2d7400ba30 tx=0x7f2d68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13250b79fd95148e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:08.303 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.301+0000 7f2d82000640 1 -- 172.21.15.121:0/1344232723 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d74036020 con 0x7f2d680039b0 2024-04-20T10:22:08.303 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.301+0000 7f2d82000640 1 -- 172.21.15.121:0/1344232723 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d74032680 con 0x7f2d680039b0 2024-04-20T10:22:08.303 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.301+0000 7f2d82000640 1 -- 172.21.15.121:0/1344232723 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7400b600 con 0x7f2d680039b0 2024-04-20T10:22:08.303 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.301+0000 7f2d89200640 1 -- 172.21.15.121:0/1344232723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d680039b0 msgr2=0x7f2d68003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:08.303 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.301+0000 7f2d89200640 1 --2- 172.21.15.121:0/1344232723 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d680039b0 0x7f2d68003dd0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2d7400ba30 tx=0x7f2d68005830 comp rx=0 tx=0).stop 2024-04-20T10:22:08.303 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.301+0000 7f2d83e00640 1 -- 172.21.15.121:0/1344232723 reap_dead start 2024-04-20T10:22:08.304 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.302+0000 7f2d89200640 1 -- 172.21.15.121:0/1344232723 shutdown_connections 2024-04-20T10:22:08.304 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.302+0000 7f2d89200640 1 -- 172.21.15.121:0/1344232723 >> 172.21.15.121:0/1344232723 conn(0x7f2d840b5270 msgr2=0x7f2d840b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:08.304 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.302+0000 7f2d89200640 1 -- 172.21.15.121:0/1344232723 shutdown_connections 2024-04-20T10:22:08.304 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.302+0000 7f2d89200640 1 -- 172.21.15.121:0/1344232723 wait complete. 2024-04-20T10:22:08.304 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.303+0000 7f2d89200640 1 Processor -- start 2024-04-20T10:22:08.304 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.303+0000 7f2d89200640 1 -- start start 2024-04-20T10:22:08.305 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.304+0000 7f2d89200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d680039b0 0x7f2d841f3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:08.306 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.304+0000 7f2d89200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2d84155a10 0x7f2d8415b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:08.306 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.304+0000 7f2d89200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d8415b810 0x7f2d84141dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:08.306 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.304+0000 7f2d89200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8406c9a0 con 0x7f2d8415b810 2024-04-20T10:22:08.306 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.304+0000 7f2d89200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2d8406c1a0 con 0x7f2d680039b0 2024-04-20T10:22:08.306 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.304+0000 7f2d89200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f2d8406e4f0 con 0x7f2d84155a10 2024-04-20T10:22:08.306 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.305+0000 7f2d83e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d8415b810 0x7f2d84141dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:08.306 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.305+0000 7f2d83e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d8415b810 0x7f2d84141dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42434/0 (socket says 172.21.15.121:42434) 2024-04-20T10:22:08.306 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.305+0000 7f2d83e00640 1 -- 172.21.15.121:0/1027108359 learned_addr learned my addr 172.21.15.121:0/1027108359 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:08.306 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.305+0000 7f2d82a00640 1 --2- 172.21.15.121:0/1027108359 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2d84155a10 0x7f2d8415b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:08.306 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.305+0000 7f2d83400640 1 --2- 172.21.15.121:0/1027108359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d680039b0 0x7f2d841f3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:08.306 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.305+0000 7f2d83e00640 1 -- 172.21.15.121:0/1027108359 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2d84155a10 msgr2=0x7f2d8415b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:08.306 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.305+0000 7f2d83e00640 1 --2- 172.21.15.121:0/1027108359 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2d84155a10 0x7f2d8415b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:08.306 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.305+0000 7f2d83e00640 1 -- 172.21.15.121:0/1027108359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d680039b0 msgr2=0x7f2d841f3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:08.307 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.305+0000 7f2d83e00640 1 --2- 172.21.15.121:0/1027108359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d680039b0 0x7f2d841f3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:08.307 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.305+0000 7f2d83e00640 1 -- 172.21.15.121:0/1027108359 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d84154560 con 0x7f2d8415b810 2024-04-20T10:22:08.307 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.305+0000 7f2d83e00640 1 --2- 172.21.15.121:0/1027108359 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d8415b810 0x7f2d84141dc0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2d840b31e0 tx=0x7f2d7c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:08.307 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.306+0000 7f2d7b400640 1 -- 172.21.15.121:0/1027108359 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c00ba80 con 0x7f2d8415b810 2024-04-20T10:22:08.307 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.306+0000 7f2d7b400640 1 -- 172.21.15.121:0/1027108359 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d7c01a070 con 0x7f2d8415b810 2024-04-20T10:22:08.308 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.306+0000 7f2d7b400640 1 -- 172.21.15.121:0/1027108359 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c00be40 con 0x7f2d8415b810 2024-04-20T10:22:08.308 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.306+0000 7f2d89200640 1 -- 172.21.15.121:0/1027108359 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d841534c0 con 0x7f2d8415b810 2024-04-20T10:22:08.308 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.306+0000 7f2d89200640 1 -- 172.21.15.121:0/1027108359 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d84153a80 con 0x7f2d8415b810 2024-04-20T10:22:08.308 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.307+0000 7f2d89200640 1 -- 172.21.15.121:0/1027108359 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d84075b30 con 0x7f2d8415b810 2024-04-20T10:22:08.308 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.308+0000 7f2d7b400640 1 -- 172.21.15.121:0/1027108359 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d7c005d10 con 0x7f2d8415b810 2024-04-20T10:22:08.309 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.309+0000 7f2d7b400640 1 --2- 172.21.15.121:0/1027108359 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f2d6406c920 0x7f2d6406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:08.309 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.309+0000 7f2d7b400640 1 -- 172.21.15.121:0/1027108359 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f2d7c08bf30 con 0x7f2d8415b810 2024-04-20T10:22:08.309 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.309+0000 7f2d83400640 1 --2- 172.21.15.121:0/1027108359 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f2d6406c920 0x7f2d6406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:08.310 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.310+0000 7f2d83400640 1 --2- 172.21.15.121:0/1027108359 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f2d6406c920 0x7f2d6406ede0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2d7400ba00 tx=0x7f2d74030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:08.313 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.313+0000 7f2d7b400640 1 -- 172.21.15.121:0/1027108359 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2d7c059f60 con 0x7f2d8415b810 2024-04-20T10:22:08.514 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.514+0000 7f2d89200640 1 -- 172.21.15.121:0/1027108359 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2d8406de40 con 0x7f2d8415b810 2024-04-20T10:22:08.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.515+0000 7f2d7b400640 1 -- 172.21.15.121:0/1027108359 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f2d7c059860 con 0x7f2d8415b810 2024-04-20T10:22:08.515 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 32 2024-04-20T10:22:08.518 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.517+0000 7f2d89200640 1 -- 172.21.15.121:0/1027108359 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f2d6406c920 msgr2=0x7f2d6406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:08.518 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.517+0000 7f2d89200640 1 --2- 172.21.15.121:0/1027108359 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f2d6406c920 0x7f2d6406ede0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2d7400ba00 tx=0x7f2d74030000 comp rx=0 tx=0).stop 2024-04-20T10:22:08.518 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.517+0000 7f2d89200640 1 -- 172.21.15.121:0/1027108359 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d8415b810 msgr2=0x7f2d84141dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:08.518 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.517+0000 7f2d89200640 1 --2- 172.21.15.121:0/1027108359 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d8415b810 0x7f2d84141dc0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2d840b31e0 tx=0x7f2d7c00b480 comp rx=0 tx=0).stop 2024-04-20T10:22:08.518 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.517+0000 7f2d89200640 1 -- 172.21.15.121:0/1027108359 shutdown_connections 2024-04-20T10:22:08.518 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.517+0000 7f2d89200640 1 --2- 172.21.15.121:0/1027108359 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2d8415b810 0x7f2d84141dc0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:08.518 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.517+0000 7f2d89200640 1 --2- 172.21.15.121:0/1027108359 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2d84155a10 0x7f2d8415b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:08.518 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.517+0000 7f2d89200640 1 --2- 172.21.15.121:0/1027108359 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f2d6406c920 0x7f2d6406ede0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:08.518 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.517+0000 7f2d89200640 1 --2- 172.21.15.121:0/1027108359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d680039b0 0x7f2d841f3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:08.519 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.517+0000 7f2d89200640 1 -- 172.21.15.121:0/1027108359 >> 172.21.15.121:0/1027108359 conn(0x7f2d840b5270 msgr2=0x7f2d840cf330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:08.519 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.517+0000 7f2d89200640 1 -- 172.21.15.121:0/1027108359 shutdown_connections 2024-04-20T10:22:08.519 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.517+0000 7f2d89200640 1 -- 172.21.15.121:0/1027108359 wait complete. 2024-04-20T10:22:08.538 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:22:08.744 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.744+0000 7f45e0200640 1 Processor -- start 2024-04-20T10:22:08.744 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.744+0000 7f45e0200640 1 -- start start 2024-04-20T10:22:08.747 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.745+0000 7f45e0200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f45d80d1a00 0x7f45d80b1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:08.747 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.745+0000 7f45e0200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45d80b17d0 0x7f45d80d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:08.747 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.745+0000 7f45e0200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45d80d1e90 con 0x7f45d80d4270 2024-04-20T10:22:08.747 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.745+0000 7f45e0200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f45d80d40c0 con 0x7f45d80d1a00 2024-04-20T10:22:08.747 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.745+0000 7f45e0200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f45d80d3820 con 0x7f45d80b17d0 2024-04-20T10:22:08.747 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.746+0000 7f45dd800640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f45d80d4270 0x7f45d80d4650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:34842/0 (socket says 172.21.15.121:34842) 2024-04-20T10:22:08.747 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.746+0000 7f45dd800640 1 -- 172.21.15.121:0/3350437567 learned_addr learned my addr 172.21.15.121:0/3350437567 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:08.747 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.746+0000 7f45dce00640 1 --2- 172.21.15.121:0/3350437567 >> v2:172.21.15.123:3300/0 conn(0x7f45d80d1a00 0x7f45d80b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:08.747 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.746+0000 7f45de200640 1 --2- 172.21.15.121:0/3350437567 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45d80b17d0 0x7f45d80d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:08.747 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.746+0000 7f45d7e00640 1 -- 172.21.15.121:0/3350437567 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3393647568 0 0) 0x7f45cc001110 con 0x7f45d80d4270 2024-04-20T10:22:08.747 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.746+0000 7f45d7e00640 1 -- 172.21.15.121:0/3350437567 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45c0003520 con 0x7f45d80d4270 2024-04-20T10:22:08.749 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.747+0000 7f45d7e00640 1 -- 172.21.15.121:0/3350437567 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3575309044 0 0) 0x7f45c0003520 con 0x7f45d80d4270 2024-04-20T10:22:08.749 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.747+0000 7f45d7e00640 1 -- 172.21.15.121:0/3350437567 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45d80b17d0 msgr2=0x7f45d80d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:08.749 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.747+0000 7f45d7e00640 1 --2- 172.21.15.121:0/3350437567 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45d80b17d0 0x7f45d80d0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:08.749 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.747+0000 7f45d7e00640 1 -- 172.21.15.121:0/3350437567 >> v2:172.21.15.123:3300/0 conn(0x7f45d80d1a00 msgr2=0x7f45d80b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:08.749 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.747+0000 7f45d7e00640 1 --2- 172.21.15.121:0/3350437567 >> v2:172.21.15.123:3300/0 conn(0x7f45d80d1a00 0x7f45d80b1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:08.749 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.747+0000 7f45d7e00640 1 -- 172.21.15.121:0/3350437567 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d80d3b00 con 0x7f45d80d4270 2024-04-20T10:22:08.749 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.747+0000 7f45d7e00640 1 -- 172.21.15.121:0/3350437567 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f45cc001840 con 0x7f45d80d4270 2024-04-20T10:22:08.749 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.747+0000 7f45d7e00640 1 -- 172.21.15.121:0/3350437567 >> v1:172.21.15.121:6789/0 conn(0x7f45d80d4270 legacy=0x7f45d80d4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:08.749 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.747+0000 7f45d7e00640 1 --2- 172.21.15.121:0/3350437567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45c00039b0 0x7f45c0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:08.749 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.747+0000 7f45d7e00640 1 --2- 172.21.15.121:0/3350437567 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45c0004800 0x7f45c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:08.750 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.747+0000 7f45d7e00640 1 --2- 172.21.15.121:0/3350437567 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45d80b17d0 0x7f45c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:08.750 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.747+0000 7f45d7e00640 1 -- 172.21.15.121:0/3350437567 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f45d80d40c0 con 0x7f45c0004800 2024-04-20T10:22:08.750 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.747+0000 7f45d7e00640 1 -- 172.21.15.121:0/3350437567 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f45d80d3820 con 0x7f45c00039b0 2024-04-20T10:22:08.750 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.747+0000 7f45d7e00640 1 -- 172.21.15.121:0/3350437567 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f45c000b9c0 con 0x7f45d80b17d0 2024-04-20T10:22:08.750 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.747+0000 7f45dce00640 1 --2- 172.21.15.121:0/3350437567 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45c0004800 0x7f45c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:08.750 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.747+0000 7f45dce00640 1 -- 172.21.15.121:0/3350437567 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45d80b17d0 msgr2=0x7f45c00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:08.750 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.747+0000 7f45dce00640 1 --2- 172.21.15.121:0/3350437567 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45d80b17d0 0x7f45c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:08.750 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.747+0000 7f45dce00640 1 -- 172.21.15.121:0/3350437567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45c00039b0 msgr2=0x7f45c0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:08.750 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.747+0000 7f45dce00640 1 --2- 172.21.15.121:0/3350437567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45c00039b0 0x7f45c0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:08.750 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.747+0000 7f45dce00640 1 -- 172.21.15.121:0/3350437567 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c0003520 con 0x7f45c0004800 2024-04-20T10:22:08.750 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.749+0000 7f45dce00640 1 --2- 172.21.15.121:0/3350437567 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45c0004800 0x7f45c0004c80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f45c8009ac0 tx=0x7f45c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de4060392ff9289 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:08.751 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.749+0000 7f45d7e00640 1 -- 172.21.15.121:0/3350437567 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c8039070 con 0x7f45c0004800 2024-04-20T10:22:08.752 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.749+0000 7f45d7e00640 1 -- 172.21.15.121:0/3350437567 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45c802c710 con 0x7f45c0004800 2024-04-20T10:22:08.752 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.749+0000 7f45d7e00640 1 -- 172.21.15.121:0/3350437567 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c8009690 con 0x7f45c0004800 2024-04-20T10:22:08.752 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.749+0000 7f45e0200640 1 -- 172.21.15.121:0/3350437567 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45c0004800 msgr2=0x7f45c0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:08.752 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.750+0000 7f45e0200640 1 --2- 172.21.15.121:0/3350437567 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45c0004800 0x7f45c0004c80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f45c8009ac0 tx=0x7f45c0005830 comp rx=0 tx=0).stop 2024-04-20T10:22:08.752 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.750+0000 7f45de200640 1 -- 172.21.15.121:0/3350437567 reap_dead start 2024-04-20T10:22:08.752 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.750+0000 7f45e0200640 1 -- 172.21.15.121:0/3350437567 shutdown_connections 2024-04-20T10:22:08.752 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.750+0000 7f45e0200640 1 -- 172.21.15.121:0/3350437567 >> 172.21.15.121:0/3350437567 conn(0x7f45d80b99c0 msgr2=0x7f45d80b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:08.752 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.750+0000 7f45e0200640 1 -- 172.21.15.121:0/3350437567 shutdown_connections 2024-04-20T10:22:08.752 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.750+0000 7f45e0200640 1 -- 172.21.15.121:0/3350437567 wait complete. 2024-04-20T10:22:08.752 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.751+0000 7f45e0200640 1 Processor -- start 2024-04-20T10:22:08.754 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.752+0000 7f45e0200640 1 -- start start 2024-04-20T10:22:08.754 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.752+0000 7f45e0200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45c0004800 0x7f45d8159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:08.754 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.752+0000 7f45e0200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45d80b3250 0x7f45d80b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:08.754 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.752+0000 7f45e0200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45d815c1c0 0x7f45d815c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:08.754 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.753+0000 7f45e0200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f45d80d53b0 con 0x7f45c0004800 2024-04-20T10:22:08.754 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.753+0000 7f45e0200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f45d80cfe60 con 0x7f45d80b3250 2024-04-20T10:22:08.754 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.753+0000 7f45e0200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f45d8161f70 con 0x7f45d815c1c0 2024-04-20T10:22:08.754 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.753+0000 7f45dd800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45c0004800 0x7f45d8159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:08.754 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.753+0000 7f45dd800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45c0004800 0x7f45d8159b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42442/0 (socket says 172.21.15.121:42442) 2024-04-20T10:22:08.754 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.753+0000 7f45dd800640 1 -- 172.21.15.121:0/2711274664 learned_addr learned my addr 172.21.15.121:0/2711274664 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:08.754 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.753+0000 7f45de200640 1 --2- 172.21.15.121:0/2711274664 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45d815c1c0 0x7f45d815c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:08.755 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.753+0000 7f45dd800640 1 -- 172.21.15.121:0/2711274664 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45d815c1c0 msgr2=0x7f45d815c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:08.755 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.753+0000 7f45dd800640 1 --2- 172.21.15.121:0/2711274664 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45d815c1c0 0x7f45d815c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:08.755 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.753+0000 7f45dce00640 1 --2- 172.21.15.121:0/2711274664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45d80b3250 0x7f45d80b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:08.755 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.753+0000 7f45dd800640 1 -- 172.21.15.121:0/2711274664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45d80b3250 msgr2=0x7f45d80b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:08.755 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.753+0000 7f45dd800640 1 --2- 172.21.15.121:0/2711274664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45d80b3250 0x7f45d80b36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:08.755 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.753+0000 7f45dd800640 1 -- 172.21.15.121:0/2711274664 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d815cde0 con 0x7f45c0004800 2024-04-20T10:22:08.755 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.754+0000 7f45dd800640 1 --2- 172.21.15.121:0/2711274664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45c0004800 0x7f45d8159b20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0012f0 tx=0x7f45cc00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:08.756 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.754+0000 7f45d5600640 1 -- 172.21.15.121:0/2711274664 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc018070 con 0x7f45c0004800 2024-04-20T10:22:08.756 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.754+0000 7f45d5600640 1 -- 172.21.15.121:0/2711274664 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45cc0074e0 con 0x7f45c0004800 2024-04-20T10:22:08.756 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.754+0000 7f45d5600640 1 -- 172.21.15.121:0/2711274664 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45cc004a00 con 0x7f45c0004800 2024-04-20T10:22:08.756 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.754+0000 7f45e0200640 1 -- 172.21.15.121:0/2711274664 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45d815d0c0 con 0x7f45c0004800 2024-04-20T10:22:08.756 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.754+0000 7f45e0200640 1 -- 172.21.15.121:0/2711274664 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45d80c81a0 con 0x7f45c0004800 2024-04-20T10:22:08.756 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.755+0000 7f45e0200640 1 -- 172.21.15.121:0/2711274664 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45d8068980 con 0x7f45c0004800 2024-04-20T10:22:08.756 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.756+0000 7f45d5600640 1 -- 172.21.15.121:0/2711274664 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f45cc004ba0 con 0x7f45c0004800 2024-04-20T10:22:08.758 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.757+0000 7f45d5600640 1 --2- 172.21.15.121:0/2711274664 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f45bc06c8f0 0x7f45bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:08.758 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.758+0000 7f45d5600640 1 -- 172.21.15.121:0/2711274664 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f45cc08b440 con 0x7f45c0004800 2024-04-20T10:22:08.758 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.758+0000 7f45dce00640 1 --2- 172.21.15.121:0/2711274664 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f45bc06c8f0 0x7f45bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:08.759 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.758+0000 7f45dce00640 1 --2- 172.21.15.121:0/2711274664 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f45bc06c8f0 0x7f45bc06edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f45c8036000 tx=0x7f45c8030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:08.762 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.762+0000 7f45d5600640 1 -- 172.21.15.121:0/2711274664 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f45d80d1e90 con 0x7f45c0004800 2024-04-20T10:22:08.959 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.958+0000 7f45e0200640 1 -- 172.21.15.121:0/2711274664 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f45d80b7940 con 0x7f45c0004800 2024-04-20T10:22:08.959 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.959+0000 7f45d5600640 1 -- 172.21.15.121:0/2711274664 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f45cc059500 con 0x7f45c0004800 2024-04-20T10:22:08.960 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2024-04-20T10:22:08.962 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.962+0000 7f45e0200640 1 -- 172.21.15.121:0/2711274664 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f45bc06c8f0 msgr2=0x7f45bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:08.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.962+0000 7f45e0200640 1 --2- 172.21.15.121:0/2711274664 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f45bc06c8f0 0x7f45bc06edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f45c8036000 tx=0x7f45c8030000 comp rx=0 tx=0).stop 2024-04-20T10:22:08.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.962+0000 7f45e0200640 1 -- 172.21.15.121:0/2711274664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45c0004800 msgr2=0x7f45d8159b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:08.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.962+0000 7f45e0200640 1 --2- 172.21.15.121:0/2711274664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45c0004800 0x7f45d8159b20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0012f0 tx=0x7f45cc00ccd0 comp rx=0 tx=0).stop 2024-04-20T10:22:08.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.962+0000 7f45e0200640 1 -- 172.21.15.121:0/2711274664 shutdown_connections 2024-04-20T10:22:08.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.962+0000 7f45e0200640 1 --2- 172.21.15.121:0/2711274664 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45d815c1c0 0x7f45d815c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:08.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.962+0000 7f45e0200640 1 --2- 172.21.15.121:0/2711274664 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f45bc06c8f0 0x7f45bc06edb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:08.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.962+0000 7f45e0200640 1 --2- 172.21.15.121:0/2711274664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45d80b3250 0x7f45d80b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:08.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.962+0000 7f45e0200640 1 --2- 172.21.15.121:0/2711274664 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45c0004800 0x7f45d8159b20 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:08.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.962+0000 7f45e0200640 1 -- 172.21.15.121:0/2711274664 >> 172.21.15.121:0/2711274664 conn(0x7f45d80b99c0 msgr2=0x7f45d8160ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:08.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.962+0000 7f45e0200640 1 -- 172.21.15.121:0/2711274664 shutdown_connections 2024-04-20T10:22:08.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:08.962+0000 7f45e0200640 1 -- 172.21.15.121:0/2711274664 wait complete. 2024-04-20T10:22:08.981 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T10:22:09.184 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.183+0000 7f7e01c00640 1 Processor -- start 2024-04-20T10:22:09.184 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.183+0000 7f7e01c00640 1 -- start start 2024-04-20T10:22:09.185 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.185+0000 7f7e01c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7dfc15e8a0 0x7f7dfc06e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:09.186 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.185+0000 7f7e01c00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dfc06f060 con 0x7f7dfc06dbb0 2024-04-20T10:22:09.186 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.185+0000 7f7e01c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dfc06e7c0 con 0x7f7dfc0d0a70 2024-04-20T10:22:09.186 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.185+0000 7f7e01c00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f7dfc06eaa0 con 0x7f7dfc15e8a0 2024-04-20T10:22:09.186 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.185+0000 7f7dfaa00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7dfc15e8a0 0x7f7dfc06e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:09.186 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.185+0000 7f7dfaa00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f7dfc15e8a0 0x7f7dfc06e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42446/0 (socket says 172.21.15.121:42446) 2024-04-20T10:22:09.186 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.185+0000 7f7dfaa00640 1 -- 172.21.15.121:0/3633335593 learned_addr learned my addr 172.21.15.121:0/3633335593 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:09.187 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.186+0000 7f7dfaa00640 1 -- 172.21.15.121:0/3633335593 >> v1:172.21.15.123:6789/0 conn(0x7f7dfc0d0a70 legacy=0x7f7dfc0ad680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:09.187 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.186+0000 7f7dfaa00640 1 -- 172.21.15.121:0/3633335593 >> v1:172.21.15.121:6789/0 conn(0x7f7dfc06dbb0 legacy=0x7f7dfc0d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:09.187 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.186+0000 7f7dfaa00640 1 -- 172.21.15.121:0/3633335593 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dfc06ed80 con 0x7f7dfc15e8a0 2024-04-20T10:22:09.187 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.186+0000 7f7dfa000640 1 -- 172.21.15.121:0/3633335593 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2777590223 0 0) 0x7f7dfc06f060 con 0x7f7dfc06dbb0 2024-04-20T10:22:09.188 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.187+0000 7f7dfaa00640 1 --2- 172.21.15.121:0/3633335593 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7dfc15e8a0 0x7f7dfc06e1e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7dec009e50 tx=0x7f7dec02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3871255e36fc224 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:09.188 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.187+0000 7f7dfa000640 1 -- 172.21.15.121:0/3633335593 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec02cb20 con 0x7f7dfc15e8a0 2024-04-20T10:22:09.188 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.187+0000 7f7dfa000640 1 -- 172.21.15.121:0/3633335593 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dec02ccc0 con 0x7f7dfc15e8a0 2024-04-20T10:22:09.188 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.187+0000 7f7dfa000640 1 -- 172.21.15.121:0/3633335593 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec034610 con 0x7f7dfc15e8a0 2024-04-20T10:22:09.188 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.188+0000 7f7e01c00640 1 -- 172.21.15.121:0/3633335593 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7dfc15e8a0 msgr2=0x7f7dfc06e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:09.188 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.188+0000 7f7e01c00640 1 --2- 172.21.15.121:0/3633335593 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7dfc15e8a0 0x7f7dfc06e1e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7dec009e50 tx=0x7f7dec02bf80 comp rx=0 tx=0).stop 2024-04-20T10:22:09.189 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.188+0000 7f7e01c00640 1 -- 172.21.15.121:0/3633335593 shutdown_connections 2024-04-20T10:22:09.189 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.188+0000 7f7e01c00640 1 --2- 172.21.15.121:0/3633335593 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7dfc15e8a0 0x7f7dfc06e1e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:09.189 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.188+0000 7f7e01c00640 1 -- 172.21.15.121:0/3633335593 >> 172.21.15.121:0/3633335593 conn(0x7f7dfc0b3350 msgr2=0x7f7dfc0b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:09.189 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.188+0000 7f7e01c00640 1 -- 172.21.15.121:0/3633335593 shutdown_connections 2024-04-20T10:22:09.189 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.188+0000 7f7e01c00640 1 -- 172.21.15.121:0/3633335593 wait complete. 2024-04-20T10:22:09.190 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.190+0000 7f7e01c00640 1 Processor -- start 2024-04-20T10:22:09.190 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.190+0000 7f7e01c00640 1 -- start start 2024-04-20T10:22:09.193 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.191+0000 7f7e01c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dfc06dbb0 0x7f7dfc153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:09.193 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.191+0000 7f7e01c00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7dfc0d0a70 0x7f7dfc0b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:09.193 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.191+0000 7f7e01c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7dfc15e8a0 0x7f7dfc0ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:09.193 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.191+0000 7f7e01c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc06fef0 con 0x7f7dfc15e8a0 2024-04-20T10:22:09.193 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.191+0000 7f7e01c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc06fd70 con 0x7f7dfc06dbb0 2024-04-20T10:22:09.193 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.191+0000 7f7e01c00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f7dfc0add90 con 0x7f7dfc0d0a70 2024-04-20T10:22:09.193 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.191+0000 7f7dfb400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7dfc15e8a0 0x7f7dfc0ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:09.193 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.191+0000 7f7dfb400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7dfc15e8a0 0x7f7dfc0ae3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42454/0 (socket says 172.21.15.121:42454) 2024-04-20T10:22:09.193 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.191+0000 7f7dfb400640 1 -- 172.21.15.121:0/1890293435 learned_addr learned my addr 172.21.15.121:0/1890293435 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:09.193 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.191+0000 7f7dfaa00640 1 --2- 172.21.15.121:0/1890293435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dfc06dbb0 0x7f7dfc153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:09.193 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.192+0000 7f7df2000640 1 --2- 172.21.15.121:0/1890293435 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7dfc0d0a70 0x7f7dfc0b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:09.194 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.192+0000 7f7dfb400640 1 -- 172.21.15.121:0/1890293435 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7dfc0d0a70 msgr2=0x7f7dfc0b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:09.194 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.192+0000 7f7dfb400640 1 --2- 172.21.15.121:0/1890293435 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7dfc0d0a70 0x7f7dfc0b12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:09.194 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.192+0000 7f7dfb400640 1 -- 172.21.15.121:0/1890293435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dfc06dbb0 msgr2=0x7f7dfc153d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:09.194 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.192+0000 7f7dfb400640 1 --2- 172.21.15.121:0/1890293435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dfc06dbb0 0x7f7dfc153d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:09.194 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.192+0000 7f7dfb400640 1 -- 172.21.15.121:0/1890293435 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dfc152cb0 con 0x7f7dfc15e8a0 2024-04-20T10:22:09.194 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.192+0000 7f7dfb400640 1 --2- 172.21.15.121:0/1890293435 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7dfc15e8a0 0x7f7dfc0ae3e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7df400c410 tx=0x7f7df400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:09.194 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.192+0000 7f7df3400640 1 -- 172.21.15.121:0/1890293435 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df4017070 con 0x7f7dfc15e8a0 2024-04-20T10:22:09.194 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.192+0000 7f7df3400640 1 -- 172.21.15.121:0/1890293435 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7df4007230 con 0x7f7dfc15e8a0 2024-04-20T10:22:09.194 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.192+0000 7f7df3400640 1 -- 172.21.15.121:0/1890293435 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7df4007620 con 0x7f7dfc15e8a0 2024-04-20T10:22:09.194 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.192+0000 7f7e01c00640 1 -- 172.21.15.121:0/1890293435 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dfc152f90 con 0x7f7dfc15e8a0 2024-04-20T10:22:09.194 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.192+0000 7f7e01c00640 1 -- 172.21.15.121:0/1890293435 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dfc153550 con 0x7f7dfc15e8a0 2024-04-20T10:22:09.195 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.194+0000 7f7e01c00640 1 -- 172.21.15.121:0/1890293435 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dfc068980 con 0x7f7dfc15e8a0 2024-04-20T10:22:09.195 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.195+0000 7f7df3400640 1 -- 172.21.15.121:0/1890293435 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7df401a070 con 0x7f7dfc15e8a0 2024-04-20T10:22:09.197 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.196+0000 7f7df3400640 1 --2- 172.21.15.121:0/1890293435 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7dd806c8f0 0x7f7dd806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:09.197 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.196+0000 7f7df3400640 1 -- 172.21.15.121:0/1890293435 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f7df408b1a0 con 0x7f7dfc15e8a0 2024-04-20T10:22:09.197 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.196+0000 7f7dfaa00640 1 --2- 172.21.15.121:0/1890293435 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7dd806c8f0 0x7f7dd806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:09.197 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.197+0000 7f7dfaa00640 1 --2- 172.21.15.121:0/1890293435 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7dd806c8f0 0x7f7dd806edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7dec035000 tx=0x7f7dec030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:09.200 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.200+0000 7f7df3400640 1 -- 172.21.15.121:0/1890293435 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7df40591c0 con 0x7f7dfc15e8a0 2024-04-20T10:22:09.402 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.401+0000 7f7e01c00640 1 -- 172.21.15.121:0/1890293435 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f7dfc06d160 con 0x7f7dfc15e8a0 2024-04-20T10:22:09.403 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.402+0000 7f7df3400640 1 -- 172.21.15.121:0/1890293435 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f7df4058ac0 con 0x7f7dfc15e8a0 2024-04-20T10:22:09.403 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 32 2024-04-20T10:22:09.405 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.405+0000 7f7e01c00640 1 -- 172.21.15.121:0/1890293435 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7dd806c8f0 msgr2=0x7f7dd806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:09.406 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.405+0000 7f7e01c00640 1 --2- 172.21.15.121:0/1890293435 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7dd806c8f0 0x7f7dd806edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7dec035000 tx=0x7f7dec030000 comp rx=0 tx=0).stop 2024-04-20T10:22:09.406 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.405+0000 7f7e01c00640 1 -- 172.21.15.121:0/1890293435 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7dfc15e8a0 msgr2=0x7f7dfc0ae3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:09.406 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.405+0000 7f7e01c00640 1 --2- 172.21.15.121:0/1890293435 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7dfc15e8a0 0x7f7dfc0ae3e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7df400c410 tx=0x7f7df400c8d0 comp rx=0 tx=0).stop 2024-04-20T10:22:09.406 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.405+0000 7f7e01c00640 1 -- 172.21.15.121:0/1890293435 shutdown_connections 2024-04-20T10:22:09.406 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.405+0000 7f7e01c00640 1 --2- 172.21.15.121:0/1890293435 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7dfc15e8a0 0x7f7dfc0ae3e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:09.406 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.405+0000 7f7e01c00640 1 --2- 172.21.15.121:0/1890293435 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7dfc0d0a70 0x7f7dfc0b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:09.406 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.405+0000 7f7e01c00640 1 --2- 172.21.15.121:0/1890293435 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7dd806c8f0 0x7f7dd806edb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:09.406 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.405+0000 7f7e01c00640 1 --2- 172.21.15.121:0/1890293435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dfc06dbb0 0x7f7dfc153d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:09.406 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.405+0000 7f7e01c00640 1 -- 172.21.15.121:0/1890293435 >> 172.21.15.121:0/1890293435 conn(0x7f7dfc0b3350 msgr2=0x7f7dfc15d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:09.407 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.405+0000 7f7e01c00640 1 -- 172.21.15.121:0/1890293435 shutdown_connections 2024-04-20T10:22:09.407 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.405+0000 7f7e01c00640 1 -- 172.21.15.121:0/1890293435 wait complete. 2024-04-20T10:22:09.426 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic' 2024-04-20T10:22:09.629 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.628+0000 7f91aba00640 1 Processor -- start 2024-04-20T10:22:09.629 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.629+0000 7f91aba00640 1 -- start start 2024-04-20T10:22:09.630 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.630+0000 7f91aba00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f91a4156c20 0x7f91a406b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:09.631 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.630+0000 7f91aba00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91a4141c60 con 0x7f91a40d0360 2024-04-20T10:22:09.631 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.630+0000 7f91aba00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91a40adab0 con 0x7f91a415ca40 2024-04-20T10:22:09.631 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.630+0000 7f91aba00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f91a415cf90 con 0x7f91a4156c20 2024-04-20T10:22:09.633 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.630+0000 7f91a9a00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f91a40d0360 0x7f91a4142140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:34864/0 (socket says 172.21.15.121:34864) 2024-04-20T10:22:09.633 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.631+0000 7f91a9a00640 1 -- 172.21.15.121:0/3505260508 learned_addr learned my addr 172.21.15.121:0/3505260508 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:09.633 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.631+0000 7f91a9000640 1 --2- 172.21.15.121:0/3505260508 >> v2:172.21.15.123:3300/0 conn(0x7f91a4156c20 0x7f91a406b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:09.633 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.631+0000 7f91a3e00640 1 -- 172.21.15.121:0/3505260508 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1920020164 0 0) 0x7f91a4141c60 con 0x7f91a40d0360 2024-04-20T10:22:09.633 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.631+0000 7f91a3e00640 1 -- 172.21.15.121:0/3505260508 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9188003520 con 0x7f91a40d0360 2024-04-20T10:22:09.633 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.631+0000 7f91a3e00640 1 -- 172.21.15.121:0/3505260508 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3513223872 0 0) 0x7f9188003520 con 0x7f91a40d0360 2024-04-20T10:22:09.633 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.631+0000 7f91a3e00640 1 -- 172.21.15.121:0/3505260508 >> v2:172.21.15.123:3300/0 conn(0x7f91a4156c20 msgr2=0x7f91a406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:09.633 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.631+0000 7f91a3e00640 1 --2- 172.21.15.121:0/3505260508 >> v2:172.21.15.123:3300/0 conn(0x7f91a4156c20 0x7f91a406b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:09.634 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.631+0000 7f91a3e00640 1 -- 172.21.15.121:0/3505260508 >> v1:172.21.15.123:6789/0 conn(0x7f91a415ca40 legacy=0x7f91a40ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:09.634 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.631+0000 7f91a3e00640 1 -- 172.21.15.121:0/3505260508 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a415d270 con 0x7f91a40d0360 2024-04-20T10:22:09.634 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.631+0000 7f91a3e00640 1 -- 172.21.15.121:0/3505260508 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f919c0019f0 con 0x7f91a40d0360 2024-04-20T10:22:09.634 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.631+0000 7f91a3e00640 1 -- 172.21.15.121:0/3505260508 >> v1:172.21.15.121:6789/0 conn(0x7f91a40d0360 legacy=0x7f91a4142140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:09.634 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.632+0000 7f91a3e00640 1 --2- 172.21.15.121:0/3505260508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91880039b0 0x7f9188003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:09.634 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.632+0000 7f91a3e00640 1 --2- 172.21.15.121:0/3505260508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9188004800 0x7f9188004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:09.634 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.632+0000 7f91a3e00640 1 --2- 172.21.15.121:0/3505260508 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f91880051c0 0x7f918800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:09.634 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.632+0000 7f91a3e00640 1 -- 172.21.15.121:0/3505260508 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f91a415cf90 con 0x7f91880039b0 2024-04-20T10:22:09.634 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.632+0000 7f91a3e00640 1 -- 172.21.15.121:0/3505260508 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f918800c060 con 0x7f9188004800 2024-04-20T10:22:09.634 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.632+0000 7f91a3e00640 1 -- 172.21.15.121:0/3505260508 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f918800c260 con 0x7f91880051c0 2024-04-20T10:22:09.634 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.632+0000 7f91a9000640 1 --2- 172.21.15.121:0/3505260508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91880039b0 0x7f9188003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:09.634 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.632+0000 7f919be00640 1 --2- 172.21.15.121:0/3505260508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9188004800 0x7f9188004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:09.634 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.632+0000 7f91a9000640 1 -- 172.21.15.121:0/3505260508 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f91880051c0 msgr2=0x7f918800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:09.635 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.632+0000 7f91a9a00640 1 --2- 172.21.15.121:0/3505260508 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f91880051c0 0x7f918800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:09.635 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.632+0000 7f91a9000640 1 --2- 172.21.15.121:0/3505260508 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f91880051c0 0x7f918800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:09.635 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.632+0000 7f91a9000640 1 -- 172.21.15.121:0/3505260508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9188004800 msgr2=0x7f9188004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:09.635 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.632+0000 7f91a9000640 1 --2- 172.21.15.121:0/3505260508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9188004800 0x7f9188004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:09.635 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.632+0000 7f91a9000640 1 -- 172.21.15.121:0/3505260508 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9188003520 con 0x7f91880039b0 2024-04-20T10:22:09.635 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.632+0000 7f919be00640 1 --2- 172.21.15.121:0/3505260508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9188004800 0x7f9188004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:09.635 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.632+0000 7f91a9a00640 1 -- 172.21.15.121:0/3505260508 reap_dead start 2024-04-20T10:22:09.636 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.633+0000 7f91a9000640 1 --2- 172.21.15.121:0/3505260508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91880039b0 0x7f9188003dd0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f9194009ac0 tx=0x7f9188005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f682d706362305 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:09.636 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.634+0000 7f91a3e00640 1 -- 172.21.15.121:0/3505260508 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9194039070 con 0x7f91880039b0 2024-04-20T10:22:09.637 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.634+0000 7f91a3e00640 1 -- 172.21.15.121:0/3505260508 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f919402c6b0 con 0x7f91880039b0 2024-04-20T10:22:09.637 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.634+0000 7f91a3e00640 1 -- 172.21.15.121:0/3505260508 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9194009690 con 0x7f91880039b0 2024-04-20T10:22:09.637 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.634+0000 7f91aba00640 1 -- 172.21.15.121:0/3505260508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91880039b0 msgr2=0x7f9188003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:09.637 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.634+0000 7f91aba00640 1 --2- 172.21.15.121:0/3505260508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91880039b0 0x7f9188003dd0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f9194009ac0 tx=0x7f9188005830 comp rx=0 tx=0).stop 2024-04-20T10:22:09.637 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.634+0000 7f91aba00640 1 -- 172.21.15.121:0/3505260508 shutdown_connections 2024-04-20T10:22:09.637 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.634+0000 7f91aba00640 1 --2- 172.21.15.121:0/3505260508 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91880039b0 0x7f9188003dd0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:09.637 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.634+0000 7f91aba00640 1 -- 172.21.15.121:0/3505260508 >> 172.21.15.121:0/3505260508 conn(0x7f91a40b3350 msgr2=0x7f91a40b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:09.637 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.635+0000 7f91aba00640 1 -- 172.21.15.121:0/3505260508 shutdown_connections 2024-04-20T10:22:09.637 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.635+0000 7f91aba00640 1 -- 172.21.15.121:0/3505260508 wait complete. 2024-04-20T10:22:09.637 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.636+0000 7f91aba00640 1 Processor -- start 2024-04-20T10:22:09.639 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.636+0000 7f91aba00640 1 -- start start 2024-04-20T10:22:09.639 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.637+0000 7f91aba00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91880039b0 0x7f91a41e6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:09.639 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.637+0000 7f91aba00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91a4160050 0x7f91a40ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:09.639 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.637+0000 7f91aba00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f91a40cd070 0x7f91a40cd520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:09.639 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.637+0000 7f91aba00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f91a40d07d0 con 0x7f91880039b0 2024-04-20T10:22:09.639 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.637+0000 7f91aba00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f91a415d830 con 0x7f91a4160050 2024-04-20T10:22:09.640 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.637+0000 7f91aba00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f91a40d0950 con 0x7f91a40cd070 2024-04-20T10:22:09.640 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.638+0000 7f91a9000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91880039b0 0x7f91a41e6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:09.640 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.638+0000 7f91a9000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91880039b0 0x7f91a41e6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42458/0 (socket says 172.21.15.121:42458) 2024-04-20T10:22:09.640 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.638+0000 7f91a9000640 1 -- 172.21.15.121:0/457859205 learned_addr learned my addr 172.21.15.121:0/457859205 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:09.640 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.638+0000 7f919be00640 1 --2- 172.21.15.121:0/457859205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91a4160050 0x7f91a40ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:09.640 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.638+0000 7f91a9000640 1 -- 172.21.15.121:0/457859205 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f91a40cd070 msgr2=0x7f91a40cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:09.640 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.638+0000 7f91a9000640 1 --2- 172.21.15.121:0/457859205 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f91a40cd070 0x7f91a40cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:09.640 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.638+0000 7f91a9000640 1 -- 172.21.15.121:0/457859205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91a4160050 msgr2=0x7f91a40ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:09.640 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.638+0000 7f91a9000640 1 --2- 172.21.15.121:0/457859205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91a4160050 0x7f91a40ccb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:09.640 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.638+0000 7f91a9000640 1 -- 172.21.15.121:0/457859205 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a40ce780 con 0x7f91880039b0 2024-04-20T10:22:09.640 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.638+0000 7f91a9000640 1 --2- 172.21.15.121:0/457859205 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91880039b0 0x7f91a41e6d70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9194000c00 tx=0x7f9194004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:09.640 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.639+0000 7f91a1600640 1 -- 172.21.15.121:0/457859205 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9194042020 con 0x7f91880039b0 2024-04-20T10:22:09.641 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.639+0000 7f91a1600640 1 -- 172.21.15.121:0/457859205 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91940044c0 con 0x7f91880039b0 2024-04-20T10:22:09.641 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.639+0000 7f91a1600640 1 -- 172.21.15.121:0/457859205 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9194039040 con 0x7f91880039b0 2024-04-20T10:22:09.641 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.639+0000 7f91aba00640 1 -- 172.21.15.121:0/457859205 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91a40cea60 con 0x7f91880039b0 2024-04-20T10:22:09.641 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.639+0000 7f91aba00640 1 -- 172.21.15.121:0/457859205 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91a40cf020 con 0x7f91880039b0 2024-04-20T10:22:09.641 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.640+0000 7f91aba00640 1 -- 172.21.15.121:0/457859205 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91a4068980 con 0x7f91880039b0 2024-04-20T10:22:09.641 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.641+0000 7f91a1600640 1 -- 172.21.15.121:0/457859205 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9194045020 con 0x7f91880039b0 2024-04-20T10:22:09.643 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.642+0000 7f91a1600640 1 --2- 172.21.15.121:0/457859205 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f918006c8f0 0x7f918006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:09.643 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.642+0000 7f91a1600640 1 -- 172.21.15.121:0/457859205 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f91940ada70 con 0x7f91880039b0 2024-04-20T10:22:09.643 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.642+0000 7f919be00640 1 --2- 172.21.15.121:0/457859205 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f918006c8f0 0x7f918006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:09.643 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.643+0000 7f919be00640 1 --2- 172.21.15.121:0/457859205 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f918006c8f0 0x7f918006edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f91a4074020 tx=0x7f918c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:09.647 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.647+0000 7f91a1600640 1 -- 172.21.15.121:0/457859205 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f919404c020 con 0x7f91880039b0 2024-04-20T10:22:09.920 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:09.920+0000 7f91aba00640 1 -- 172.21.15.121:0/457859205 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]} v 0) -- 0x7f91a40cfd90 con 0x7f91880039b0 2024-04-20T10:22:10.027 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.026+0000 7f91a1600640 1 -- 172.21.15.121:0/457859205 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.tests_from_xfstests_dev.TestXFSTestsDev.test_generic"]}]=0 v85) ==== 147+0+0 (secure 0 0 0) 0x7f919407bbb0 con 0x7f91880039b0 2024-04-20T10:22:10.031 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.030+0000 7f91aba00640 1 -- 172.21.15.121:0/457859205 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f918006c8f0 msgr2=0x7f918006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:10.031 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.030+0000 7f91aba00640 1 --2- 172.21.15.121:0/457859205 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f918006c8f0 0x7f918006edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f91a4074020 tx=0x7f918c009040 comp rx=0 tx=0).stop 2024-04-20T10:22:10.031 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.030+0000 7f91aba00640 1 -- 172.21.15.121:0/457859205 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91880039b0 msgr2=0x7f91a41e6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:10.031 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.030+0000 7f91aba00640 1 --2- 172.21.15.121:0/457859205 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91880039b0 0x7f91a41e6d70 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9194000c00 tx=0x7f9194004730 comp rx=0 tx=0).stop 2024-04-20T10:22:10.031 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.030+0000 7f91aba00640 1 -- 172.21.15.121:0/457859205 shutdown_connections 2024-04-20T10:22:10.031 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.030+0000 7f91aba00640 1 --2- 172.21.15.121:0/457859205 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f91a40cd070 0x7f91a40cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:10.031 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.030+0000 7f91aba00640 1 --2- 172.21.15.121:0/457859205 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f918006c8f0 0x7f918006edb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:10.031 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.030+0000 7f91aba00640 1 --2- 172.21.15.121:0/457859205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91a4160050 0x7f91a40ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:10.031 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.030+0000 7f91aba00640 1 --2- 172.21.15.121:0/457859205 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f91880039b0 0x7f91a41e6d70 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:10.031 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.030+0000 7f91aba00640 1 -- 172.21.15.121:0/457859205 >> 172.21.15.121:0/457859205 conn(0x7f91a40b3350 msgr2=0x7f91a415db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:10.031 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.030+0000 7f91aba00640 1 -- 172.21.15.121:0/457859205 shutdown_connections 2024-04-20T10:22:10.031 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.030+0000 7f91aba00640 1 -- 172.21.15.121:0/457859205 wait complete. 2024-04-20T10:22:10.053 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-20T10:22:10.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.260+0000 7f96e9000640 1 Processor -- start 2024-04-20T10:22:10.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.261+0000 7f96e9000640 1 -- start start 2024-04-20T10:22:10.263 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.262+0000 7f96e9000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f96e4156c20 0x7f96e406b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:10.263 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.262+0000 7f96e9000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96e415ca40 0x7f96e40ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:10.263 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.262+0000 7f96e9000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96e4141c60 con 0x7f96e40ad8e0 2024-04-20T10:22:10.264 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.262+0000 7f96e9000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f96e415cf90 con 0x7f96e4156c20 2024-04-20T10:22:10.264 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.262+0000 7f96e9000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f96e415d270 con 0x7f96e415ca40 2024-04-20T10:22:10.264 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.262+0000 7f96e1e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f96e4156c20 0x7f96e406b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:10.264 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.263+0000 7f96e2800640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f96e40ad8e0 0x7f96e40d0360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:34870/0 (socket says 172.21.15.121:34870) 2024-04-20T10:22:10.264 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.263+0000 7f96e2800640 1 -- 172.21.15.121:0/1156844238 learned_addr learned my addr 172.21.15.121:0/1156844238 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:10.264 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.263+0000 7f96d9400640 1 --2- 172.21.15.121:0/1156844238 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96e415ca40 0x7f96e40ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:10.264 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.263+0000 7f96e1e00640 1 -- 172.21.15.121:0/1156844238 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96e415ca40 msgr2=0x7f96e40ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:10.264 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.263+0000 7f96e1e00640 1 --2- 172.21.15.121:0/1156844238 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96e415ca40 0x7f96e40ad3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:10.264 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.263+0000 7f96e1e00640 1 -- 172.21.15.121:0/1156844238 >> v1:172.21.15.121:6789/0 conn(0x7f96e40ad8e0 legacy=0x7f96e40d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:10.264 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.263+0000 7f96e1e00640 1 -- 172.21.15.121:0/1156844238 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96e415d550 con 0x7f96e4156c20 2024-04-20T10:22:10.266 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.264+0000 7f96e1e00640 1 --2- 172.21.15.121:0/1156844238 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96e4156c20 0x7f96e406b900 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f96d40097a0 tx=0x7f96d402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b0a0863f35fdd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:10.266 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.264+0000 7f96e1400640 1 -- 172.21.15.121:0/1156844238 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d4035020 con 0x7f96e4156c20 2024-04-20T10:22:10.266 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.264+0000 7f96e1400640 1 -- 172.21.15.121:0/1156844238 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96d4031810 con 0x7f96e4156c20 2024-04-20T10:22:10.266 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.264+0000 7f96e1400640 1 -- 172.21.15.121:0/1156844238 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d4031be0 con 0x7f96e4156c20 2024-04-20T10:22:10.266 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.265+0000 7f96e9000640 1 -- 172.21.15.121:0/1156844238 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96e4156c20 msgr2=0x7f96e406b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:10.266 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.265+0000 7f96e9000640 1 --2- 172.21.15.121:0/1156844238 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96e4156c20 0x7f96e406b900 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f96d40097a0 tx=0x7f96d402b8b0 comp rx=0 tx=0).stop 2024-04-20T10:22:10.266 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.265+0000 7f96e9000640 1 -- 172.21.15.121:0/1156844238 shutdown_connections 2024-04-20T10:22:10.266 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.265+0000 7f96e9000640 1 --2- 172.21.15.121:0/1156844238 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96e415ca40 0x7f96e40ad3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:10.267 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.265+0000 7f96e9000640 1 --2- 172.21.15.121:0/1156844238 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96e4156c20 0x7f96e406b900 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:10.267 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.265+0000 7f96e9000640 1 -- 172.21.15.121:0/1156844238 >> 172.21.15.121:0/1156844238 conn(0x7f96e40b3350 msgr2=0x7f96e40b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:10.267 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.265+0000 7f96e9000640 1 -- 172.21.15.121:0/1156844238 shutdown_connections 2024-04-20T10:22:10.267 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.265+0000 7f96e9000640 1 -- 172.21.15.121:0/1156844238 wait complete. 2024-04-20T10:22:10.267 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.267+0000 7f96e9000640 1 Processor -- start 2024-04-20T10:22:10.267 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.267+0000 7f96e9000640 1 -- start start 2024-04-20T10:22:10.269 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.268+0000 7f96e9000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96e40ad8e0 0x7f96e41f59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:10.270 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.268+0000 7f96e9000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96e4156c20 0x7f96e40cea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:10.270 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.268+0000 7f96e9000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96e415ca40 0x7f96e414fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:10.270 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.268+0000 7f96e9000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f96e406bfc0 con 0x7f96e40ad8e0 2024-04-20T10:22:10.270 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.268+0000 7f96e9000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f96e406be40 con 0x7f96e415ca40 2024-04-20T10:22:10.270 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.268+0000 7f96e9000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f96e40d0a70 con 0x7f96e4156c20 2024-04-20T10:22:10.270 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.268+0000 7f96e1e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96e40ad8e0 0x7f96e41f59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:10.270 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.269+0000 7f96e1e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96e40ad8e0 0x7f96e41f59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42470/0 (socket says 172.21.15.121:42470) 2024-04-20T10:22:10.270 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.269+0000 7f96e1e00640 1 -- 172.21.15.121:0/1395490220 learned_addr learned my addr 172.21.15.121:0/1395490220 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:10.270 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.269+0000 7f96d9400640 1 --2- 172.21.15.121:0/1395490220 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96e4156c20 0x7f96e40cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:10.270 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.269+0000 7f96e2800640 1 --2- 172.21.15.121:0/1395490220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96e415ca40 0x7f96e414fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:10.270 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.269+0000 7f96e1e00640 1 -- 172.21.15.121:0/1395490220 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96e4156c20 msgr2=0x7f96e40cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:10.271 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.269+0000 7f96e1e00640 1 --2- 172.21.15.121:0/1395490220 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96e4156c20 0x7f96e40cea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:10.271 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.269+0000 7f96e1e00640 1 -- 172.21.15.121:0/1395490220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96e415ca40 msgr2=0x7f96e414fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:10.271 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.269+0000 7f96e1e00640 1 --2- 172.21.15.121:0/1395490220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96e415ca40 0x7f96e414fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:10.271 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.269+0000 7f96e1e00640 1 -- 172.21.15.121:0/1395490220 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96e40cdc70 con 0x7f96e40ad8e0 2024-04-20T10:22:10.271 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.269+0000 7f96e2800640 1 --2- 172.21.15.121:0/1395490220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96e415ca40 0x7f96e414fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:10.271 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.269+0000 7f96e1e00640 1 --2- 172.21.15.121:0/1395490220 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96e40ad8e0 0x7f96e41f59a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f96d4009890 tx=0x7f96d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:10.272 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.270+0000 7f96daa00640 1 -- 172.21.15.121:0/1395490220 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d4041070 con 0x7f96e40ad8e0 2024-04-20T10:22:10.272 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.270+0000 7f96daa00640 1 -- 172.21.15.121:0/1395490220 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96d403f070 con 0x7f96e40ad8e0 2024-04-20T10:22:10.272 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.270+0000 7f96e9000640 1 -- 172.21.15.121:0/1395490220 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96e40cdf50 con 0x7f96e40ad8e0 2024-04-20T10:22:10.272 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.270+0000 7f96daa00640 1 -- 172.21.15.121:0/1395490220 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d4035040 con 0x7f96e40ad8e0 2024-04-20T10:22:10.272 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.270+0000 7f96e9000640 1 -- 172.21.15.121:0/1395490220 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96e41f5ee0 con 0x7f96e40ad8e0 2024-04-20T10:22:10.272 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.271+0000 7f96e9000640 1 -- 172.21.15.121:0/1395490220 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96e4068980 con 0x7f96e40ad8e0 2024-04-20T10:22:10.273 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.272+0000 7f96daa00640 1 -- 172.21.15.121:0/1395490220 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f96d4038020 con 0x7f96e40ad8e0 2024-04-20T10:22:10.274 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.273+0000 7f96daa00640 1 --2- 172.21.15.121:0/1395490220 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f96c006c8f0 0x7f96c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:10.274 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.274+0000 7f96daa00640 1 -- 172.21.15.121:0/1395490220 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f96d40add10 con 0x7f96e40ad8e0 2024-04-20T10:22:10.274 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.274+0000 7f96d9400640 1 --2- 172.21.15.121:0/1395490220 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f96c006c8f0 0x7f96c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:10.275 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.274+0000 7f96d9400640 1 --2- 172.21.15.121:0/1395490220 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f96c006c8f0 0x7f96c006edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f96cc006fd0 tx=0x7f96cc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:10.279 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.278+0000 7f96daa00640 1 -- 172.21.15.121:0/1395490220 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f96d407bd40 con 0x7f96e40ad8e0 2024-04-20T10:22:10.481 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.480+0000 7f96e9000640 1 -- 172.21.15.121:0/1395490220 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f96e40cfd90 con 0x7f96e40ad8e0 2024-04-20T10:22:10.488 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.488+0000 7f96daa00640 1 -- 172.21.15.121:0/1395490220 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f96d407b640 con 0x7f96e40ad8e0 2024-04-20T10:22:10.491 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.491+0000 7f96e9000640 1 -- 172.21.15.121:0/1395490220 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f96c006c8f0 msgr2=0x7f96c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:10.492 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.491+0000 7f96e9000640 1 --2- 172.21.15.121:0/1395490220 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f96c006c8f0 0x7f96c006edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f96cc006fd0 tx=0x7f96cc008040 comp rx=0 tx=0).stop 2024-04-20T10:22:10.492 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.491+0000 7f96e9000640 1 -- 172.21.15.121:0/1395490220 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96e40ad8e0 msgr2=0x7f96e41f59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:10.492 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.491+0000 7f96e9000640 1 --2- 172.21.15.121:0/1395490220 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96e40ad8e0 0x7f96e41f59a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f96d4009890 tx=0x7f96d40046c0 comp rx=0 tx=0).stop 2024-04-20T10:22:10.492 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.491+0000 7f96e9000640 1 -- 172.21.15.121:0/1395490220 shutdown_connections 2024-04-20T10:22:10.492 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.491+0000 7f96e9000640 1 --2- 172.21.15.121:0/1395490220 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f96c006c8f0 0x7f96c006edb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:10.492 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.491+0000 7f96e9000640 1 --2- 172.21.15.121:0/1395490220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96e415ca40 0x7f96e414fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:10.492 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.491+0000 7f96e9000640 1 --2- 172.21.15.121:0/1395490220 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f96e4156c20 0x7f96e40cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:10.492 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.491+0000 7f96e9000640 1 --2- 172.21.15.121:0/1395490220 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f96e40ad8e0 0x7f96e41f59a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:10.492 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.491+0000 7f96e9000640 1 -- 172.21.15.121:0/1395490220 >> 172.21.15.121:0/1395490220 conn(0x7f96e40b3350 msgr2=0x7f96e4099830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:10.492 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.491+0000 7f96e9000640 1 -- 172.21.15.121:0/1395490220 shutdown_connections 2024-04-20T10:22:10.492 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:10.491+0000 7f96e9000640 1 -- 172.21.15.121:0/1395490220 wait complete. 2024-04-20T10:22:10.512 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T10:22:10.512 DEBUG:teuthology.orchestra.run.smithi121:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T10:22:10.617 DEBUG:teuthology.orchestra.run.smithi123:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T10:22:10.732 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T10:22:10.732 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T10:22:10.732 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:22:10.790 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T10:22:10.790 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:22:10.846 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:22:11.037 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.037+0000 7fed7ce00640 1 Processor -- start 2024-04-20T10:22:11.037 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.037+0000 7fed7ce00640 1 -- start start 2024-04-20T10:22:11.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.038+0000 7fed7ce00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fed78071df0 0x7fed780d15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:11.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.038+0000 7fed7ce00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed780d1b10 0x7fed780b1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:11.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.038+0000 7fed7ce00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed780cff70 con 0x7fed78158d70 2024-04-20T10:22:11.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.038+0000 7fed7ce00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fed78159150 con 0x7fed78071df0 2024-04-20T10:22:11.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.038+0000 7fed7ce00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fed7815f940 con 0x7fed780d1b10 2024-04-20T10:22:11.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.039+0000 7fed75e00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fed78158d70 0x7fed780716e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:34878/0 (socket says 172.21.15.121:34878) 2024-04-20T10:22:11.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.039+0000 7fed75e00640 1 -- 172.21.15.121:0/200290287 learned_addr learned my addr 172.21.15.121:0/200290287 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:11.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.039+0000 7fed75400640 1 --2- 172.21.15.121:0/200290287 >> v2:172.21.15.121:3300/0 conn(0x7fed78071df0 0x7fed780d15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:11.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.039+0000 7fed76800640 1 --2- 172.21.15.121:0/200290287 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed780d1b10 0x7fed780b1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:11.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.039+0000 7fed75400640 1 -- 172.21.15.121:0/200290287 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed780d1b10 msgr2=0x7fed780b1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:11.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.039+0000 7fed75400640 1 --2- 172.21.15.121:0/200290287 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed780d1b10 0x7fed780b1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:11.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.039+0000 7fed74a00640 1 -- 172.21.15.121:0/200290287 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 695367644 0 0) 0x7fed780cff70 con 0x7fed78158d70 2024-04-20T10:22:11.041 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.039+0000 7fed75400640 1 -- 172.21.15.121:0/200290287 >> v1:172.21.15.121:6789/0 conn(0x7fed78158d70 legacy=0x7fed780716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:11.041 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.039+0000 7fed75400640 1 -- 172.21.15.121:0/200290287 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed7815f010 con 0x7fed78071df0 2024-04-20T10:22:11.041 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.039+0000 7fed76800640 1 --2- 172.21.15.121:0/200290287 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed780d1b10 0x7fed780b1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:11.042 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.040+0000 7fed75400640 1 --2- 172.21.15.121:0/200290287 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed78071df0 0x7fed780d15d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fed60009d90 tx=0x7fed6002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b33bf2337c47226d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:11.042 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.040+0000 7fed74a00640 1 -- 172.21.15.121:0/200290287 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed60039070 con 0x7fed78071df0 2024-04-20T10:22:11.042 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.041+0000 7fed74a00640 1 -- 172.21.15.121:0/200290287 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed6002cbc0 con 0x7fed78071df0 2024-04-20T10:22:11.042 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.041+0000 7fed74a00640 1 -- 172.21.15.121:0/200290287 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed60034440 con 0x7fed78071df0 2024-04-20T10:22:11.042 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.041+0000 7fed7ce00640 1 -- 172.21.15.121:0/200290287 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed78071df0 msgr2=0x7fed780d15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:11.042 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.041+0000 7fed7ce00640 1 --2- 172.21.15.121:0/200290287 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed78071df0 0x7fed780d15d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fed60009d90 tx=0x7fed6002be10 comp rx=0 tx=0).stop 2024-04-20T10:22:11.043 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.041+0000 7fed7ce00640 1 -- 172.21.15.121:0/200290287 shutdown_connections 2024-04-20T10:22:11.043 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.041+0000 7fed7ce00640 1 --2- 172.21.15.121:0/200290287 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed780d1b10 0x7fed780b1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:11.043 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.041+0000 7fed7ce00640 1 --2- 172.21.15.121:0/200290287 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed78071df0 0x7fed780d15d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:11.043 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.041+0000 7fed7ce00640 1 -- 172.21.15.121:0/200290287 >> 172.21.15.121:0/200290287 conn(0x7fed780d23a0 msgr2=0x7fed780d27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:11.043 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.041+0000 7fed7ce00640 1 -- 172.21.15.121:0/200290287 shutdown_connections 2024-04-20T10:22:11.043 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.041+0000 7fed7ce00640 1 -- 172.21.15.121:0/200290287 wait complete. 2024-04-20T10:22:11.043 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.043+0000 7fed7ce00640 1 Processor -- start 2024-04-20T10:22:11.044 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.043+0000 7fed7ce00640 1 -- start start 2024-04-20T10:22:11.046 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.044+0000 7fed7ce00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed78071df0 0x7fed78146ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:11.046 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.044+0000 7fed7ce00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed780d1b10 0x7fed78151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:11.046 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.044+0000 7fed7ce00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed78158d70 0x7fed780b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:11.046 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.044+0000 7fed7ce00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fed7815ec90 con 0x7fed78158d70 2024-04-20T10:22:11.046 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.044+0000 7fed7ce00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fed7815f5d0 con 0x7fed780d1b10 2024-04-20T10:22:11.046 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.044+0000 7fed7ce00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fed780d1f60 con 0x7fed78071df0 2024-04-20T10:22:11.046 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.044+0000 7fed76800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed78158d70 0x7fed780b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:11.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.045+0000 7fed76800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed78158d70 0x7fed780b6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42488/0 (socket says 172.21.15.121:42488) 2024-04-20T10:22:11.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.045+0000 7fed76800640 1 -- 172.21.15.121:0/660434429 learned_addr learned my addr 172.21.15.121:0/660434429 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:11.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.045+0000 7fed75e00640 1 --2- 172.21.15.121:0/660434429 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed78071df0 0x7fed78146ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:11.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.045+0000 7fed75400640 1 --2- 172.21.15.121:0/660434429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed780d1b10 0x7fed78151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:11.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.045+0000 7fed76800640 1 -- 172.21.15.121:0/660434429 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed78071df0 msgr2=0x7fed78146ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:11.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.045+0000 7fed76800640 1 --2- 172.21.15.121:0/660434429 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed78071df0 0x7fed78146ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:11.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.045+0000 7fed76800640 1 -- 172.21.15.121:0/660434429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed780d1b10 msgr2=0x7fed78151860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:11.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.045+0000 7fed76800640 1 --2- 172.21.15.121:0/660434429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed780d1b10 0x7fed78151860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:11.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.045+0000 7fed76800640 1 -- 172.21.15.121:0/660434429 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed780b7580 con 0x7fed78158d70 2024-04-20T10:22:11.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.045+0000 7fed75e00640 1 --2- 172.21.15.121:0/660434429 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed78071df0 0x7fed78146ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:11.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.045+0000 7fed76800640 1 --2- 172.21.15.121:0/660434429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed78158d70 0x7fed780b6b70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fed7000c7f0 tx=0x7fed7000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:11.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.045+0000 7fed6e000640 1 -- 172.21.15.121:0/660434429 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed70015070 con 0x7fed78158d70 2024-04-20T10:22:11.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.045+0000 7fed6e000640 1 -- 172.21.15.121:0/660434429 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed70007820 con 0x7fed78158d70 2024-04-20T10:22:11.048 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.045+0000 7fed6e000640 1 -- 172.21.15.121:0/660434429 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed70007c50 con 0x7fed78158d70 2024-04-20T10:22:11.048 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.046+0000 7fed7ce00640 1 -- 172.21.15.121:0/660434429 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed78150d80 con 0x7fed78158d70 2024-04-20T10:22:11.048 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.046+0000 7fed7ce00640 1 -- 172.21.15.121:0/660434429 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed78151340 con 0x7fed78158d70 2024-04-20T10:22:11.048 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.047+0000 7fed7ce00640 1 -- 172.21.15.121:0/660434429 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed78068980 con 0x7fed78158d70 2024-04-20T10:22:11.048 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.048+0000 7fed6e000640 1 -- 172.21.15.121:0/660434429 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fed70018070 con 0x7fed78158d70 2024-04-20T10:22:11.049 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.048+0000 7fed6e000640 1 --2- 172.21.15.121:0/660434429 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fed5406c8f0 0x7fed5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:11.049 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.049+0000 7fed6e000640 1 -- 172.21.15.121:0/660434429 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fed7008c0d0 con 0x7fed78158d70 2024-04-20T10:22:11.049 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.049+0000 7fed75e00640 1 --2- 172.21.15.121:0/660434429 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fed5406c8f0 0x7fed5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:11.050 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.049+0000 7fed75e00640 1 --2- 172.21.15.121:0/660434429 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fed5406c8f0 0x7fed5406edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fed68000f60 tx=0x7fed68009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:11.053 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.053+0000 7fed6e000640 1 -- 172.21.15.121:0/660434429 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fed7005a0f0 con 0x7fed78158d70 2024-04-20T10:22:11.256 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.256+0000 7fed7ce00640 1 -- 172.21.15.121:0/660434429 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fed78155580 con 0x7fed78158d70 2024-04-20T10:22:11.258 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.258+0000 7fed6e000640 1 -- 172.21.15.121:0/660434429 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18940 (secure 0 0 0) 0x7fed700599f0 con 0x7fed78158d70 2024-04-20T10:22:11.259 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:22:11.259 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":27,"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","created":"2024-04-20T10:20:38.076076+0000","modified":"2024-04-20T10:21:44.715898+0000","last_up_change":"2024-04-20T10:20:50.162700+0000","last_in_change":"2024-04-20T10:20:41.510382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:20:51.141365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:21:00.436937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:21:01.026957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:21:04.862693+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"629a7b8c-a40a-4a2a-a3cc-d9f357774310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6809","nonce":1044588835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6811","nonce":1044588835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6815","nonce":1044588835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6813","nonce":1044588835}]},"public_addr":"172.21.15.121:6809/1044588835","cluster_addr":"172.21.15.121:6811/1044588835","heartbeat_back_addr":"172.21.15.121:6815/1044588835","heartbeat_front_addr":"172.21.15.121:6813/1044588835","state":["exists","up"]},{"osd":1,"uuid":"f2570399-aa50-4ff3-bf5d-3d09207290db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6801","nonce":3505199624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6803","nonce":3505199624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6807","nonce":3505199624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6805","nonce":3505199624}]},"public_addr":"172.21.15.121:6801/3505199624","cluster_addr":"172.21.15.121:6803/3505199624","heartbeat_back_addr":"172.21.15.121:6807/3505199624","heartbeat_front_addr":"172.21.15.121:6805/3505199624","state":["exists","up"]},{"osd":2,"uuid":"50893807-dc30-49f4-b41f-739366626658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6817","nonce":3656724207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6819","nonce":3656724207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6827","nonce":3656724207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6822","nonce":3656724207}]},"public_addr":"172.21.15.121:6817/3656724207","cluster_addr":"172.21.15.121:6819/3656724207","heartbeat_back_addr":"172.21.15.121:6827/3656724207","heartbeat_front_addr":"172.21.15.121:6822/3656724207","state":["exists","up"]},{"osd":3,"uuid":"dadb01bc-5491-4ffc-8824-d48a057240b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6823","nonce":2232342832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6826","nonce":2232342832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6831","nonce":2232342832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6829","nonce":2232342832}]},"public_addr":"172.21.15.121:6823/2232342832","cluster_addr":"172.21.15.121:6826/2232342832","heartbeat_back_addr":"172.21.15.121:6831/2232342832","heartbeat_front_addr":"172.21.15.121:6829/2232342832","state":["exists","up"]},{"osd":4,"uuid":"b3f3b4a1-b375-4526-b69e-344ab24c2da1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6801","nonce":99353111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6803","nonce":99353111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6807","nonce":99353111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6805","nonce":99353111}]},"public_addr":"172.21.15.123:6801/99353111","cluster_addr":"172.21.15.123:6803/99353111","heartbeat_back_addr":"172.21.15.123:6807/99353111","heartbeat_front_addr":"172.21.15.123:6805/99353111","state":["exists","up"]},{"osd":5,"uuid":"2ee24a08-1624-4961-9642-d7b510282484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6818","nonce":2504374688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6822","nonce":2504374688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6830","nonce":2504374688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6826","nonce":2504374688}]},"public_addr":"172.21.15.123:6818/2504374688","cluster_addr":"172.21.15.123:6822/2504374688","heartbeat_back_addr":"172.21.15.123:6830/2504374688","heartbeat_front_addr":"172.21.15.123:6826/2504374688","state":["exists","up"]},{"osd":6,"uuid":"186118cc-6128-49de-acca-55260945c1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6819","nonce":4139299599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6823","nonce":4139299599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6829","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6831","nonce":4139299599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6827","nonce":4139299599}]},"public_addr":"172.21.15.123:6819/4139299599","cluster_addr":"172.21.15.123:6823/4139299599","heartbeat_back_addr":"172.21.15.123:6831/4139299599","heartbeat_front_addr":"172.21.15.123:6827/4139299599","state":["exists","up"]},{"osd":7,"uuid":"a00aac6e-4bd9-415a-a93d-ae85210865ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6809","nonce":4043981069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6811","nonce":4043981069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6815","nonce":4043981069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6813","nonce":4043981069}]},"public_addr":"172.21.15.123:6809/4043981069","cluster_addr":"172.21.15.123:6811/4043981069","heartbeat_back_addr":"172.21.15.123:6815/4043981069","heartbeat_front_addr":"172.21.15.123:6813/4043981069","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.904540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.958585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.616367+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.885244+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.103127+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.167248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.884348+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.862356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.23","mappings":[{"from":3,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:22:11.262 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.261+0000 7fed7ce00640 1 -- 172.21.15.121:0/660434429 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fed5406c8f0 msgr2=0x7fed5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:11.262 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.261+0000 7fed7ce00640 1 --2- 172.21.15.121:0/660434429 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fed5406c8f0 0x7fed5406edb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fed68000f60 tx=0x7fed68009040 comp rx=0 tx=0).stop 2024-04-20T10:22:11.262 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.261+0000 7fed7ce00640 1 -- 172.21.15.121:0/660434429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed78158d70 msgr2=0x7fed780b6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:11.262 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.261+0000 7fed7ce00640 1 --2- 172.21.15.121:0/660434429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed78158d70 0x7fed780b6b70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fed7000c7f0 tx=0x7fed7000ccb0 comp rx=0 tx=0).stop 2024-04-20T10:22:11.262 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.261+0000 7fed7ce00640 1 -- 172.21.15.121:0/660434429 shutdown_connections 2024-04-20T10:22:11.262 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.261+0000 7fed7ce00640 1 --2- 172.21.15.121:0/660434429 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fed78158d70 0x7fed780b6b70 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:11.262 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.261+0000 7fed7ce00640 1 --2- 172.21.15.121:0/660434429 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fed5406c8f0 0x7fed5406edb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:11.262 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.261+0000 7fed7ce00640 1 --2- 172.21.15.121:0/660434429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed780d1b10 0x7fed78151860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:11.262 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.261+0000 7fed7ce00640 1 --2- 172.21.15.121:0/660434429 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fed78071df0 0x7fed78146ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:11.263 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.261+0000 7fed7ce00640 1 -- 172.21.15.121:0/660434429 >> 172.21.15.121:0/660434429 conn(0x7fed780d23a0 msgr2=0x7fed780716e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:11.263 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.261+0000 7fed7ce00640 1 -- 172.21.15.121:0/660434429 shutdown_connections 2024-04-20T10:22:11.263 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.261+0000 7fed7ce00640 1 -- 172.21.15.121:0/660434429 wait complete. 2024-04-20T10:22:11.281 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:20:51.141365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:21:00.436937+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:21:01.026957+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T10:21:04.862693+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T10:22:11.282 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:22:11.484 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.484+0000 7f83aac00640 1 Processor -- start 2024-04-20T10:22:11.485 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.484+0000 7f83aac00640 1 -- start start 2024-04-20T10:22:11.487 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.485+0000 7f83aac00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f83a415e8a0 0x7f83a406e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:11.487 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.485+0000 7f83aac00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f83a40d0a70 0x7f83a40ad680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:11.487 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.485+0000 7f83aac00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83a406f060 con 0x7f83a406dbb0 2024-04-20T10:22:11.487 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.485+0000 7f83aac00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f83a406e7c0 con 0x7f83a415e8a0 2024-04-20T10:22:11.487 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.485+0000 7f83aac00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f83a406eaa0 con 0x7f83a40d0a70 2024-04-20T10:22:11.487 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.486+0000 7f83a3e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f83a415e8a0 0x7f83a406e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:11.487 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.486+0000 7f83a3e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f83a415e8a0 0x7f83a406e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42496/0 (socket says 172.21.15.121:42496) 2024-04-20T10:22:11.487 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.486+0000 7f83a3e00640 1 -- 172.21.15.121:0/2450156517 learned_addr learned my addr 172.21.15.121:0/2450156517 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:11.487 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.486+0000 7f83a8c00640 1 --2- 172.21.15.121:0/2450156517 >> v2:172.21.15.123:3300/0 conn(0x7f83a40d0a70 0x7f83a40ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:11.487 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.486+0000 7f83a3e00640 1 -- 172.21.15.121:0/2450156517 >> v2:172.21.15.123:3300/0 conn(0x7f83a40d0a70 msgr2=0x7f83a40ad680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:11.487 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.486+0000 7f83a3e00640 1 --2- 172.21.15.121:0/2450156517 >> v2:172.21.15.123:3300/0 conn(0x7f83a40d0a70 0x7f83a40ad680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:11.488 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.486+0000 7f83a3e00640 1 -- 172.21.15.121:0/2450156517 >> v1:172.21.15.123:6789/0 conn(0x7f83a406dbb0 legacy=0x7f83a40d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:11.488 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.486+0000 7f83a3e00640 1 -- 172.21.15.121:0/2450156517 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a406ed80 con 0x7f83a415e8a0 2024-04-20T10:22:11.488 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.488+0000 7f83a3e00640 1 --2- 172.21.15.121:0/2450156517 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83a415e8a0 0x7f83a406e1e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f83a40d0f70 tx=0x7f839c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208fe7900d184453 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:11.488 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.488+0000 7f83a2a00640 1 -- 172.21.15.121:0/2450156517 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c035020 con 0x7f83a415e8a0 2024-04-20T10:22:11.489 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.488+0000 7f83a2a00640 1 -- 172.21.15.121:0/2450156517 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f839c031870 con 0x7f83a415e8a0 2024-04-20T10:22:11.489 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.488+0000 7f83a2a00640 1 -- 172.21.15.121:0/2450156517 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c031c40 con 0x7f83a415e8a0 2024-04-20T10:22:11.489 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.489+0000 7f83aac00640 1 -- 172.21.15.121:0/2450156517 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83a415e8a0 msgr2=0x7f83a406e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:11.489 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.489+0000 7f83aac00640 1 --2- 172.21.15.121:0/2450156517 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83a415e8a0 0x7f83a406e1e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f83a40d0f70 tx=0x7f839c02b910 comp rx=0 tx=0).stop 2024-04-20T10:22:11.489 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.489+0000 7f83aac00640 1 -- 172.21.15.121:0/2450156517 shutdown_connections 2024-04-20T10:22:11.490 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.489+0000 7f83aac00640 1 --2- 172.21.15.121:0/2450156517 >> v2:172.21.15.123:3300/0 conn(0x7f83a40d0a70 0x7f83a40ad680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:11.490 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.489+0000 7f83aac00640 1 --2- 172.21.15.121:0/2450156517 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83a415e8a0 0x7f83a406e1e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:11.490 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.489+0000 7f83aac00640 1 -- 172.21.15.121:0/2450156517 >> 172.21.15.121:0/2450156517 conn(0x7f83a40b3350 msgr2=0x7f83a40b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:11.490 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.489+0000 7f83aac00640 1 -- 172.21.15.121:0/2450156517 shutdown_connections 2024-04-20T10:22:11.490 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.489+0000 7f83aac00640 1 -- 172.21.15.121:0/2450156517 wait complete. 2024-04-20T10:22:11.496 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.490+0000 7f83aac00640 1 Processor -- start 2024-04-20T10:22:11.496 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.491+0000 7f83aac00640 1 -- start start 2024-04-20T10:22:11.496 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.491+0000 7f83aac00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83a406dbb0 0x7f83a40ae030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:11.496 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.491+0000 7f83aac00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83a40d0a70 0x7f83a40b09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:11.496 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.492+0000 7f83aac00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f83a415e8a0 0x7f83a41f7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:11.496 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.492+0000 7f83aac00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f83a406fef0 con 0x7f83a406dbb0 2024-04-20T10:22:11.496 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.492+0000 7f83aac00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f83a406fd70 con 0x7f83a40d0a70 2024-04-20T10:22:11.496 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.492+0000 7f83aac00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f83a40adbc0 con 0x7f83a415e8a0 2024-04-20T10:22:11.496 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.492+0000 7f83a3e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83a406dbb0 0x7f83a40ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:11.496 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.492+0000 7f83a3e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83a406dbb0 0x7f83a40ae030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42502/0 (socket says 172.21.15.121:42502) 2024-04-20T10:22:11.496 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.492+0000 7f83a3e00640 1 -- 172.21.15.121:0/406367133 learned_addr learned my addr 172.21.15.121:0/406367133 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:11.496 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.492+0000 7f83a3e00640 1 -- 172.21.15.121:0/406367133 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f83a415e8a0 msgr2=0x7f83a41f7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:11.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.492+0000 7f83a3e00640 1 --2- 172.21.15.121:0/406367133 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f83a415e8a0 0x7f83a41f7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:11.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.492+0000 7f83a3400640 1 --2- 172.21.15.121:0/406367133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83a40d0a70 0x7f83a40b09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:11.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.492+0000 7f83a3e00640 1 -- 172.21.15.121:0/406367133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83a40d0a70 msgr2=0x7f83a40b09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:11.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.492+0000 7f83a3e00640 1 --2- 172.21.15.121:0/406367133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83a40d0a70 0x7f83a40b09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:11.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.492+0000 7f83a3e00640 1 -- 172.21.15.121:0/406367133 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a4151710 con 0x7f83a406dbb0 2024-04-20T10:22:11.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.493+0000 7f83a3e00640 1 --2- 172.21.15.121:0/406367133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83a406dbb0 0x7f83a40ae030 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f839c000c00 tx=0x7f839c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:11.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.493+0000 7f839be00640 1 -- 172.21.15.121:0/406367133 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c040070 con 0x7f83a406dbb0 2024-04-20T10:22:11.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.493+0000 7f839be00640 1 -- 172.21.15.121:0/406367133 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f839c030d00 con 0x7f83a406dbb0 2024-04-20T10:22:11.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.493+0000 7f839be00640 1 -- 172.21.15.121:0/406367133 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c035040 con 0x7f83a406dbb0 2024-04-20T10:22:11.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.493+0000 7f83aac00640 1 -- 172.21.15.121:0/406367133 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83a4154130 con 0x7f83a406dbb0 2024-04-20T10:22:11.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.493+0000 7f83aac00640 1 -- 172.21.15.121:0/406367133 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83a414d600 con 0x7f83a406dbb0 2024-04-20T10:22:11.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.494+0000 7f83aac00640 1 -- 172.21.15.121:0/406367133 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83a4068980 con 0x7f83a406dbb0 2024-04-20T10:22:11.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.495+0000 7f839be00640 1 -- 172.21.15.121:0/406367133 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f839c04b060 con 0x7f83a406dbb0 2024-04-20T10:22:11.498 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.496+0000 7f839be00640 1 --2- 172.21.15.121:0/406367133 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f838006c920 0x7f838006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:11.498 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.496+0000 7f839be00640 1 -- 172.21.15.121:0/406367133 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f839c0add20 con 0x7f83a406dbb0 2024-04-20T10:22:11.498 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.497+0000 7f83a3400640 1 --2- 172.21.15.121:0/406367133 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f838006c920 0x7f838006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:11.498 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.497+0000 7f83a3400640 1 --2- 172.21.15.121:0/406367133 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f838006c920 0x7f838006ede0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f83a414dd80 tx=0x7f838c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:11.501 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.501+0000 7f839be00640 1 -- 172.21.15.121:0/406367133 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f839c07bd40 con 0x7f83a406dbb0 2024-04-20T10:22:11.701 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.700+0000 7f83aac00640 1 -- 172.21.15.121:0/406367133 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f83a406d160 con 0x7f83a406dbb0 2024-04-20T10:22:11.702 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.701+0000 7f839be00640 1 -- 172.21.15.121:0/406367133 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f839c07b640 con 0x7f83a406dbb0 2024-04-20T10:22:11.702 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2024-04-20T10:22:11.705 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.704+0000 7f83aac00640 1 -- 172.21.15.121:0/406367133 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f838006c920 msgr2=0x7f838006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:11.705 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.704+0000 7f83aac00640 1 --2- 172.21.15.121:0/406367133 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f838006c920 0x7f838006ede0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f83a414dd80 tx=0x7f838c009040 comp rx=0 tx=0).stop 2024-04-20T10:22:11.705 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.704+0000 7f83aac00640 1 -- 172.21.15.121:0/406367133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83a406dbb0 msgr2=0x7f83a40ae030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:11.705 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.704+0000 7f83aac00640 1 --2- 172.21.15.121:0/406367133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83a406dbb0 0x7f83a40ae030 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f839c000c00 tx=0x7f839c0046c0 comp rx=0 tx=0).stop 2024-04-20T10:22:11.705 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.704+0000 7f83aac00640 1 -- 172.21.15.121:0/406367133 shutdown_connections 2024-04-20T10:22:11.705 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.704+0000 7f83aac00640 1 --2- 172.21.15.121:0/406367133 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f83a415e8a0 0x7f83a41f7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:11.705 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.704+0000 7f83aac00640 1 --2- 172.21.15.121:0/406367133 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f838006c920 0x7f838006ede0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:11.705 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.704+0000 7f83aac00640 1 --2- 172.21.15.121:0/406367133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83a40d0a70 0x7f83a40b09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:11.705 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.704+0000 7f83aac00640 1 --2- 172.21.15.121:0/406367133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f83a406dbb0 0x7f83a40ae030 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:11.705 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.704+0000 7f83aac00640 1 -- 172.21.15.121:0/406367133 >> 172.21.15.121:0/406367133 conn(0x7f83a40b3350 msgr2=0x7f83a415f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:11.705 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.704+0000 7f83aac00640 1 -- 172.21.15.121:0/406367133 shutdown_connections 2024-04-20T10:22:11.705 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.704+0000 7f83aac00640 1 -- 172.21.15.121:0/406367133 wait complete. 2024-04-20T10:22:11.725 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:22:11.930 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.929+0000 7f0713400640 1 Processor -- start 2024-04-20T10:22:11.930 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.930+0000 7f0713400640 1 -- start start 2024-04-20T10:22:11.931 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.931+0000 7f0713400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f070c0d0360 0x7f070c0d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:11.931 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.931+0000 7f0713400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f070c141c60 con 0x7f070c15ca40 2024-04-20T10:22:11.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.931+0000 7f0713400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f070c0adab0 con 0x7f070c156c20 2024-04-20T10:22:11.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.931+0000 7f0713400640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f070c06c010 con 0x7f070c0d0360 2024-04-20T10:22:11.934 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.931+0000 7f0703e00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f070c15ca40 0x7f070c0ad3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:34886/0 (socket says 172.21.15.121:34886) 2024-04-20T10:22:11.934 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.931+0000 7f0703e00640 1 -- 172.21.15.121:0/911810218 learned_addr learned my addr 172.21.15.121:0/911810218 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:11.934 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.932+0000 7f0711400640 1 --2- 172.21.15.121:0/911810218 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f070c0d0360 0x7f070c0d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:11.934 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.932+0000 7f070be00640 1 -- 172.21.15.121:0/911810218 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1885915950 0 0) 0x7f070c141c60 con 0x7f070c15ca40 2024-04-20T10:22:11.934 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.932+0000 7f070be00640 1 -- 172.21.15.121:0/911810218 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f06f0003520 con 0x7f070c15ca40 2024-04-20T10:22:11.934 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.932+0000 7f070be00640 1 -- 172.21.15.121:0/911810218 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3920293766 0 0) 0x7f06f0003520 con 0x7f070c15ca40 2024-04-20T10:22:11.934 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.932+0000 7f070be00640 1 -- 172.21.15.121:0/911810218 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f070c0d0360 msgr2=0x7f070c0d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:11.934 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.932+0000 7f070be00640 1 --2- 172.21.15.121:0/911810218 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f070c0d0360 0x7f070c0d07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:11.934 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.932+0000 7f070be00640 1 -- 172.21.15.121:0/911810218 >> v1:172.21.15.123:6789/0 conn(0x7f070c156c20 legacy=0x7f070c06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:11.934 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.932+0000 7f070be00640 1 -- 172.21.15.121:0/911810218 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f070c15cf90 con 0x7f070c15ca40 2024-04-20T10:22:11.934 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.932+0000 7f070be00640 1 -- 172.21.15.121:0/911810218 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f06f4001670 con 0x7f070c15ca40 2024-04-20T10:22:11.935 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.932+0000 7f070be00640 1 -- 172.21.15.121:0/911810218 >> v1:172.21.15.121:6789/0 conn(0x7f070c15ca40 legacy=0x7f070c0ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:11.935 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.932+0000 7f070be00640 1 --2- 172.21.15.121:0/911810218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06f00039e0 0x7f06f0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:11.935 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.932+0000 7f070be00640 1 --2- 172.21.15.121:0/911810218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06f0004800 0x7f06f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:11.935 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.932+0000 7f070be00640 1 --2- 172.21.15.121:0/911810218 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f070c0d0360 0x7f06f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:11.935 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.933+0000 7f070be00640 1 -- 172.21.15.121:0/911810218 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f070c06c010 con 0x7f06f0004800 2024-04-20T10:22:11.935 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.933+0000 7f070be00640 1 -- 172.21.15.121:0/911810218 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f06f000baa0 con 0x7f06f00039e0 2024-04-20T10:22:11.935 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.933+0000 7f070be00640 1 -- 172.21.15.121:0/911810218 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f06f000bca0 con 0x7f070c0d0360 2024-04-20T10:22:11.935 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.933+0000 7f0703e00640 1 --2- 172.21.15.121:0/911810218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06f0004800 0x7f06f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:11.935 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.933+0000 7f0710a00640 1 --2- 172.21.15.121:0/911810218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06f00039e0 0x7f06f0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:11.935 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.933+0000 7f0703e00640 1 -- 172.21.15.121:0/911810218 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f070c0d0360 msgr2=0x7f06f00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:11.935 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.933+0000 7f0711400640 1 --2- 172.21.15.121:0/911810218 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f070c0d0360 0x7f06f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:11.935 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.933+0000 7f0703e00640 1 --2- 172.21.15.121:0/911810218 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f070c0d0360 0x7f06f00051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:11.935 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.933+0000 7f0703e00640 1 -- 172.21.15.121:0/911810218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06f00039e0 msgr2=0x7f06f0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:11.935 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.933+0000 7f0703e00640 1 --2- 172.21.15.121:0/911810218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06f00039e0 0x7f06f0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:11.936 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.933+0000 7f0703e00640 1 -- 172.21.15.121:0/911810218 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f0003520 con 0x7f06f0004800 2024-04-20T10:22:11.937 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.934+0000 7f0703e00640 1 --2- 172.21.15.121:0/911810218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06f0004800 0x7f06f0004c80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f06f400c160 tx=0x7f06f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22cd2fa10eafce89 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:11.937 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.935+0000 7f070be00640 1 -- 172.21.15.121:0/911810218 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f4007c60 con 0x7f06f0004800 2024-04-20T10:22:11.937 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.935+0000 7f070be00640 1 -- 172.21.15.121:0/911810218 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06f4030070 con 0x7f06f0004800 2024-04-20T10:22:11.937 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.935+0000 7f070be00640 1 -- 172.21.15.121:0/911810218 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f4005d50 con 0x7f06f0004800 2024-04-20T10:22:11.937 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.935+0000 7f0713400640 1 -- 172.21.15.121:0/911810218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06f0004800 msgr2=0x7f06f0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:11.937 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.935+0000 7f0713400640 1 --2- 172.21.15.121:0/911810218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f06f0004800 0x7f06f0004c80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f06f400c160 tx=0x7f06f0005830 comp rx=0 tx=0).stop 2024-04-20T10:22:11.937 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.935+0000 7f0711400640 1 -- 172.21.15.121:0/911810218 reap_dead start 2024-04-20T10:22:11.937 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.935+0000 7f0713400640 1 -- 172.21.15.121:0/911810218 shutdown_connections 2024-04-20T10:22:11.937 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.935+0000 7f0713400640 1 -- 172.21.15.121:0/911810218 >> 172.21.15.121:0/911810218 conn(0x7f070c0b3350 msgr2=0x7f070c0b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:11.937 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.935+0000 7f0713400640 1 -- 172.21.15.121:0/911810218 shutdown_connections 2024-04-20T10:22:11.937 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.935+0000 7f0713400640 1 -- 172.21.15.121:0/911810218 wait complete. 2024-04-20T10:22:11.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.937+0000 7f0713400640 1 Processor -- start 2024-04-20T10:22:11.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.937+0000 7f0713400640 1 -- start start 2024-04-20T10:22:11.939 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.938+0000 7f0713400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06f0004800 0x7f070c0b1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:11.939 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.938+0000 7f0713400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f070c152590 0x7f070c1e5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:11.939 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.938+0000 7f0713400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f070c0cf4a0 0x7f070c1e6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:11.939 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.938+0000 7f0713400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f070c15d830 con 0x7f070c152590 2024-04-20T10:22:11.939 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.938+0000 7f0713400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f070c15d550 con 0x7f06f0004800 2024-04-20T10:22:11.939 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.938+0000 7f0713400640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f070c0d07f0 con 0x7f070c0cf4a0 2024-04-20T10:22:11.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.939+0000 7f0703e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f070c152590 0x7f070c1e5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:11.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.939+0000 7f0703e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f070c152590 0x7f070c1e5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42520/0 (socket says 172.21.15.121:42520) 2024-04-20T10:22:11.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.939+0000 7f0703e00640 1 -- 172.21.15.121:0/4287580531 learned_addr learned my addr 172.21.15.121:0/4287580531 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:11.941 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.939+0000 7f0710a00640 1 --2- 172.21.15.121:0/4287580531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06f0004800 0x7f070c0b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:11.941 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.939+0000 7f0703e00640 1 -- 172.21.15.121:0/4287580531 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f070c0cf4a0 msgr2=0x7f070c1e6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:11.941 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.939+0000 7f0711400640 1 --2- 172.21.15.121:0/4287580531 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f070c0cf4a0 0x7f070c1e6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:11.941 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.939+0000 7f0703e00640 1 --2- 172.21.15.121:0/4287580531 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f070c0cf4a0 0x7f070c1e6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:11.941 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.939+0000 7f0703e00640 1 -- 172.21.15.121:0/4287580531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06f0004800 msgr2=0x7f070c0b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:11.941 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.939+0000 7f0703e00640 1 --2- 172.21.15.121:0/4287580531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06f0004800 0x7f070c0b1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:11.941 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.939+0000 7f0703e00640 1 -- 172.21.15.121:0/4287580531 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f070c14ff00 con 0x7f070c152590 2024-04-20T10:22:11.941 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.939+0000 7f0703e00640 1 --2- 172.21.15.121:0/4287580531 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f070c152590 0x7f070c1e5e50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f06f402c460 tx=0x7f06f4001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:11.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.940+0000 7f0709600640 1 -- 172.21.15.121:0/4287580531 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f4045070 con 0x7f070c152590 2024-04-20T10:22:11.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.940+0000 7f0709600640 1 -- 172.21.15.121:0/4287580531 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06f4004bf0 con 0x7f070c152590 2024-04-20T10:22:11.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.940+0000 7f0709600640 1 -- 172.21.15.121:0/4287580531 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f4007c60 con 0x7f070c152590 2024-04-20T10:22:11.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.940+0000 7f0713400640 1 -- 172.21.15.121:0/4287580531 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f070c0b1f90 con 0x7f070c152590 2024-04-20T10:22:11.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.940+0000 7f0713400640 1 -- 172.21.15.121:0/4287580531 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f070c0b2550 con 0x7f070c152590 2024-04-20T10:22:11.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.941+0000 7f0713400640 1 -- 172.21.15.121:0/4287580531 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f070c0742b0 con 0x7f070c152590 2024-04-20T10:22:11.943 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.942+0000 7f0709600640 1 -- 172.21.15.121:0/4287580531 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f06f4007070 con 0x7f070c152590 2024-04-20T10:22:11.943 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.943+0000 7f0709600640 1 --2- 172.21.15.121:0/4287580531 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f06e806c8f0 0x7f06e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:11.943 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.943+0000 7f0709600640 1 -- 172.21.15.121:0/4287580531 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f06f40acdb0 con 0x7f070c152590 2024-04-20T10:22:11.944 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.943+0000 7f0710a00640 1 --2- 172.21.15.121:0/4287580531 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f06e806c8f0 0x7f06e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:11.944 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.944+0000 7f0710a00640 1 --2- 172.21.15.121:0/4287580531 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f06e806c8f0 0x7f06e806edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f06fc007fd0 tx=0x7f06fc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:11.948 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:11.947+0000 7f0709600640 1 -- 172.21.15.121:0/4287580531 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f070c0742b0 con 0x7f070c152590 2024-04-20T10:22:12.146 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.145+0000 7f0713400640 1 -- 172.21.15.121:0/4287580531 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f070c068980 con 0x7f070c152590 2024-04-20T10:22:12.146 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.146+0000 7f0709600640 1 -- 172.21.15.121:0/4287580531 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f070c14ff00 con 0x7f070c152590 2024-04-20T10:22:12.147 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 32 2024-04-20T10:22:12.150 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.149+0000 7f0713400640 1 -- 172.21.15.121:0/4287580531 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f06e806c8f0 msgr2=0x7f06e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:12.150 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.149+0000 7f0713400640 1 --2- 172.21.15.121:0/4287580531 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f06e806c8f0 0x7f06e806edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f06fc007fd0 tx=0x7f06fc009040 comp rx=0 tx=0).stop 2024-04-20T10:22:12.150 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.149+0000 7f0713400640 1 -- 172.21.15.121:0/4287580531 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f070c152590 msgr2=0x7f070c1e5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:12.150 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.149+0000 7f0713400640 1 --2- 172.21.15.121:0/4287580531 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f070c152590 0x7f070c1e5e50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f06f402c460 tx=0x7f06f4001970 comp rx=0 tx=0).stop 2024-04-20T10:22:12.150 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.149+0000 7f0713400640 1 -- 172.21.15.121:0/4287580531 shutdown_connections 2024-04-20T10:22:12.150 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.149+0000 7f0713400640 1 --2- 172.21.15.121:0/4287580531 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f070c0cf4a0 0x7f070c1e6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:12.150 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.149+0000 7f0713400640 1 --2- 172.21.15.121:0/4287580531 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f070c152590 0x7f070c1e5e50 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:12.150 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.149+0000 7f0713400640 1 --2- 172.21.15.121:0/4287580531 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f06e806c8f0 0x7f06e806edb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:12.150 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.149+0000 7f0713400640 1 --2- 172.21.15.121:0/4287580531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f06f0004800 0x7f070c0b1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:12.151 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.149+0000 7f0713400640 1 -- 172.21.15.121:0/4287580531 >> 172.21.15.121:0/4287580531 conn(0x7f070c0b3350 msgr2=0x7f070c154ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:12.151 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.149+0000 7f0713400640 1 -- 172.21.15.121:0/4287580531 shutdown_connections 2024-04-20T10:22:12.151 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.149+0000 7f0713400640 1 -- 172.21.15.121:0/4287580531 wait complete. 2024-04-20T10:22:12.169 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:22:12.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.373+0000 7f2414a00640 1 Processor -- start 2024-04-20T10:22:12.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.374+0000 7f2414a00640 1 -- start start 2024-04-20T10:22:12.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.375+0000 7f2414a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f24100d4270 0x7f24100d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:12.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.375+0000 7f2414a00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24100d0380 0x7f24100d0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:12.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.375+0000 7f2414a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24100d0d50 con 0x7f24100d4650 2024-04-20T10:22:12.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.375+0000 7f2414a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f24100d40c0 con 0x7f24100d4270 2024-04-20T10:22:12.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.375+0000 7f2414a00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f24100d3820 con 0x7f24100d0380 2024-04-20T10:22:12.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.375+0000 7f240de00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f24100d4270 0x7f24100d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:12.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.376+0000 7f240d400640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f24100d4650 0x7f24100b1290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:34896/0 (socket says 172.21.15.121:34896) 2024-04-20T10:22:12.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.376+0000 7f240d400640 1 -- 172.21.15.121:0/2421227092 learned_addr learned my addr 172.21.15.121:0/2421227092 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:12.377 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.376+0000 7f240de00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f24100d4270 0x7f24100d1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42532/0 (socket says 172.21.15.121:42532) 2024-04-20T10:22:12.377 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.376+0000 7f240de00640 1 -- 172.21.15.121:0/2421227092 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24100d0380 msgr2=0x7f24100d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:12.377 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.376+0000 7f240de00640 1 --2- 172.21.15.121:0/2421227092 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24100d0380 0x7f24100d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:12.378 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.376+0000 7f240de00640 1 -- 172.21.15.121:0/2421227092 >> v1:172.21.15.121:6789/0 conn(0x7f24100d4650 legacy=0x7f24100b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:12.378 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.376+0000 7f240de00640 1 -- 172.21.15.121:0/2421227092 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24100d3b00 con 0x7f24100d4270 2024-04-20T10:22:12.378 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.376+0000 7f240ca00640 1 -- 172.21.15.121:0/2421227092 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3942813640 0 0) 0x7f24100d0d50 con 0x7f24100d4650 2024-04-20T10:22:12.378 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.377+0000 7f240de00640 1 --2- 172.21.15.121:0/2421227092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24100d4270 0x7f24100d1a00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f24000097a0 tx=0x7f240002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf11df336080a44 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:12.379 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.378+0000 7f240ca00640 1 -- 172.21.15.121:0/2421227092 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2400035020 con 0x7f24100d4270 2024-04-20T10:22:12.379 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.378+0000 7f240ca00640 1 -- 172.21.15.121:0/2421227092 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2400031810 con 0x7f24100d4270 2024-04-20T10:22:12.379 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.378+0000 7f240ca00640 1 -- 172.21.15.121:0/2421227092 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2400031be0 con 0x7f24100d4270 2024-04-20T10:22:12.379 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.378+0000 7f2414a00640 1 -- 172.21.15.121:0/2421227092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24100d4270 msgr2=0x7f24100d1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:12.379 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.378+0000 7f2414a00640 1 --2- 172.21.15.121:0/2421227092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24100d4270 0x7f24100d1a00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f24000097a0 tx=0x7f240002b8b0 comp rx=0 tx=0).stop 2024-04-20T10:22:12.379 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.378+0000 7f2414a00640 1 -- 172.21.15.121:0/2421227092 shutdown_connections 2024-04-20T10:22:12.379 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.378+0000 7f2414a00640 1 --2- 172.21.15.121:0/2421227092 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24100d0380 0x7f24100d0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:12.379 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.378+0000 7f2414a00640 1 --2- 172.21.15.121:0/2421227092 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24100d4270 0x7f24100d1a00 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:12.380 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.378+0000 7f2414a00640 1 -- 172.21.15.121:0/2421227092 >> 172.21.15.121:0/2421227092 conn(0x7f24100b99c0 msgr2=0x7f24100b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:12.380 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.378+0000 7f2414a00640 1 -- 172.21.15.121:0/2421227092 shutdown_connections 2024-04-20T10:22:12.380 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.378+0000 7f2414a00640 1 -- 172.21.15.121:0/2421227092 wait complete. 2024-04-20T10:22:12.381 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.380+0000 7f2414a00640 1 Processor -- start 2024-04-20T10:22:12.381 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.380+0000 7f2414a00640 1 -- start start 2024-04-20T10:22:12.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.381+0000 7f2414a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24100d0380 0x7f24101f8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:12.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.381+0000 7f2414a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24100d4270 0x7f241015b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:12.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.381+0000 7f2414a00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24100d4650 0x7f24101f47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:12.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.381+0000 7f2414a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f24100b19a0 con 0x7f24100d4270 2024-04-20T10:22:12.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.381+0000 7f2414a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f24100cfe60 con 0x7f24100d0380 2024-04-20T10:22:12.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.381+0000 7f2414a00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f24100d53b0 con 0x7f24100d4650 2024-04-20T10:22:12.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.382+0000 7f240d400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24100d4270 0x7f241015b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:12.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.382+0000 7f240d400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24100d4270 0x7f241015b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42546/0 (socket says 172.21.15.121:42546) 2024-04-20T10:22:12.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.382+0000 7f240d400640 1 -- 172.21.15.121:0/3106963704 learned_addr learned my addr 172.21.15.121:0/3106963704 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:12.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.382+0000 7f240de00640 1 --2- 172.21.15.121:0/3106963704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24100d0380 0x7f24101f8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:12.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.382+0000 7f240e800640 1 --2- 172.21.15.121:0/3106963704 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24100d4650 0x7f24101f47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:12.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.382+0000 7f240d400640 1 -- 172.21.15.121:0/3106963704 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24100d4650 msgr2=0x7f24101f47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:12.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.382+0000 7f240d400640 1 --2- 172.21.15.121:0/3106963704 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24100d4650 0x7f24101f47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:12.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.382+0000 7f240d400640 1 -- 172.21.15.121:0/3106963704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24100d0380 msgr2=0x7f24101f8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:12.384 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.382+0000 7f240d400640 1 --2- 172.21.15.121:0/3106963704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24100d0380 0x7f24101f8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:12.384 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.382+0000 7f240d400640 1 -- 172.21.15.121:0/3106963704 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24101f5690 con 0x7f24100d4270 2024-04-20T10:22:12.384 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.382+0000 7f240d400640 1 --2- 172.21.15.121:0/3106963704 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24100d4270 0x7f241015b9c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f23f800c890 tx=0x7f23f800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:12.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.383+0000 7f2406000640 1 -- 172.21.15.121:0/3106963704 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f8018070 con 0x7f24100d4270 2024-04-20T10:22:12.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.383+0000 7f2406000640 1 -- 172.21.15.121:0/3106963704 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23f800a640 con 0x7f24100d4270 2024-04-20T10:22:12.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.383+0000 7f2406000640 1 -- 172.21.15.121:0/3106963704 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f800aa30 con 0x7f24100d4270 2024-04-20T10:22:12.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.383+0000 7f2414a00640 1 -- 172.21.15.121:0/3106963704 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24100cb270 con 0x7f24100d4270 2024-04-20T10:22:12.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.383+0000 7f2414a00640 1 -- 172.21.15.121:0/3106963704 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24101f7e60 con 0x7f24100d4270 2024-04-20T10:22:12.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.384+0000 7f2414a00640 1 -- 172.21.15.121:0/3106963704 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2410075c20 con 0x7f24100d4270 2024-04-20T10:22:12.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.385+0000 7f2406000640 1 -- 172.21.15.121:0/3106963704 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f23f801b070 con 0x7f24100d4270 2024-04-20T10:22:12.386 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.386+0000 7f2406000640 1 --2- 172.21.15.121:0/3106963704 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f23ec06c8f0 0x7f23ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:12.387 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.386+0000 7f2406000640 1 -- 172.21.15.121:0/3106963704 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f23f808b630 con 0x7f24100d4270 2024-04-20T10:22:12.387 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.386+0000 7f240de00640 1 --2- 172.21.15.121:0/3106963704 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f23ec06c8f0 0x7f23ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:12.387 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.387+0000 7f240de00640 1 --2- 172.21.15.121:0/3106963704 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f23ec06c8f0 0x7f23ec06edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2400036000 tx=0x7f2400008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:12.393 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.392+0000 7f2406000640 1 -- 172.21.15.121:0/3106963704 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2410075c20 con 0x7f24100d4270 2024-04-20T10:22:12.591 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.590+0000 7f2414a00640 1 -- 172.21.15.121:0/3106963704 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f24101612e0 con 0x7f24100d4270 2024-04-20T10:22:12.591 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.591+0000 7f2406000640 1 -- 172.21.15.121:0/3106963704 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f23f8059660 con 0x7f24100d4270 2024-04-20T10:22:12.592 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2024-04-20T10:22:12.594 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.594+0000 7f2414a00640 1 -- 172.21.15.121:0/3106963704 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f23ec06c8f0 msgr2=0x7f23ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:12.595 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.594+0000 7f2414a00640 1 --2- 172.21.15.121:0/3106963704 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f23ec06c8f0 0x7f23ec06edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2400036000 tx=0x7f2400008030 comp rx=0 tx=0).stop 2024-04-20T10:22:12.595 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.594+0000 7f2414a00640 1 -- 172.21.15.121:0/3106963704 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24100d4270 msgr2=0x7f241015b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:12.595 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.594+0000 7f2414a00640 1 --2- 172.21.15.121:0/3106963704 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24100d4270 0x7f241015b9c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f23f800c890 tx=0x7f23f800cd50 comp rx=0 tx=0).stop 2024-04-20T10:22:12.595 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.594+0000 7f2414a00640 1 -- 172.21.15.121:0/3106963704 shutdown_connections 2024-04-20T10:22:12.595 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.594+0000 7f2414a00640 1 --2- 172.21.15.121:0/3106963704 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f24100d4650 0x7f24101f47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:12.595 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.594+0000 7f2414a00640 1 --2- 172.21.15.121:0/3106963704 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f24100d4270 0x7f241015b9c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:12.595 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.594+0000 7f2414a00640 1 --2- 172.21.15.121:0/3106963704 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f23ec06c8f0 0x7f23ec06edb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:12.595 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.594+0000 7f2414a00640 1 --2- 172.21.15.121:0/3106963704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24100d0380 0x7f24101f8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:12.595 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.594+0000 7f2414a00640 1 -- 172.21.15.121:0/3106963704 >> 172.21.15.121:0/3106963704 conn(0x7f24100b99c0 msgr2=0x7f24101605f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:12.595 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.594+0000 7f2414a00640 1 -- 172.21.15.121:0/3106963704 shutdown_connections 2024-04-20T10:22:12.595 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.594+0000 7f2414a00640 1 -- 172.21.15.121:0/3106963704 wait complete. 2024-04-20T10:22:12.613 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T10:22:12.813 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.812+0000 7f4871200640 1 Processor -- start 2024-04-20T10:22:12.813 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.812+0000 7f4871200640 1 -- start start 2024-04-20T10:22:12.814 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.814+0000 7f4871200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f486c15e8a0 0x7f486c06e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:12.814 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.814+0000 7f4871200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f486c0d0a70 0x7f486c0ad680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:12.814 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.814+0000 7f4871200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f486c06f060 con 0x7f486c06dbb0 2024-04-20T10:22:12.814 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.814+0000 7f4871200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f486c06e7c0 con 0x7f486c0d0a70 2024-04-20T10:22:12.815 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.814+0000 7f4871200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f486c06eaa0 con 0x7f486c15e8a0 2024-04-20T10:22:12.815 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.814+0000 7f4869400640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f486c06dbb0 0x7f486c0d0360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:34908/0 (socket says 172.21.15.121:34908) 2024-04-20T10:22:12.815 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.814+0000 7f4869400640 1 -- 172.21.15.121:0/837908303 learned_addr learned my addr 172.21.15.121:0/837908303 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:12.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.815+0000 7f486a800640 1 --2- 172.21.15.121:0/837908303 >> v2:172.21.15.123:3300/0 conn(0x7f486c0d0a70 0x7f486c0ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:12.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.815+0000 7f4869e00640 1 --2- 172.21.15.121:0/837908303 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f486c15e8a0 0x7f486c06e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:12.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.815+0000 7f4868a00640 1 -- 172.21.15.121:0/837908303 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 894016368 0 0) 0x7f486c06f060 con 0x7f486c06dbb0 2024-04-20T10:22:12.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.815+0000 7f4868a00640 1 -- 172.21.15.121:0/837908303 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4850003520 con 0x7f486c06dbb0 2024-04-20T10:22:12.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.815+0000 7f4868a00640 1 -- 172.21.15.121:0/837908303 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3057813483 0 0) 0x7f4850003520 con 0x7f486c06dbb0 2024-04-20T10:22:12.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.815+0000 7f4868a00640 1 -- 172.21.15.121:0/837908303 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f486c15e8a0 msgr2=0x7f486c06e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:12.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.815+0000 7f4868a00640 1 --2- 172.21.15.121:0/837908303 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f486c15e8a0 0x7f486c06e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:12.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.815+0000 7f4868a00640 1 -- 172.21.15.121:0/837908303 >> v2:172.21.15.123:3300/0 conn(0x7f486c0d0a70 msgr2=0x7f486c0ad680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:12.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.815+0000 7f4868a00640 1 --2- 172.21.15.121:0/837908303 >> v2:172.21.15.123:3300/0 conn(0x7f486c0d0a70 0x7f486c0ad680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:12.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.815+0000 7f4868a00640 1 -- 172.21.15.121:0/837908303 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486c06ed80 con 0x7f486c06dbb0 2024-04-20T10:22:12.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.815+0000 7f4868a00640 1 -- 172.21.15.121:0/837908303 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f485c001670 con 0x7f486c06dbb0 2024-04-20T10:22:12.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.815+0000 7f4868a00640 1 -- 172.21.15.121:0/837908303 >> v1:172.21.15.121:6789/0 conn(0x7f486c06dbb0 legacy=0x7f486c0d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:12.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.815+0000 7f4868a00640 1 --2- 172.21.15.121:0/837908303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48500039b0 0x7f4850003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:12.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.815+0000 7f4868a00640 1 --2- 172.21.15.121:0/837908303 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f486c15e8a0 0x7f4850004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:12.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.815+0000 7f4868a00640 1 --2- 172.21.15.121:0/837908303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4850004d40 0x7f48500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:12.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.815+0000 7f4868a00640 1 -- 172.21.15.121:0/837908303 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f486c06e7c0 con 0x7f48500039b0 2024-04-20T10:22:12.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.815+0000 7f4868a00640 1 -- 172.21.15.121:0/837908303 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f486c06eaa0 con 0x7f4850004d40 2024-04-20T10:22:12.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.815+0000 7f4868a00640 1 -- 172.21.15.121:0/837908303 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f485000ba40 con 0x7f486c15e8a0 2024-04-20T10:22:12.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.815+0000 7f4869e00640 1 --2- 172.21.15.121:0/837908303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48500039b0 0x7f4850003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:12.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.816+0000 7f4869400640 1 --2- 172.21.15.121:0/837908303 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f486c15e8a0 0x7f4850004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:12.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.816+0000 7f486a800640 1 --2- 172.21.15.121:0/837908303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4850004d40 0x7f48500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:12.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.816+0000 7f4869e00640 1 -- 172.21.15.121:0/837908303 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f486c15e8a0 msgr2=0x7f4850004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:12.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.816+0000 7f4869e00640 1 --2- 172.21.15.121:0/837908303 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f486c15e8a0 0x7f4850004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:12.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.816+0000 7f4869e00640 1 -- 172.21.15.121:0/837908303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4850004d40 msgr2=0x7f48500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:12.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.816+0000 7f4869e00640 1 --2- 172.21.15.121:0/837908303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4850004d40 0x7f48500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:12.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.816+0000 7f4869e00640 1 -- 172.21.15.121:0/837908303 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4850003520 con 0x7f48500039b0 2024-04-20T10:22:12.819 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.817+0000 7f4869e00640 1 --2- 172.21.15.121:0/837908303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48500039b0 0x7f4850003dd0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f485400ba30 tx=0x7f4850005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9569a74a415f43f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:12.819 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.818+0000 7f4868a00640 1 -- 172.21.15.121:0/837908303 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4854036020 con 0x7f48500039b0 2024-04-20T10:22:12.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.818+0000 7f4868a00640 1 -- 172.21.15.121:0/837908303 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4854032640 con 0x7f48500039b0 2024-04-20T10:22:12.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.818+0000 7f4868a00640 1 -- 172.21.15.121:0/837908303 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485400b600 con 0x7f48500039b0 2024-04-20T10:22:12.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.818+0000 7f4871200640 1 -- 172.21.15.121:0/837908303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48500039b0 msgr2=0x7f4850003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:12.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.818+0000 7f4871200640 1 --2- 172.21.15.121:0/837908303 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f48500039b0 0x7f4850003dd0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f485400ba30 tx=0x7f4850005830 comp rx=0 tx=0).stop 2024-04-20T10:22:12.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.818+0000 7f486a800640 1 -- 172.21.15.121:0/837908303 reap_dead start 2024-04-20T10:22:12.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.818+0000 7f4871200640 1 -- 172.21.15.121:0/837908303 shutdown_connections 2024-04-20T10:22:12.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.818+0000 7f4871200640 1 -- 172.21.15.121:0/837908303 >> 172.21.15.121:0/837908303 conn(0x7f486c0b3350 msgr2=0x7f486c0b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:12.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.819+0000 7f4871200640 1 -- 172.21.15.121:0/837908303 shutdown_connections 2024-04-20T10:22:12.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.819+0000 7f4871200640 1 -- 172.21.15.121:0/837908303 wait complete. 2024-04-20T10:22:12.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.820+0000 7f4871200640 1 Processor -- start 2024-04-20T10:22:12.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.820+0000 7f4871200640 1 -- start start 2024-04-20T10:22:12.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.821+0000 7f4871200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48500039b0 0x7f486c09c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:12.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.821+0000 7f4871200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f486c09a240 0x7f486c158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:12.823 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.821+0000 7f4871200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f486c160710 0x7f486c160b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:12.823 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.821+0000 7f4871200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f486c06fef0 con 0x7f486c160710 2024-04-20T10:22:12.823 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.821+0000 7f4871200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f486c06fd70 con 0x7f48500039b0 2024-04-20T10:22:12.823 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.821+0000 7f4871200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f486c06e1e0 con 0x7f486c09a240 2024-04-20T10:22:12.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.822+0000 7f486a800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f486c160710 0x7f486c160b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:12.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.822+0000 7f486a800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f486c160710 0x7f486c160b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42564/0 (socket says 172.21.15.121:42564) 2024-04-20T10:22:12.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.822+0000 7f486a800640 1 -- 172.21.15.121:0/983033470 learned_addr learned my addr 172.21.15.121:0/983033470 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:12.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.822+0000 7f4869400640 1 --2- 172.21.15.121:0/983033470 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f486c09a240 0x7f486c158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:12.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.822+0000 7f4869e00640 1 --2- 172.21.15.121:0/983033470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48500039b0 0x7f486c09c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:12.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.822+0000 7f486a800640 1 -- 172.21.15.121:0/983033470 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f486c09a240 msgr2=0x7f486c158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:12.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.822+0000 7f486a800640 1 --2- 172.21.15.121:0/983033470 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f486c09a240 0x7f486c158920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:12.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.822+0000 7f486a800640 1 -- 172.21.15.121:0/983033470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48500039b0 msgr2=0x7f486c09c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:12.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.822+0000 7f486a800640 1 --2- 172.21.15.121:0/983033470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48500039b0 0x7f486c09c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:12.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.822+0000 7f486a800640 1 -- 172.21.15.121:0/983033470 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486c09a6c0 con 0x7f486c160710 2024-04-20T10:22:12.825 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.823+0000 7f486a800640 1 --2- 172.21.15.121:0/983033470 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f486c160710 0x7f486c160b90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f486c0d0f90 tx=0x7f48640115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:12.825 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.823+0000 7f4862000640 1 -- 172.21.15.121:0/983033470 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4864011e40 con 0x7f486c160710 2024-04-20T10:22:12.825 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.823+0000 7f4862000640 1 -- 172.21.15.121:0/983033470 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f486401b070 con 0x7f486c160710 2024-04-20T10:22:12.825 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.823+0000 7f4871200640 1 -- 172.21.15.121:0/983033470 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f486c09cdf0 con 0x7f486c160710 2024-04-20T10:22:12.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.823+0000 7f4862000640 1 -- 172.21.15.121:0/983033470 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4864003990 con 0x7f486c160710 2024-04-20T10:22:12.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.823+0000 7f4871200640 1 -- 172.21.15.121:0/983033470 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f486c09d3b0 con 0x7f486c160710 2024-04-20T10:22:12.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.824+0000 7f4871200640 1 -- 172.21.15.121:0/983033470 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f486c068980 con 0x7f486c160710 2024-04-20T10:22:12.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.825+0000 7f4862000640 1 -- 172.21.15.121:0/983033470 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4864005d10 con 0x7f486c160710 2024-04-20T10:22:12.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.826+0000 7f4862000640 1 --2- 172.21.15.121:0/983033470 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f484806c920 0x7f484806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:12.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.826+0000 7f4862000640 1 -- 172.21.15.121:0/983033470 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f486408c1d0 con 0x7f486c160710 2024-04-20T10:22:12.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.826+0000 7f4869e00640 1 --2- 172.21.15.121:0/983033470 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f484806c920 0x7f484806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:12.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.827+0000 7f4869e00640 1 --2- 172.21.15.121:0/983033470 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f484806c920 0x7f484806ede0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f485400ba00 tx=0x7f485403d040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:12.831 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:12.831+0000 7f4862000640 1 -- 172.21.15.121:0/983033470 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f486401d310 con 0x7f486c160710 2024-04-20T10:22:13.029 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.029+0000 7f4871200640 1 -- 172.21.15.121:0/983033470 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f486c06d160 con 0x7f486c160710 2024-04-20T10:22:13.030 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.030+0000 7f4862000640 1 -- 172.21.15.121:0/983033470 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f486405a1f0 con 0x7f486c160710 2024-04-20T10:22:13.030 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 32 2024-04-20T10:22:13.033 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.032+0000 7f4871200640 1 -- 172.21.15.121:0/983033470 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f484806c920 msgr2=0x7f484806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:13.033 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.032+0000 7f4871200640 1 --2- 172.21.15.121:0/983033470 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f484806c920 0x7f484806ede0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f485400ba00 tx=0x7f485403d040 comp rx=0 tx=0).stop 2024-04-20T10:22:13.033 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.032+0000 7f4871200640 1 -- 172.21.15.121:0/983033470 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f486c160710 msgr2=0x7f486c160b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:13.033 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.032+0000 7f4871200640 1 --2- 172.21.15.121:0/983033470 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f486c160710 0x7f486c160b90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f486c0d0f90 tx=0x7f48640115a0 comp rx=0 tx=0).stop 2024-04-20T10:22:13.033 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.032+0000 7f4871200640 1 -- 172.21.15.121:0/983033470 shutdown_connections 2024-04-20T10:22:13.034 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.033+0000 7f4871200640 1 --2- 172.21.15.121:0/983033470 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f486c160710 0x7f486c160b90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:13.034 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.033+0000 7f4871200640 1 --2- 172.21.15.121:0/983033470 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f486c09a240 0x7f486c158920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:13.034 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.033+0000 7f4871200640 1 --2- 172.21.15.121:0/983033470 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f484806c920 0x7f484806ede0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:13.034 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.033+0000 7f4871200640 1 --2- 172.21.15.121:0/983033470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48500039b0 0x7f486c09c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:13.034 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.033+0000 7f4871200640 1 -- 172.21.15.121:0/983033470 >> 172.21.15.121:0/983033470 conn(0x7f486c0b3350 msgr2=0x7f486c071b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:13.034 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.033+0000 7f4871200640 1 -- 172.21.15.121:0/983033470 shutdown_connections 2024-04-20T10:22:13.034 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.033+0000 7f4871200640 1 -- 172.21.15.121:0/983033470 wait complete. 2024-04-20T10:22:13.052 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T10:22:13.251 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.250+0000 7f016b200640 1 Processor -- start 2024-04-20T10:22:13.251 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.251+0000 7f016b200640 1 -- start start 2024-04-20T10:22:13.252 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.252+0000 7f016b200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f016c161d50 0x7f016c0d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:13.252 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.252+0000 7f016b200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f016c06cde0 0x7f016c0b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:13.253 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.252+0000 7f016b200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f016c06c190 con 0x7f016c0b3330 2024-04-20T10:22:13.253 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.252+0000 7f016b200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f016c162130 con 0x7f016c161d50 2024-04-20T10:22:13.253 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.252+0000 7f016b200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f016c06c470 con 0x7f016c06cde0 2024-04-20T10:22:13.254 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.252+0000 7f0169400640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f016c0b3330 0x7f016c06c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:34910/0 (socket says 172.21.15.121:34910) 2024-04-20T10:22:13.254 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.252+0000 7f0169400640 1 -- 172.21.15.121:0/1813234315 learned_addr learned my addr 172.21.15.121:0/1813234315 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:13.255 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.253+0000 7f0169e00640 1 --2- 172.21.15.121:0/1813234315 >> v2:172.21.15.123:3300/0 conn(0x7f016c161d50 0x7f016c0d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:13.255 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.253+0000 7f016a800640 1 --2- 172.21.15.121:0/1813234315 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f016c06cde0 0x7f016c0b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:13.255 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.253+0000 7f0168a00640 1 -- 172.21.15.121:0/1813234315 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 293418409 0 0) 0x7f016c06c190 con 0x7f016c0b3330 2024-04-20T10:22:13.255 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.253+0000 7f0168a00640 1 -- 172.21.15.121:0/1813234315 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f014c003520 con 0x7f016c0b3330 2024-04-20T10:22:13.255 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.253+0000 7f0168a00640 1 -- 172.21.15.121:0/1813234315 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2098902963 0 0) 0x7f014c003520 con 0x7f016c0b3330 2024-04-20T10:22:13.255 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.253+0000 7f0168a00640 1 -- 172.21.15.121:0/1813234315 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f016c06cde0 msgr2=0x7f016c0b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:13.255 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.253+0000 7f0168a00640 1 --2- 172.21.15.121:0/1813234315 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f016c06cde0 0x7f016c0b3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:13.255 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.253+0000 7f0168a00640 1 -- 172.21.15.121:0/1813234315 >> v2:172.21.15.123:3300/0 conn(0x7f016c161d50 msgr2=0x7f016c0d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:13.255 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.253+0000 7f0168a00640 1 --2- 172.21.15.121:0/1813234315 >> v2:172.21.15.123:3300/0 conn(0x7f016c161d50 0x7f016c0d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:13.255 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.253+0000 7f0168a00640 1 -- 172.21.15.121:0/1813234315 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f016c0d2e00 con 0x7f016c0b3330 2024-04-20T10:22:13.255 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.253+0000 7f0168a00640 1 -- 172.21.15.121:0/1813234315 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f0154001a10 con 0x7f016c0b3330 2024-04-20T10:22:13.255 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.253+0000 7f0168a00640 1 -- 172.21.15.121:0/1813234315 >> v1:172.21.15.121:6789/0 conn(0x7f016c0b3330 legacy=0x7f016c06c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:13.255 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.253+0000 7f0168a00640 1 --2- 172.21.15.121:0/1813234315 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f014c0039b0 0x7f014c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:13.255 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.253+0000 7f0168a00640 1 --2- 172.21.15.121:0/1813234315 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f016c06cde0 0x7f014c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:13.256 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.254+0000 7f0168a00640 1 --2- 172.21.15.121:0/1813234315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f014c004d40 0x7f014c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:13.256 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.254+0000 7f0168a00640 1 -- 172.21.15.121:0/1813234315 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f016c06c470 con 0x7f014c0039b0 2024-04-20T10:22:13.256 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.254+0000 7f0168a00640 1 -- 172.21.15.121:0/1813234315 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f014c00b9c0 con 0x7f014c004d40 2024-04-20T10:22:13.256 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.254+0000 7f0168a00640 1 -- 172.21.15.121:0/1813234315 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f014c00bb40 con 0x7f016c06cde0 2024-04-20T10:22:13.256 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.254+0000 7f0169e00640 1 --2- 172.21.15.121:0/1813234315 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f014c0039b0 0x7f014c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:13.256 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.254+0000 7f0169e00640 1 -- 172.21.15.121:0/1813234315 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f016c06cde0 msgr2=0x7f014c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:13.256 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.254+0000 7f0169e00640 1 --2- 172.21.15.121:0/1813234315 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f016c06cde0 0x7f014c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:13.256 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.254+0000 7f0169e00640 1 -- 172.21.15.121:0/1813234315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f014c004d40 msgr2=0x7f014c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:13.256 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.254+0000 7f0169e00640 1 --2- 172.21.15.121:0/1813234315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f014c004d40 0x7f014c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:13.256 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.254+0000 7f0169e00640 1 -- 172.21.15.121:0/1813234315 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f014c003520 con 0x7f014c0039b0 2024-04-20T10:22:13.257 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.255+0000 7f0169e00640 1 --2- 172.21.15.121:0/1813234315 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f014c0039b0 0x7f014c003dd0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f015c009ac0 tx=0x7f014c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=547c3db07f7c83c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:13.257 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.256+0000 7f0168a00640 1 -- 172.21.15.121:0/1813234315 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c039070 con 0x7f014c0039b0 2024-04-20T10:22:13.257 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.256+0000 7f0168a00640 1 -- 172.21.15.121:0/1813234315 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f015c02c6d0 con 0x7f014c0039b0 2024-04-20T10:22:13.257 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.256+0000 7f0168a00640 1 -- 172.21.15.121:0/1813234315 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c009690 con 0x7f014c0039b0 2024-04-20T10:22:13.257 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.256+0000 7f016b200640 1 -- 172.21.15.121:0/1813234315 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f014c0039b0 msgr2=0x7f014c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:13.258 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.256+0000 7f016b200640 1 --2- 172.21.15.121:0/1813234315 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f014c0039b0 0x7f014c003dd0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f015c009ac0 tx=0x7f014c005830 comp rx=0 tx=0).stop 2024-04-20T10:22:13.258 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.256+0000 7f016a800640 1 -- 172.21.15.121:0/1813234315 reap_dead start 2024-04-20T10:22:13.258 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.256+0000 7f016b200640 1 -- 172.21.15.121:0/1813234315 shutdown_connections 2024-04-20T10:22:13.258 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.256+0000 7f016b200640 1 -- 172.21.15.121:0/1813234315 >> 172.21.15.121:0/1813234315 conn(0x7f016c0d4450 msgr2=0x7f016c0d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:13.258 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.256+0000 7f016b200640 1 -- 172.21.15.121:0/1813234315 shutdown_connections 2024-04-20T10:22:13.258 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.256+0000 7f016b200640 1 -- 172.21.15.121:0/1813234315 wait complete. 2024-04-20T10:22:13.258 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.258+0000 7f016b200640 1 Processor -- start 2024-04-20T10:22:13.258 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.258+0000 7f016b200640 1 -- start start 2024-04-20T10:22:13.260 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.259+0000 7f016b200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f014c0039b0 0x7f016c1f71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:13.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.259+0000 7f016b200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f016c0b8420 0x7f016c0b88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:13.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.259+0000 7f016b200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f016c0cc840 0x7f016c0cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:13.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.259+0000 7f016b200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f016c0b3880 con 0x7f014c0039b0 2024-04-20T10:22:13.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.259+0000 7f016b200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f016c0d33c0 con 0x7f016c0b8420 2024-04-20T10:22:13.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.259+0000 7f016b200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f016c0b3a00 con 0x7f016c0cc840 2024-04-20T10:22:13.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.259+0000 7f0169e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f014c0039b0 0x7f016c1f71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:13.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.259+0000 7f0169e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f014c0039b0 0x7f016c1f71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42580/0 (socket says 172.21.15.121:42580) 2024-04-20T10:22:13.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.259+0000 7f0169e00640 1 -- 172.21.15.121:0/525300602 learned_addr learned my addr 172.21.15.121:0/525300602 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:13.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.260+0000 7f0169400640 1 --2- 172.21.15.121:0/525300602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f016c0b8420 0x7f016c0b88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:13.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.260+0000 7f0169e00640 1 -- 172.21.15.121:0/525300602 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f016c0cc840 msgr2=0x7f016c0cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:13.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.260+0000 7f016a800640 1 --2- 172.21.15.121:0/525300602 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f016c0cc840 0x7f016c0cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:13.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.260+0000 7f0169e00640 1 --2- 172.21.15.121:0/525300602 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f016c0cc840 0x7f016c0cccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:13.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.260+0000 7f0169e00640 1 -- 172.21.15.121:0/525300602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f016c0b8420 msgr2=0x7f016c0b88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:13.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.260+0000 7f0169e00640 1 --2- 172.21.15.121:0/525300602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f016c0b8420 0x7f016c0b88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:13.262 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.260+0000 7f0169e00640 1 -- 172.21.15.121:0/525300602 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f016c0cd5b0 con 0x7f014c0039b0 2024-04-20T10:22:13.262 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.260+0000 7f0169400640 1 --2- 172.21.15.121:0/525300602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f016c0b8420 0x7f016c0b88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:13.262 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.260+0000 7f0169e00640 1 --2- 172.21.15.121:0/525300602 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f014c0039b0 0x7f016c1f71b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f015c000c00 tx=0x7f015c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:13.262 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.261+0000 7f0162000640 1 -- 172.21.15.121:0/525300602 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c039050 con 0x7f014c0039b0 2024-04-20T10:22:13.263 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.261+0000 7f0162000640 1 -- 172.21.15.121:0/525300602 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f015c045070 con 0x7f014c0039b0 2024-04-20T10:22:13.263 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.261+0000 7f0162000640 1 -- 172.21.15.121:0/525300602 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015c0044e0 con 0x7f014c0039b0 2024-04-20T10:22:13.263 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.261+0000 7f016b200640 1 -- 172.21.15.121:0/525300602 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f016c0cd890 con 0x7f014c0039b0 2024-04-20T10:22:13.263 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.261+0000 7f016b200640 1 -- 172.21.15.121:0/525300602 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f016c0cde50 con 0x7f014c0039b0 2024-04-20T10:22:13.263 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.262+0000 7f016b200640 1 -- 172.21.15.121:0/525300602 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f016c075b60 con 0x7f014c0039b0 2024-04-20T10:22:13.263 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.263+0000 7f0162000640 1 -- 172.21.15.121:0/525300602 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f015c003a50 con 0x7f014c0039b0 2024-04-20T10:22:13.264 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.264+0000 7f0162000640 1 --2- 172.21.15.121:0/525300602 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f014806c8f0 0x7f014806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:13.264 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.264+0000 7f0162000640 1 -- 172.21.15.121:0/525300602 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f015c0ada50 con 0x7f014c0039b0 2024-04-20T10:22:13.264 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.264+0000 7f0169400640 1 --2- 172.21.15.121:0/525300602 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f014806c8f0 0x7f014806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:13.265 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.265+0000 7f0169400640 1 --2- 172.21.15.121:0/525300602 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f014806c8f0 0x7f014806edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f016c155170 tx=0x7f0154009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:13.269 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.269+0000 7f0162000640 1 -- 172.21.15.121:0/525300602 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f015c04e020 con 0x7f014c0039b0 2024-04-20T10:22:13.483 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.482+0000 7f016b200640 1 -- 172.21.15.121:0/525300602 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f016c160e90 con 0x7f014c0039b0 2024-04-20T10:22:13.484 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.483+0000 7f0162000640 1 -- 172.21.15.121:0/525300602 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f015c07ba70 con 0x7f014c0039b0 2024-04-20T10:22:13.484 INFO:teuthology.orchestra.run.smithi121.stderr:listed 0 entries 2024-04-20T10:22:13.487 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.486+0000 7f016b200640 1 -- 172.21.15.121:0/525300602 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f014806c8f0 msgr2=0x7f014806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:13.487 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.486+0000 7f016b200640 1 --2- 172.21.15.121:0/525300602 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f014806c8f0 0x7f014806edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f016c155170 tx=0x7f0154009040 comp rx=0 tx=0).stop 2024-04-20T10:22:13.487 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.486+0000 7f016b200640 1 -- 172.21.15.121:0/525300602 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f014c0039b0 msgr2=0x7f016c1f71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:13.487 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.486+0000 7f016b200640 1 --2- 172.21.15.121:0/525300602 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f014c0039b0 0x7f016c1f71b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f015c000c00 tx=0x7f015c004730 comp rx=0 tx=0).stop 2024-04-20T10:22:13.487 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.486+0000 7f016b200640 1 -- 172.21.15.121:0/525300602 shutdown_connections 2024-04-20T10:22:13.488 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.486+0000 7f016b200640 1 --2- 172.21.15.121:0/525300602 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f016c0cc840 0x7f016c0cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:13.488 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.486+0000 7f016b200640 1 --2- 172.21.15.121:0/525300602 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f014806c8f0 0x7f014806edb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:13.488 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.486+0000 7f016b200640 1 --2- 172.21.15.121:0/525300602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f016c0b8420 0x7f016c0b88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:13.488 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.486+0000 7f016b200640 1 --2- 172.21.15.121:0/525300602 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f014c0039b0 0x7f016c1f71b0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:13.488 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.486+0000 7f016b200640 1 -- 172.21.15.121:0/525300602 >> 172.21.15.121:0/525300602 conn(0x7f016c0d4450 msgr2=0x7f016c0d2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:13.488 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.486+0000 7f016b200640 1 -- 172.21.15.121:0/525300602 shutdown_connections 2024-04-20T10:22:13.488 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.486+0000 7f016b200640 1 -- 172.21.15.121:0/525300602 wait complete. 2024-04-20T10:22:13.506 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T10:22:13.506 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:22:13.559 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:22:13.763 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.762+0000 7fa343000640 1 Processor -- start 2024-04-20T10:22:13.763 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.762+0000 7fa343000640 1 -- start start 2024-04-20T10:22:13.764 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.764+0000 7fa343000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa33c06dcb0 0x7fa33c071b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:13.765 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.764+0000 7fa343000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa33c072090 0x7fa33c0d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:13.765 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.764+0000 7fa343000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa33c0ade40 con 0x7fa33c15b3b0 2024-04-20T10:22:13.765 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.764+0000 7fa343000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fa33c06f1b0 con 0x7fa33c072090 2024-04-20T10:22:13.765 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.764+0000 7fa343000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fa33c0ae0e0 con 0x7fa33c06dcb0 2024-04-20T10:22:13.765 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.764+0000 7fa33be00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fa33c15b3b0 0x7fa33c06eaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:34926/0 (socket says 172.21.15.121:34926) 2024-04-20T10:22:13.765 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.764+0000 7fa33be00640 1 -- 172.21.15.121:0/2532070834 learned_addr learned my addr 172.21.15.121:0/2532070834 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:13.765 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.764+0000 7fa341000640 1 --2- 172.21.15.121:0/2532070834 >> v2:172.21.15.121:3300/0 conn(0x7fa33c072090 0x7fa33c0d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:13.765 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.764+0000 7fa33b400640 1 --2- 172.21.15.121:0/2532070834 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa33c06dcb0 0x7fa33c071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:13.766 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.764+0000 7fa33aa00640 1 -- 172.21.15.121:0/2532070834 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4018812590 0 0) 0x7fa33c0ade40 con 0x7fa33c15b3b0 2024-04-20T10:22:13.766 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.765+0000 7fa33aa00640 1 -- 172.21.15.121:0/2532070834 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa320005730 con 0x7fa33c15b3b0 2024-04-20T10:22:13.766 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.765+0000 7fa341000640 1 -- 172.21.15.121:0/2532070834 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa33c06dcb0 msgr2=0x7fa33c071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:13.766 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.765+0000 7fa341000640 1 --2- 172.21.15.121:0/2532070834 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa33c06dcb0 0x7fa33c071b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:13.766 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.765+0000 7fa341000640 1 -- 172.21.15.121:0/2532070834 >> v1:172.21.15.121:6789/0 conn(0x7fa33c15b3b0 legacy=0x7fa33c06eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:13.766 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.765+0000 7fa341000640 1 -- 172.21.15.121:0/2532070834 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa33c15b790 con 0x7fa33c072090 2024-04-20T10:22:13.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.766+0000 7fa341000640 1 --2- 172.21.15.121:0/2532070834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa33c072090 0x7fa33c0d0360 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa334009d90 tx=0x7fa33402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d818472ee70809 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:13.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.766+0000 7fa33aa00640 1 -- 172.21.15.121:0/2532070834 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa334039070 con 0x7fa33c072090 2024-04-20T10:22:13.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.766+0000 7fa33aa00640 1 -- 172.21.15.121:0/2532070834 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa33402cc60 con 0x7fa33c072090 2024-04-20T10:22:13.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.766+0000 7fa33aa00640 1 -- 172.21.15.121:0/2532070834 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa334034660 con 0x7fa33c072090 2024-04-20T10:22:13.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.767+0000 7fa343000640 1 -- 172.21.15.121:0/2532070834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa33c072090 msgr2=0x7fa33c0d0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:13.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.767+0000 7fa343000640 1 --2- 172.21.15.121:0/2532070834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa33c072090 0x7fa33c0d0360 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa334009d90 tx=0x7fa33402beb0 comp rx=0 tx=0).stop 2024-04-20T10:22:13.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.767+0000 7fa343000640 1 -- 172.21.15.121:0/2532070834 shutdown_connections 2024-04-20T10:22:13.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.767+0000 7fa343000640 1 --2- 172.21.15.121:0/2532070834 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa33c072090 0x7fa33c0d0360 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:13.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.767+0000 7fa343000640 1 --2- 172.21.15.121:0/2532070834 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa33c06dcb0 0x7fa33c071b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:13.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.767+0000 7fa343000640 1 -- 172.21.15.121:0/2532070834 >> 172.21.15.121:0/2532070834 conn(0x7fa33c0b3350 msgr2=0x7fa33c0b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:13.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.767+0000 7fa343000640 1 -- 172.21.15.121:0/2532070834 shutdown_connections 2024-04-20T10:22:13.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.767+0000 7fa343000640 1 -- 172.21.15.121:0/2532070834 wait complete. 2024-04-20T10:22:13.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.769+0000 7fa343000640 1 Processor -- start 2024-04-20T10:22:13.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.769+0000 7fa343000640 1 -- start start 2024-04-20T10:22:13.771 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.770+0000 7fa343000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa33c06dcb0 0x7fa33c1501d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:13.771 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.770+0000 7fa343000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa33c15b3b0 0x7fa33c15c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:13.771 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.770+0000 7fa343000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa33c15cb30 0x7fa33c153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:13.771 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.770+0000 7fa343000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c0ae940 con 0x7fa33c15b3b0 2024-04-20T10:22:13.772 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.770+0000 7fa343000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa33c0ae420 con 0x7fa33c15cb30 2024-04-20T10:22:13.772 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.770+0000 7fa343000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fa33c0aec20 con 0x7fa33c06dcb0 2024-04-20T10:22:13.772 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.770+0000 7fa33b400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa33c15b3b0 0x7fa33c15c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:13.772 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.770+0000 7fa33b400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa33c15b3b0 0x7fa33c15c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42602/0 (socket says 172.21.15.121:42602) 2024-04-20T10:22:13.772 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.770+0000 7fa33b400640 1 -- 172.21.15.121:0/366358052 learned_addr learned my addr 172.21.15.121:0/366358052 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:13.772 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.771+0000 7fa33be00640 1 --2- 172.21.15.121:0/366358052 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa33c06dcb0 0x7fa33c1501d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:13.772 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.771+0000 7fa341000640 1 --2- 172.21.15.121:0/366358052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa33c15cb30 0x7fa33c153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:13.772 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.771+0000 7fa33b400640 1 -- 172.21.15.121:0/366358052 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa33c06dcb0 msgr2=0x7fa33c1501d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:13.772 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.771+0000 7fa33b400640 1 --2- 172.21.15.121:0/366358052 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa33c06dcb0 0x7fa33c1501d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:13.772 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.771+0000 7fa33b400640 1 -- 172.21.15.121:0/366358052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa33c15cb30 msgr2=0x7fa33c153f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:13.773 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.771+0000 7fa33b400640 1 --2- 172.21.15.121:0/366358052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa33c15cb30 0x7fa33c153f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:13.773 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.771+0000 7fa33b400640 1 -- 172.21.15.121:0/366358052 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa33c153180 con 0x7fa33c15b3b0 2024-04-20T10:22:13.773 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.771+0000 7fa33b400640 1 --2- 172.21.15.121:0/366358052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa33c15b3b0 0x7fa33c15c5f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa32800d800 tx=0x7fa32800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:13.774 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.771+0000 7fa333e00640 1 -- 172.21.15.121:0/366358052 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32800b6c0 con 0x7fa33c15b3b0 2024-04-20T10:22:13.774 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.771+0000 7fa333e00640 1 -- 172.21.15.121:0/366358052 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa328019070 con 0x7fa33c15b3b0 2024-04-20T10:22:13.774 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.771+0000 7fa333e00640 1 -- 172.21.15.121:0/366358052 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32800ba80 con 0x7fa33c15b3b0 2024-04-20T10:22:13.774 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.771+0000 7fa343000640 1 -- 172.21.15.121:0/366358052 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa33c153420 con 0x7fa33c15b3b0 2024-04-20T10:22:13.774 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.772+0000 7fa343000640 1 -- 172.21.15.121:0/366358052 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa33c15d040 con 0x7fa33c15b3b0 2024-04-20T10:22:13.774 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.773+0000 7fa343000640 1 -- 172.21.15.121:0/366358052 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa33c153bd0 con 0x7fa33c15b3b0 2024-04-20T10:22:13.774 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.773+0000 7fa333e00640 1 -- 172.21.15.121:0/366358052 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa32800bc20 con 0x7fa33c15b3b0 2024-04-20T10:22:13.775 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.775+0000 7fa333e00640 1 --2- 172.21.15.121:0/366358052 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fa31c06c920 0x7fa31c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:13.775 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.775+0000 7fa333e00640 1 -- 172.21.15.121:0/366358052 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa32808c480 con 0x7fa33c15b3b0 2024-04-20T10:22:13.775 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.775+0000 7fa33be00640 1 --2- 172.21.15.121:0/366358052 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fa31c06c920 0x7fa31c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:13.776 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.776+0000 7fa33be00640 1 --2- 172.21.15.121:0/366358052 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fa31c06c920 0x7fa31c06ede0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa32c007fd0 tx=0x7fa32c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:13.780 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.779+0000 7fa333e00640 1 -- 172.21.15.121:0/366358052 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa32801c360 con 0x7fa33c15b3b0 2024-04-20T10:22:13.981 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.981+0000 7fa343000640 1 -- 172.21.15.121:0/366358052 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa33c074dd0 con 0x7fa33c15b3b0 2024-04-20T10:22:13.984 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.984+0000 7fa333e00640 1 -- 172.21.15.121:0/366358052 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18940 (secure 0 0 0) 0x7fa32805a410 con 0x7fa33c15b3b0 2024-04-20T10:22:13.985 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:22:13.985 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":27,"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","created":"2024-04-20T10:20:38.076076+0000","modified":"2024-04-20T10:21:44.715898+0000","last_up_change":"2024-04-20T10:20:50.162700+0000","last_in_change":"2024-04-20T10:20:41.510382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:20:51.141365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:21:00.436937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:21:01.026957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:21:04.862693+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"629a7b8c-a40a-4a2a-a3cc-d9f357774310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6809","nonce":1044588835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6811","nonce":1044588835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6815","nonce":1044588835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6813","nonce":1044588835}]},"public_addr":"172.21.15.121:6809/1044588835","cluster_addr":"172.21.15.121:6811/1044588835","heartbeat_back_addr":"172.21.15.121:6815/1044588835","heartbeat_front_addr":"172.21.15.121:6813/1044588835","state":["exists","up"]},{"osd":1,"uuid":"f2570399-aa50-4ff3-bf5d-3d09207290db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6801","nonce":3505199624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6803","nonce":3505199624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6807","nonce":3505199624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6805","nonce":3505199624}]},"public_addr":"172.21.15.121:6801/3505199624","cluster_addr":"172.21.15.121:6803/3505199624","heartbeat_back_addr":"172.21.15.121:6807/3505199624","heartbeat_front_addr":"172.21.15.121:6805/3505199624","state":["exists","up"]},{"osd":2,"uuid":"50893807-dc30-49f4-b41f-739366626658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6817","nonce":3656724207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6819","nonce":3656724207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6827","nonce":3656724207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6822","nonce":3656724207}]},"public_addr":"172.21.15.121:6817/3656724207","cluster_addr":"172.21.15.121:6819/3656724207","heartbeat_back_addr":"172.21.15.121:6827/3656724207","heartbeat_front_addr":"172.21.15.121:6822/3656724207","state":["exists","up"]},{"osd":3,"uuid":"dadb01bc-5491-4ffc-8824-d48a057240b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6823","nonce":2232342832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6826","nonce":2232342832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6831","nonce":2232342832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6829","nonce":2232342832}]},"public_addr":"172.21.15.121:6823/2232342832","cluster_addr":"172.21.15.121:6826/2232342832","heartbeat_back_addr":"172.21.15.121:6831/2232342832","heartbeat_front_addr":"172.21.15.121:6829/2232342832","state":["exists","up"]},{"osd":4,"uuid":"b3f3b4a1-b375-4526-b69e-344ab24c2da1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6801","nonce":99353111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6803","nonce":99353111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6807","nonce":99353111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6805","nonce":99353111}]},"public_addr":"172.21.15.123:6801/99353111","cluster_addr":"172.21.15.123:6803/99353111","heartbeat_back_addr":"172.21.15.123:6807/99353111","heartbeat_front_addr":"172.21.15.123:6805/99353111","state":["exists","up"]},{"osd":5,"uuid":"2ee24a08-1624-4961-9642-d7b510282484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6818","nonce":2504374688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6822","nonce":2504374688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6830","nonce":2504374688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6826","nonce":2504374688}]},"public_addr":"172.21.15.123:6818/2504374688","cluster_addr":"172.21.15.123:6822/2504374688","heartbeat_back_addr":"172.21.15.123:6830/2504374688","heartbeat_front_addr":"172.21.15.123:6826/2504374688","state":["exists","up"]},{"osd":6,"uuid":"186118cc-6128-49de-acca-55260945c1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6819","nonce":4139299599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6823","nonce":4139299599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6829","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6831","nonce":4139299599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6827","nonce":4139299599}]},"public_addr":"172.21.15.123:6819/4139299599","cluster_addr":"172.21.15.123:6823/4139299599","heartbeat_back_addr":"172.21.15.123:6831/4139299599","heartbeat_front_addr":"172.21.15.123:6827/4139299599","state":["exists","up"]},{"osd":7,"uuid":"a00aac6e-4bd9-415a-a93d-ae85210865ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6809","nonce":4043981069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6811","nonce":4043981069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6815","nonce":4043981069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6813","nonce":4043981069}]},"public_addr":"172.21.15.123:6809/4043981069","cluster_addr":"172.21.15.123:6811/4043981069","heartbeat_back_addr":"172.21.15.123:6815/4043981069","heartbeat_front_addr":"172.21.15.123:6813/4043981069","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.904540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.958585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.616367+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.885244+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.103127+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.167248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.884348+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.862356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.23","mappings":[{"from":3,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:22:13.987 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.986+0000 7fa343000640 1 -- 172.21.15.121:0/366358052 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fa31c06c920 msgr2=0x7fa31c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:13.988 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.986+0000 7fa343000640 1 --2- 172.21.15.121:0/366358052 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fa31c06c920 0x7fa31c06ede0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa32c007fd0 tx=0x7fa32c009040 comp rx=0 tx=0).stop 2024-04-20T10:22:13.988 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.986+0000 7fa343000640 1 -- 172.21.15.121:0/366358052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa33c15b3b0 msgr2=0x7fa33c15c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:13.988 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.986+0000 7fa343000640 1 --2- 172.21.15.121:0/366358052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa33c15b3b0 0x7fa33c15c5f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa32800d800 tx=0x7fa32800dcc0 comp rx=0 tx=0).stop 2024-04-20T10:22:13.988 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.987+0000 7fa343000640 1 -- 172.21.15.121:0/366358052 shutdown_connections 2024-04-20T10:22:13.988 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.987+0000 7fa343000640 1 --2- 172.21.15.121:0/366358052 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fa31c06c920 0x7fa31c06ede0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:13.988 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.987+0000 7fa343000640 1 --2- 172.21.15.121:0/366358052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa33c15cb30 0x7fa33c153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:13.988 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.987+0000 7fa343000640 1 --2- 172.21.15.121:0/366358052 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa33c15b3b0 0x7fa33c15c5f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:13.988 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.987+0000 7fa343000640 1 --2- 172.21.15.121:0/366358052 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa33c06dcb0 0x7fa33c1501d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:13.988 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.987+0000 7fa343000640 1 -- 172.21.15.121:0/366358052 >> 172.21.15.121:0/366358052 conn(0x7fa33c0b3350 msgr2=0x7fa33c072d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:13.988 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.987+0000 7fa343000640 1 -- 172.21.15.121:0/366358052 shutdown_connections 2024-04-20T10:22:13.988 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:13.987+0000 7fa343000640 1 -- 172.21.15.121:0/366358052 wait complete. 2024-04-20T10:22:14.007 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:20:51.141365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:21:00.436937+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:21:01.026957+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T10:21:04.862693+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T10:22:14.008 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:22:14.204 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.203+0000 7f5496c00640 1 Processor -- start 2024-04-20T10:22:14.204 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.203+0000 7f5496c00640 1 -- start start 2024-04-20T10:22:14.205 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.205+0000 7f5496c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5490145830 0x7f54900b5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:14.205 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.205+0000 7f5496c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f54900b5340 0x7f54900cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:14.205 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.205+0000 7f5496c00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5490145c10 con 0x7f54900b5f90 2024-04-20T10:22:14.206 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.205+0000 7f5496c00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f549015caf0 con 0x7f54900b5340 2024-04-20T10:22:14.206 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.205+0000 7f5496c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f54900cbb20 con 0x7f5490145830 2024-04-20T10:22:14.207 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.205+0000 7f5494c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f54900b5340 0x7f54900cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:14.207 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.205+0000 7f5494c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f54900b5340 0x7f54900cc8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42608/0 (socket says 172.21.15.121:42608) 2024-04-20T10:22:14.207 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.205+0000 7f5494c00640 1 -- 172.21.15.121:0/3828676439 learned_addr learned my addr 172.21.15.121:0/3828676439 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:14.207 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.205+0000 7f5487400640 1 --1- 172.21.15.121:0/3828676439 >> v1:172.21.15.121:6789/0 conn(0x7f54900b5f90 0x7f54900b4c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:34930/0 (socket says 172.21.15.121:34930) 2024-04-20T10:22:14.207 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.206+0000 7f548fe00640 1 --2- 172.21.15.121:0/3828676439 >> v2:172.21.15.123:3300/0 conn(0x7f5490145830 0x7f54900b5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:14.207 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.206+0000 7f5494c00640 1 -- 172.21.15.121:0/3828676439 >> v2:172.21.15.123:3300/0 conn(0x7f5490145830 msgr2=0x7f54900b5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:14.207 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.206+0000 7f5494c00640 1 --2- 172.21.15.121:0/3828676439 >> v2:172.21.15.123:3300/0 conn(0x7f5490145830 0x7f54900b5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:14.207 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.206+0000 7f5494c00640 1 -- 172.21.15.121:0/3828676439 >> v1:172.21.15.121:6789/0 conn(0x7f54900b5f90 legacy=0x7f54900b4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:14.207 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.206+0000 7f5494c00640 1 -- 172.21.15.121:0/3828676439 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54900cbe00 con 0x7f54900b5340 2024-04-20T10:22:14.208 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.208+0000 7f5494c00640 1 --2- 172.21.15.121:0/3828676439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54900b5340 0x7f54900cc8e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5490075340 tx=0x7f548802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c33fe3dc223a3b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:14.208 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.208+0000 7f548f400640 1 -- 172.21.15.121:0/3828676439 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5488034070 con 0x7f54900b5340 2024-04-20T10:22:14.209 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.208+0000 7f548f400640 1 -- 172.21.15.121:0/3828676439 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5488030860 con 0x7f54900b5340 2024-04-20T10:22:14.209 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.208+0000 7f548f400640 1 -- 172.21.15.121:0/3828676439 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5488030c30 con 0x7f54900b5340 2024-04-20T10:22:14.209 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.209+0000 7f5496c00640 1 -- 172.21.15.121:0/3828676439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54900b5340 msgr2=0x7f54900cc8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:14.209 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.209+0000 7f5496c00640 1 --2- 172.21.15.121:0/3828676439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54900b5340 0x7f54900cc8e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5490075340 tx=0x7f548802b9a0 comp rx=0 tx=0).stop 2024-04-20T10:22:14.210 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.209+0000 7f5496c00640 1 -- 172.21.15.121:0/3828676439 shutdown_connections 2024-04-20T10:22:14.210 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.209+0000 7f5496c00640 1 --2- 172.21.15.121:0/3828676439 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f54900b5340 0x7f54900cc8e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:14.210 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.209+0000 7f5496c00640 1 --2- 172.21.15.121:0/3828676439 >> v2:172.21.15.123:3300/0 conn(0x7f5490145830 0x7f54900b5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:14.210 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.209+0000 7f5496c00640 1 -- 172.21.15.121:0/3828676439 >> 172.21.15.121:0/3828676439 conn(0x7f5490070390 msgr2=0x7f54900707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:14.210 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.209+0000 7f5496c00640 1 -- 172.21.15.121:0/3828676439 shutdown_connections 2024-04-20T10:22:14.210 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.209+0000 7f5496c00640 1 -- 172.21.15.121:0/3828676439 wait complete. 2024-04-20T10:22:14.211 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.211+0000 7f5496c00640 1 Processor -- start 2024-04-20T10:22:14.211 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.211+0000 7f5496c00640 1 -- start start 2024-04-20T10:22:14.213 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.212+0000 7f5496c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54900b5340 0x7f549006d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:14.213 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.212+0000 7f5496c00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f54900b5f90 0x7f549006e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:14.213 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.212+0000 7f5496c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5490145830 0x7f549006e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:14.213 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.212+0000 7f5496c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f54900cce20 con 0x7f5490145830 2024-04-20T10:22:14.213 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.212+0000 7f5496c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f54900cc3c0 con 0x7f54900b5340 2024-04-20T10:22:14.213 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.212+0000 7f5496c00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f54900ccfa0 con 0x7f54900b5f90 2024-04-20T10:22:14.213 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.212+0000 7f5494c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5490145830 0x7f549006e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:14.213 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.212+0000 7f5494c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5490145830 0x7f549006e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42624/0 (socket says 172.21.15.121:42624) 2024-04-20T10:22:14.213 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.212+0000 7f5494c00640 1 -- 172.21.15.121:0/437793789 learned_addr learned my addr 172.21.15.121:0/437793789 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:14.214 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.213+0000 7f5494c00640 1 -- 172.21.15.121:0/437793789 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f54900b5f90 msgr2=0x7f549006e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:14.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.213+0000 7f5487400640 1 --2- 172.21.15.121:0/437793789 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f54900b5f90 0x7f549006e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:14.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.213+0000 7f548fe00640 1 --2- 172.21.15.121:0/437793789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54900b5340 0x7f549006d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:14.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.213+0000 7f5494c00640 1 --2- 172.21.15.121:0/437793789 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f54900b5f90 0x7f549006e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:14.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.213+0000 7f5494c00640 1 -- 172.21.15.121:0/437793789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54900b5340 msgr2=0x7f549006d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:14.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.213+0000 7f5494c00640 1 --2- 172.21.15.121:0/437793789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54900b5340 0x7f549006d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:14.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.213+0000 7f5494c00640 1 -- 172.21.15.121:0/437793789 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f549015cea0 con 0x7f5490145830 2024-04-20T10:22:14.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.213+0000 7f5494c00640 1 --2- 172.21.15.121:0/437793789 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5490145830 0x7f549006e760 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f548802b9e0 tx=0x7f5488004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:14.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.213+0000 7f548cc00640 1 -- 172.21.15.121:0/437793789 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5488048070 con 0x7f5490145830 2024-04-20T10:22:14.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.213+0000 7f548cc00640 1 -- 172.21.15.121:0/437793789 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5488030660 con 0x7f5490145830 2024-04-20T10:22:14.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.213+0000 7f5496c00640 1 -- 172.21.15.121:0/437793789 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f549015d180 con 0x7f5490145830 2024-04-20T10:22:14.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.213+0000 7f548cc00640 1 -- 172.21.15.121:0/437793789 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5488034040 con 0x7f5490145830 2024-04-20T10:22:14.215 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.214+0000 7f5496c00640 1 -- 172.21.15.121:0/437793789 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f549015d740 con 0x7f5490145830 2024-04-20T10:22:14.216 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.215+0000 7f5496c00640 1 -- 172.21.15.121:0/437793789 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5490069090 con 0x7f5490145830 2024-04-20T10:22:14.216 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.216+0000 7f548cc00640 1 -- 172.21.15.121:0/437793789 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f548804b070 con 0x7f5490145830 2024-04-20T10:22:14.217 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.217+0000 7f548cc00640 1 --2- 172.21.15.121:0/437793789 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f546c06c8f0 0x7f546c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:14.218 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.217+0000 7f548cc00640 1 -- 172.21.15.121:0/437793789 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f54880ae080 con 0x7f5490145830 2024-04-20T10:22:14.218 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.218+0000 7f548fe00640 1 --2- 172.21.15.121:0/437793789 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f546c06c8f0 0x7f546c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:14.218 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.218+0000 7f548fe00640 1 --2- 172.21.15.121:0/437793789 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f546c06c8f0 0x7f546c06edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5480009e90 tx=0x7f548000b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:14.222 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.222+0000 7f548cc00640 1 -- 172.21.15.121:0/437793789 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f548807c0d0 con 0x7f5490145830 2024-04-20T10:22:14.419 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.419+0000 7f5496c00640 1 -- 172.21.15.121:0/437793789 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5490074dd0 con 0x7f5490145830 2024-04-20T10:22:14.420 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.420+0000 7f548cc00640 1 -- 172.21.15.121:0/437793789 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f548807b9d0 con 0x7f5490145830 2024-04-20T10:22:14.420 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2024-04-20T10:22:14.422 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.422+0000 7f5496c00640 1 -- 172.21.15.121:0/437793789 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f546c06c8f0 msgr2=0x7f546c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:14.422 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.422+0000 7f5496c00640 1 --2- 172.21.15.121:0/437793789 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f546c06c8f0 0x7f546c06edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5480009e90 tx=0x7f548000b040 comp rx=0 tx=0).stop 2024-04-20T10:22:14.423 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.422+0000 7f5496c00640 1 -- 172.21.15.121:0/437793789 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5490145830 msgr2=0x7f549006e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:14.423 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.422+0000 7f5496c00640 1 --2- 172.21.15.121:0/437793789 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5490145830 0x7f549006e760 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f548802b9e0 tx=0x7f5488004730 comp rx=0 tx=0).stop 2024-04-20T10:22:14.423 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.422+0000 7f5496c00640 1 -- 172.21.15.121:0/437793789 shutdown_connections 2024-04-20T10:22:14.423 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.422+0000 7f5496c00640 1 --2- 172.21.15.121:0/437793789 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5490145830 0x7f549006e760 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:14.423 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.422+0000 7f5496c00640 1 --2- 172.21.15.121:0/437793789 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f54900b5f90 0x7f549006e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:14.423 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.422+0000 7f5496c00640 1 --2- 172.21.15.121:0/437793789 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f546c06c8f0 0x7f546c06edb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:14.423 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.422+0000 7f5496c00640 1 --2- 172.21.15.121:0/437793789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54900b5340 0x7f549006d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:14.424 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.422+0000 7f5496c00640 1 -- 172.21.15.121:0/437793789 >> 172.21.15.121:0/437793789 conn(0x7f5490070390 msgr2=0x7f5490070770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:14.424 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.423+0000 7f5496c00640 1 -- 172.21.15.121:0/437793789 shutdown_connections 2024-04-20T10:22:14.424 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.423+0000 7f5496c00640 1 -- 172.21.15.121:0/437793789 wait complete. 2024-04-20T10:22:14.442 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:22:14.650 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.649+0000 7ff644e00640 1 Processor -- start 2024-04-20T10:22:14.650 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.649+0000 7ff644e00640 1 -- start start 2024-04-20T10:22:14.651 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.650+0000 7ff644e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff640158d70 0x7ff6400716e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:14.651 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.651+0000 7ff644e00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6400d2110 con 0x7ff6400d1ce0 2024-04-20T10:22:14.651 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.651+0000 7ff644e00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6400cff70 con 0x7ff640071c20 2024-04-20T10:22:14.652 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.651+0000 7ff644e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7ff640072080 con 0x7ff640158d70 2024-04-20T10:22:14.652 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.651+0000 7ff63de00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff640158d70 0x7ff6400716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:14.652 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.651+0000 7ff63e800640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7ff6400d1ce0 0x7ff6400b1e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:34934/0 (socket says 172.21.15.121:34934) 2024-04-20T10:22:14.652 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.651+0000 7ff63e800640 1 -- 172.21.15.121:0/1147134953 learned_addr learned my addr 172.21.15.121:0/1147134953 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:14.653 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.652+0000 7ff63de00640 1 -- 172.21.15.121:0/1147134953 >> v1:172.21.15.123:6789/0 conn(0x7ff640071c20 legacy=0x7ff6400d15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:14.653 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.652+0000 7ff63de00640 1 -- 172.21.15.121:0/1147134953 >> v1:172.21.15.121:6789/0 conn(0x7ff6400d1ce0 legacy=0x7ff6400b1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:14.653 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.652+0000 7ff63de00640 1 -- 172.21.15.121:0/1147134953 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff64015f010 con 0x7ff640158d70 2024-04-20T10:22:14.653 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.653+0000 7ff63de00640 1 --2- 172.21.15.121:0/1147134953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff640158d70 0x7ff6400716e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff630009d90 tx=0x7ff63002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca560e76867ec3c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:14.653 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.653+0000 7ff63d400640 1 -- 172.21.15.121:0/1147134953 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff630039070 con 0x7ff640158d70 2024-04-20T10:22:14.654 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.653+0000 7ff63d400640 1 -- 172.21.15.121:0/1147134953 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff63002cc60 con 0x7ff640158d70 2024-04-20T10:22:14.654 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.653+0000 7ff63d400640 1 -- 172.21.15.121:0/1147134953 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff630034660 con 0x7ff640158d70 2024-04-20T10:22:14.655 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.654+0000 7ff644e00640 1 -- 172.21.15.121:0/1147134953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff640158d70 msgr2=0x7ff6400716e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:14.655 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.654+0000 7ff644e00640 1 --2- 172.21.15.121:0/1147134953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff640158d70 0x7ff6400716e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff630009d90 tx=0x7ff63002beb0 comp rx=0 tx=0).stop 2024-04-20T10:22:14.655 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.654+0000 7ff644e00640 1 -- 172.21.15.121:0/1147134953 shutdown_connections 2024-04-20T10:22:14.655 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.654+0000 7ff644e00640 1 --2- 172.21.15.121:0/1147134953 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff640158d70 0x7ff6400716e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:14.655 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.654+0000 7ff644e00640 1 -- 172.21.15.121:0/1147134953 >> 172.21.15.121:0/1147134953 conn(0x7ff6400d23a0 msgr2=0x7ff6400d27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:14.655 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.654+0000 7ff644e00640 1 -- 172.21.15.121:0/1147134953 shutdown_connections 2024-04-20T10:22:14.655 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.654+0000 7ff644e00640 1 -- 172.21.15.121:0/1147134953 wait complete. 2024-04-20T10:22:14.656 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.656+0000 7ff644e00640 1 Processor -- start 2024-04-20T10:22:14.656 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.656+0000 7ff644e00640 1 -- start start 2024-04-20T10:22:14.658 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.657+0000 7ff644e00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff640071c20 0x7ff6401552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:14.658 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.657+0000 7ff644e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6400d1ce0 0x7ff6400b63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:14.658 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.657+0000 7ff644e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff640158d70 0x7ff6400b6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:14.658 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.657+0000 7ff644e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff64015f940 con 0x7ff640158d70 2024-04-20T10:22:14.658 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.657+0000 7ff644e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff640159150 con 0x7ff6400d1ce0 2024-04-20T10:22:14.658 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.657+0000 7ff644e00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ff64015f5d0 con 0x7ff640071c20 2024-04-20T10:22:14.658 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.657+0000 7ff63e800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff640158d70 0x7ff6400b6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:14.658 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.657+0000 7ff63e800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff640158d70 0x7ff6400b6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42648/0 (socket says 172.21.15.121:42648) 2024-04-20T10:22:14.658 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.657+0000 7ff63e800640 1 -- 172.21.15.121:0/3988392574 learned_addr learned my addr 172.21.15.121:0/3988392574 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:14.660 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.658+0000 7ff63e800640 1 -- 172.21.15.121:0/3988392574 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff640071c20 msgr2=0x7ff6401552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:14.660 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.658+0000 7ff63e800640 1 --2- 172.21.15.121:0/3988392574 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff640071c20 0x7ff6401552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:14.660 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.658+0000 7ff63e800640 1 -- 172.21.15.121:0/3988392574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6400d1ce0 msgr2=0x7ff6400b63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:14.660 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.658+0000 7ff635400640 1 --2- 172.21.15.121:0/3988392574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6400d1ce0 0x7ff6400b63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:14.660 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.658+0000 7ff63e800640 1 --2- 172.21.15.121:0/3988392574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6400d1ce0 0x7ff6400b63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:14.660 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.658+0000 7ff63e800640 1 -- 172.21.15.121:0/3988392574 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6400b7040 con 0x7ff640158d70 2024-04-20T10:22:14.660 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.658+0000 7ff63e800640 1 --2- 172.21.15.121:0/3988392574 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff640158d70 0x7ff6400b6900 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff63800c420 tx=0x7ff63800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:14.660 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.658+0000 7ff636a00640 1 -- 172.21.15.121:0/3988392574 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff638017070 con 0x7ff640158d70 2024-04-20T10:22:14.660 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.658+0000 7ff636a00640 1 -- 172.21.15.121:0/3988392574 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff638005ce0 con 0x7ff640158d70 2024-04-20T10:22:14.660 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.658+0000 7ff636a00640 1 -- 172.21.15.121:0/3988392574 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6380072d0 con 0x7ff640158d70 2024-04-20T10:22:14.660 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.658+0000 7ff644e00640 1 -- 172.21.15.121:0/3988392574 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6400b7320 con 0x7ff640158d70 2024-04-20T10:22:14.660 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.659+0000 7ff644e00640 1 -- 172.21.15.121:0/3988392574 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff640155810 con 0x7ff640158d70 2024-04-20T10:22:14.661 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.660+0000 7ff644e00640 1 -- 172.21.15.121:0/3988392574 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff640068980 con 0x7ff640158d70 2024-04-20T10:22:14.661 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.661+0000 7ff636a00640 1 -- 172.21.15.121:0/3988392574 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff63801a070 con 0x7ff640158d70 2024-04-20T10:22:14.662 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.661+0000 7ff636a00640 1 --2- 172.21.15.121:0/3988392574 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff61c06c8f0 0x7ff61c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:14.662 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.662+0000 7ff636a00640 1 -- 172.21.15.121:0/3988392574 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ff63808b1f0 con 0x7ff640158d70 2024-04-20T10:22:14.662 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.662+0000 7ff63de00640 1 --2- 172.21.15.121:0/3988392574 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff61c06c8f0 0x7ff61c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:14.663 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.663+0000 7ff63de00640 1 --2- 172.21.15.121:0/3988392574 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff61c06c8f0 0x7ff61c06edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff630036000 tx=0x7ff63002e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:14.666 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.666+0000 7ff636a00640 1 -- 172.21.15.121:0/3988392574 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff6400d2110 con 0x7ff640158d70 2024-04-20T10:22:14.862 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.862+0000 7ff644e00640 1 -- 172.21.15.121:0/3988392574 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff640158520 con 0x7ff640158d70 2024-04-20T10:22:14.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.863+0000 7ff636a00640 1 -- 172.21.15.121:0/3988392574 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7ff638058bb0 con 0x7ff640158d70 2024-04-20T10:22:14.863 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 32 2024-04-20T10:22:14.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.865+0000 7ff644e00640 1 -- 172.21.15.121:0/3988392574 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff61c06c8f0 msgr2=0x7ff61c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:14.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.865+0000 7ff644e00640 1 --2- 172.21.15.121:0/3988392574 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff61c06c8f0 0x7ff61c06edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff630036000 tx=0x7ff63002e000 comp rx=0 tx=0).stop 2024-04-20T10:22:14.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.865+0000 7ff644e00640 1 -- 172.21.15.121:0/3988392574 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff640158d70 msgr2=0x7ff6400b6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:14.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.865+0000 7ff644e00640 1 --2- 172.21.15.121:0/3988392574 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff640158d70 0x7ff6400b6900 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff63800c420 tx=0x7ff63800c8e0 comp rx=0 tx=0).stop 2024-04-20T10:22:14.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.865+0000 7ff644e00640 1 -- 172.21.15.121:0/3988392574 shutdown_connections 2024-04-20T10:22:14.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.865+0000 7ff644e00640 1 --2- 172.21.15.121:0/3988392574 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff640158d70 0x7ff6400b6900 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:14.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.865+0000 7ff644e00640 1 --2- 172.21.15.121:0/3988392574 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff61c06c8f0 0x7ff61c06edb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:14.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.865+0000 7ff644e00640 1 --2- 172.21.15.121:0/3988392574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6400d1ce0 0x7ff6400b63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:14.867 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.865+0000 7ff644e00640 1 --2- 172.21.15.121:0/3988392574 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff640071c20 0x7ff6401552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:14.867 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.865+0000 7ff644e00640 1 -- 172.21.15.121:0/3988392574 >> 172.21.15.121:0/3988392574 conn(0x7ff6400d23a0 msgr2=0x7ff6400d0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:14.867 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.866+0000 7ff644e00640 1 -- 172.21.15.121:0/3988392574 shutdown_connections 2024-04-20T10:22:14.867 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:14.866+0000 7ff644e00640 1 -- 172.21.15.121:0/3988392574 wait complete. 2024-04-20T10:22:14.885 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:22:15.083 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.082+0000 7fbe83000640 1 Processor -- start 2024-04-20T10:22:15.083 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.082+0000 7fbe83000640 1 -- start start 2024-04-20T10:22:15.084 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.084+0000 7fbe83000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbe7c0d0a70 0x7fbe7c06e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:15.085 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.084+0000 7fbe83000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe7c06f060 con 0x7fbe7c15e8a0 2024-04-20T10:22:15.085 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.084+0000 7fbe83000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe7c06e7c0 con 0x7fbe7c06dbb0 2024-04-20T10:22:15.085 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.084+0000 7fbe83000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fbe7c06eaa0 con 0x7fbe7c0d0a70 2024-04-20T10:22:15.085 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.084+0000 7fbe7be00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fbe7c15e8a0 0x7fbe7c0d0360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:34944/0 (socket says 172.21.15.121:34944) 2024-04-20T10:22:15.085 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.084+0000 7fbe7be00640 1 -- 172.21.15.121:0/2195424951 learned_addr learned my addr 172.21.15.121:0/2195424951 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:15.086 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.085+0000 7fbe7b400640 1 --2- 172.21.15.121:0/2195424951 >> v2:172.21.15.123:3300/0 conn(0x7fbe7c0d0a70 0x7fbe7c06e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:15.086 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.085+0000 7fbe7aa00640 1 -- 172.21.15.121:0/2195424951 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2357317873 0 0) 0x7fbe7c06f060 con 0x7fbe7c15e8a0 2024-04-20T10:22:15.086 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.085+0000 7fbe7aa00640 1 -- 172.21.15.121:0/2195424951 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe60003520 con 0x7fbe7c15e8a0 2024-04-20T10:22:15.086 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.085+0000 7fbe7b400640 1 -- 172.21.15.121:0/2195424951 >> v1:172.21.15.123:6789/0 conn(0x7fbe7c06dbb0 legacy=0x7fbe7c0ad680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:15.086 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.085+0000 7fbe7aa00640 1 -- 172.21.15.121:0/2195424951 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2253595418 0 0) 0x7fbe60003520 con 0x7fbe7c15e8a0 2024-04-20T10:22:15.086 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.085+0000 7fbe7b400640 1 -- 172.21.15.121:0/2195424951 >> v1:172.21.15.121:6789/0 conn(0x7fbe7c15e8a0 legacy=0x7fbe7c0d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:15.086 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.085+0000 7fbe7b400640 1 -- 172.21.15.121:0/2195424951 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe7c06ed80 con 0x7fbe7c0d0a70 2024-04-20T10:22:15.087 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.087+0000 7fbe7b400640 1 --2- 172.21.15.121:0/2195424951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe7c0d0a70 0x7fbe7c06e1e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbe680097a0 tx=0x7fbe6802b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bc44c31b9bd63a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:15.088 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.087+0000 7fbe7aa00640 1 -- 172.21.15.121:0/2195424951 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe68038070 con 0x7fbe7c0d0a70 2024-04-20T10:22:15.088 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.087+0000 7fbe7aa00640 1 -- 172.21.15.121:0/2195424951 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe68031500 con 0x7fbe7c0d0a70 2024-04-20T10:22:15.088 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.087+0000 7fbe7aa00640 1 -- 172.21.15.121:0/2195424951 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c001670 con 0x7fbe7c0d0a70 2024-04-20T10:22:15.089 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.088+0000 7fbe83000640 1 -- 172.21.15.121:0/2195424951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe7c0d0a70 msgr2=0x7fbe7c06e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:15.089 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.088+0000 7fbe83000640 1 --2- 172.21.15.121:0/2195424951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe7c0d0a70 0x7fbe7c06e1e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbe680097a0 tx=0x7fbe6802b820 comp rx=0 tx=0).stop 2024-04-20T10:22:15.089 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.088+0000 7fbe83000640 1 -- 172.21.15.121:0/2195424951 shutdown_connections 2024-04-20T10:22:15.089 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.088+0000 7fbe83000640 1 --2- 172.21.15.121:0/2195424951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe7c0d0a70 0x7fbe7c06e1e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:15.089 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.088+0000 7fbe83000640 1 -- 172.21.15.121:0/2195424951 >> 172.21.15.121:0/2195424951 conn(0x7fbe7c0b3350 msgr2=0x7fbe7c0b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:15.089 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.088+0000 7fbe83000640 1 -- 172.21.15.121:0/2195424951 shutdown_connections 2024-04-20T10:22:15.089 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.088+0000 7fbe83000640 1 -- 172.21.15.121:0/2195424951 wait complete. 2024-04-20T10:22:15.090 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.090+0000 7fbe83000640 1 Processor -- start 2024-04-20T10:22:15.090 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.090+0000 7fbe83000640 1 -- start start 2024-04-20T10:22:15.091 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.091+0000 7fbe83000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe7c06dbb0 0x7fbe7c14a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:15.091 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.091+0000 7fbe83000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe7c0d0a70 0x7fbe7c159a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:15.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.091+0000 7fbe83000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe7c15e8a0 0x7fbe7c159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:15.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.091+0000 7fbe83000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7c06fef0 con 0x7fbe7c15e8a0 2024-04-20T10:22:15.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.091+0000 7fbe83000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7c06fd70 con 0x7fbe7c06dbb0 2024-04-20T10:22:15.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.091+0000 7fbe83000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fbe7c0add90 con 0x7fbe7c0d0a70 2024-04-20T10:22:15.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.091+0000 7fbe81000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe7c15e8a0 0x7fbe7c159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:15.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.091+0000 7fbe81000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe7c15e8a0 0x7fbe7c159fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42658/0 (socket says 172.21.15.121:42658) 2024-04-20T10:22:15.092 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.091+0000 7fbe81000640 1 -- 172.21.15.121:0/1055950498 learned_addr learned my addr 172.21.15.121:0/1055950498 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:15.093 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.091+0000 7fbe81000640 1 -- 172.21.15.121:0/1055950498 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe7c0d0a70 msgr2=0x7fbe7c159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:15.093 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.091+0000 7fbe81000640 1 --2- 172.21.15.121:0/1055950498 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe7c0d0a70 0x7fbe7c159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:15.093 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.091+0000 7fbe81000640 1 -- 172.21.15.121:0/1055950498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe7c06dbb0 msgr2=0x7fbe7c14a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:15.093 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.091+0000 7fbe7be00640 1 --2- 172.21.15.121:0/1055950498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe7c06dbb0 0x7fbe7c14a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:15.093 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.091+0000 7fbe81000640 1 --2- 172.21.15.121:0/1055950498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe7c06dbb0 0x7fbe7c14a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:15.093 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.091+0000 7fbe81000640 1 -- 172.21.15.121:0/1055950498 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe7c0b0410 con 0x7fbe7c15e8a0 2024-04-20T10:22:15.094 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.092+0000 7fbe81000640 1 --2- 172.21.15.121:0/1055950498 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe7c15e8a0 0x7fbe7c159fd0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbe7400c730 tx=0x7fbe7400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:15.094 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.092+0000 7fbe73e00640 1 -- 172.21.15.121:0/1055950498 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe74003b70 con 0x7fbe7c15e8a0 2024-04-20T10:22:15.094 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.092+0000 7fbe73e00640 1 -- 172.21.15.121:0/1055950498 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe74003d30 con 0x7fbe7c15e8a0 2024-04-20T10:22:15.094 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.092+0000 7fbe83000640 1 -- 172.21.15.121:0/1055950498 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe7c0b06f0 con 0x7fbe7c15e8a0 2024-04-20T10:22:15.094 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.092+0000 7fbe73e00640 1 -- 172.21.15.121:0/1055950498 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe74005310 con 0x7fbe7c15e8a0 2024-04-20T10:22:15.094 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.092+0000 7fbe83000640 1 -- 172.21.15.121:0/1055950498 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe7c0b0cb0 con 0x7fbe7c15e8a0 2024-04-20T10:22:15.094 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.093+0000 7fbe83000640 1 -- 172.21.15.121:0/1055950498 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe7c068980 con 0x7fbe7c15e8a0 2024-04-20T10:22:15.094 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.094+0000 7fbe73e00640 1 -- 172.21.15.121:0/1055950498 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbe740056c0 con 0x7fbe7c15e8a0 2024-04-20T10:22:15.096 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.096+0000 7fbe73e00640 1 --2- 172.21.15.121:0/1055950498 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fbe5c06c8f0 0x7fbe5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:15.096 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.096+0000 7fbe73e00640 1 -- 172.21.15.121:0/1055950498 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fbe7408bf00 con 0x7fbe7c15e8a0 2024-04-20T10:22:15.096 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.096+0000 7fbe7be00640 1 --2- 172.21.15.121:0/1055950498 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fbe5c06c8f0 0x7fbe5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:15.097 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.096+0000 7fbe7be00640 1 --2- 172.21.15.121:0/1055950498 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fbe5c06c8f0 0x7fbe5c06edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c0017e0 tx=0x7fbe6c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:15.100 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.100+0000 7fbe73e00640 1 -- 172.21.15.121:0/1055950498 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbe74059fc0 con 0x7fbe7c15e8a0 2024-04-20T10:22:15.302 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.302+0000 7fbe83000640 1 -- 172.21.15.121:0/1055950498 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbe7c06d160 con 0x7fbe7c15e8a0 2024-04-20T10:22:15.303 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.303+0000 7fbe73e00640 1 -- 172.21.15.121:0/1055950498 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fbe740598c0 con 0x7fbe7c15e8a0 2024-04-20T10:22:15.303 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2024-04-20T10:22:15.306 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.305+0000 7fbe83000640 1 -- 172.21.15.121:0/1055950498 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fbe5c06c8f0 msgr2=0x7fbe5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:15.306 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.305+0000 7fbe83000640 1 --2- 172.21.15.121:0/1055950498 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fbe5c06c8f0 0x7fbe5c06edb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c0017e0 tx=0x7fbe6c009040 comp rx=0 tx=0).stop 2024-04-20T10:22:15.306 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.305+0000 7fbe83000640 1 -- 172.21.15.121:0/1055950498 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe7c15e8a0 msgr2=0x7fbe7c159fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:15.306 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.305+0000 7fbe83000640 1 --2- 172.21.15.121:0/1055950498 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe7c15e8a0 0x7fbe7c159fd0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbe7400c730 tx=0x7fbe7400cbf0 comp rx=0 tx=0).stop 2024-04-20T10:22:15.307 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.306+0000 7fbe83000640 1 -- 172.21.15.121:0/1055950498 shutdown_connections 2024-04-20T10:22:15.307 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.306+0000 7fbe83000640 1 --2- 172.21.15.121:0/1055950498 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbe7c15e8a0 0x7fbe7c159fd0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:15.307 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.306+0000 7fbe83000640 1 --2- 172.21.15.121:0/1055950498 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbe7c0d0a70 0x7fbe7c159a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:15.307 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.306+0000 7fbe83000640 1 --2- 172.21.15.121:0/1055950498 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fbe5c06c8f0 0x7fbe5c06edb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:15.307 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.306+0000 7fbe83000640 1 --2- 172.21.15.121:0/1055950498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe7c06dbb0 0x7fbe7c14a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:15.307 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.306+0000 7fbe83000640 1 -- 172.21.15.121:0/1055950498 >> 172.21.15.121:0/1055950498 conn(0x7fbe7c0b3350 msgr2=0x7fbe7c071b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:15.307 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.306+0000 7fbe83000640 1 -- 172.21.15.121:0/1055950498 shutdown_connections 2024-04-20T10:22:15.307 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.306+0000 7fbe83000640 1 -- 172.21.15.121:0/1055950498 wait complete. 2024-04-20T10:22:15.324 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T10:22:15.517 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.516+0000 7f5179600640 1 Processor -- start 2024-04-20T10:22:15.517 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.517+0000 7f5179600640 1 -- start start 2024-04-20T10:22:15.519 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.518+0000 7f5179600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5174162700 0x7f5174162ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:15.519 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.518+0000 7f5179600640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f517415ead0 0x7f51740cf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:15.519 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.518+0000 7f5179600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f517406f2b0 con 0x7f51740b6c20 2024-04-20T10:22:15.519 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.518+0000 7f5179600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f51740b6900 con 0x7f5174162700 2024-04-20T10:22:15.519 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.518+0000 7f5179600640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f51740b6620 con 0x7f517415ead0 2024-04-20T10:22:15.520 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.519+0000 7f5173400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f517415ead0 0x7f51740cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:15.520 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.519+0000 7f5173400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f517415ead0 0x7f51740cf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.121:38506/0 (socket says 172.21.15.121:38506) 2024-04-20T10:22:15.520 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.519+0000 7f5173400640 1 -- 172.21.15.121:0/1383554727 learned_addr learned my addr 172.21.15.121:0/1383554727 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:15.520 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.519+0000 7f5172a00640 1 --2- 172.21.15.121:0/1383554727 >> v2:172.21.15.123:3300/0 conn(0x7f5174162700 0x7f5174162ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:15.520 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.520+0000 7f5173400640 1 -- 172.21.15.121:0/1383554727 >> v2:172.21.15.123:3300/0 conn(0x7f5174162700 msgr2=0x7f5174162ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:15.521 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.520+0000 7f5173400640 1 --2- 172.21.15.121:0/1383554727 >> v2:172.21.15.123:3300/0 conn(0x7f5174162700 0x7f5174162ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:15.521 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.520+0000 7f5173400640 1 -- 172.21.15.121:0/1383554727 >> v1:172.21.15.123:6789/0 conn(0x7f51740b6c20 legacy=0x7f517415e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:15.521 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.520+0000 7f5173400640 1 -- 172.21.15.121:0/1383554727 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51740b6160 con 0x7f517415ead0 2024-04-20T10:22:15.521 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.520+0000 7f5172a00640 1 --2- 172.21.15.121:0/1383554727 >> v2:172.21.15.123:3300/0 conn(0x7f5174162700 0x7f5174162ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:22:15.522 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.521+0000 7f5173400640 1 --2- 172.21.15.121:0/1383554727 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f517415ead0 0x7f51740cf710 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f516c009870 tx=0x7f516c02b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a03a9dd44fd89fe server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:15.522 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.522+0000 7f5171600640 1 -- 172.21.15.121:0/1383554727 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c0317c0 con 0x7f517415ead0 2024-04-20T10:22:15.522 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.522+0000 7f5171600640 1 -- 172.21.15.121:0/1383554727 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f516c037070 con 0x7f517415ead0 2024-04-20T10:22:15.522 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.522+0000 7f5171600640 1 -- 172.21.15.121:0/1383554727 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516c031b80 con 0x7f517415ead0 2024-04-20T10:22:15.523 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.522+0000 7f5179600640 1 -- 172.21.15.121:0/1383554727 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f517415ead0 msgr2=0x7f51740cf710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:15.523 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.522+0000 7f5179600640 1 --2- 172.21.15.121:0/1383554727 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f517415ead0 0x7f51740cf710 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f516c009870 tx=0x7f516c02b860 comp rx=0 tx=0).stop 2024-04-20T10:22:15.523 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.523+0000 7f5179600640 1 -- 172.21.15.121:0/1383554727 shutdown_connections 2024-04-20T10:22:15.523 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.523+0000 7f5179600640 1 --2- 172.21.15.121:0/1383554727 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f517415ead0 0x7f51740cf710 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:15.523 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.523+0000 7f5179600640 1 --2- 172.21.15.121:0/1383554727 >> v2:172.21.15.123:3300/0 conn(0x7f5174162700 0x7f5174162ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:15.523 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.523+0000 7f5179600640 1 -- 172.21.15.121:0/1383554727 >> 172.21.15.121:0/1383554727 conn(0x7f517409d910 msgr2=0x7f51740b9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:15.523 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.523+0000 7f5179600640 1 -- 172.21.15.121:0/1383554727 shutdown_connections 2024-04-20T10:22:15.524 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.523+0000 7f5179600640 1 -- 172.21.15.121:0/1383554727 wait complete. 2024-04-20T10:22:15.525 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.524+0000 7f5179600640 1 Processor -- start 2024-04-20T10:22:15.525 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.525+0000 7f5179600640 1 -- start start 2024-04-20T10:22:15.526 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.525+0000 7f5179600640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51740b6c20 0x7f51741f3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:15.526 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.525+0000 7f5179600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f517415ead0 0x7f51740b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:15.526 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.525+0000 7f5179600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5174162700 0x7f51740b1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:15.526 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.525+0000 7f5179600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f51740b3690 con 0x7f517415ead0 2024-04-20T10:22:15.526 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.525+0000 7f5179600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f51740b3510 con 0x7f5174162700 2024-04-20T10:22:15.526 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.525+0000 7f5179600640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f51740b3810 con 0x7f51740b6c20 2024-04-20T10:22:15.527 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.526+0000 7f5172000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f517415ead0 0x7f51740b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:15.527 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.526+0000 7f5172000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f517415ead0 0x7f51740b1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42672/0 (socket says 172.21.15.121:42672) 2024-04-20T10:22:15.527 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.526+0000 7f5172000640 1 -- 172.21.15.121:0/2008361949 learned_addr learned my addr 172.21.15.121:0/2008361949 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:15.528 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.526+0000 7f5173400640 1 --2- 172.21.15.121:0/2008361949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5174162700 0x7f51740b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:15.528 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.526+0000 7f5172a00640 1 --2- 172.21.15.121:0/2008361949 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51740b6c20 0x7f51741f3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:15.528 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.526+0000 7f5172000640 1 -- 172.21.15.121:0/2008361949 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51740b6c20 msgr2=0x7f51741f3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:15.528 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.526+0000 7f5172000640 1 --2- 172.21.15.121:0/2008361949 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51740b6c20 0x7f51741f3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:15.528 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.526+0000 7f5172000640 1 -- 172.21.15.121:0/2008361949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5174162700 msgr2=0x7f51740b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:15.528 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.526+0000 7f5172000640 1 --2- 172.21.15.121:0/2008361949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5174162700 0x7f51740b1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:15.528 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.526+0000 7f5172000640 1 -- 172.21.15.121:0/2008361949 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51741f40b0 con 0x7f517415ead0 2024-04-20T10:22:15.528 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.526+0000 7f5173400640 1 --2- 172.21.15.121:0/2008361949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5174162700 0x7f51740b1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:15.528 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.527+0000 7f5172000640 1 --2- 172.21.15.121:0/2008361949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f517415ead0 0x7f51740b1350 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f515c00c7a0 tx=0x7f515c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:15.528 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.527+0000 7f516aa00640 1 -- 172.21.15.121:0/2008361949 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515c003b90 con 0x7f517415ead0 2024-04-20T10:22:15.529 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.527+0000 7f516aa00640 1 -- 172.21.15.121:0/2008361949 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f515c003d50 con 0x7f517415ead0 2024-04-20T10:22:15.529 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.527+0000 7f516aa00640 1 -- 172.21.15.121:0/2008361949 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f515c00aa90 con 0x7f517415ead0 2024-04-20T10:22:15.529 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.527+0000 7f5179600640 1 -- 172.21.15.121:0/2008361949 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51740cc7e0 con 0x7f517415ead0 2024-04-20T10:22:15.529 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.527+0000 7f5179600640 1 -- 172.21.15.121:0/2008361949 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51740ccd80 con 0x7f517415ead0 2024-04-20T10:22:15.529 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.528+0000 7f5179600640 1 -- 172.21.15.121:0/2008361949 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5174068980 con 0x7f517415ead0 2024-04-20T10:22:15.530 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.529+0000 7f516aa00640 1 -- 172.21.15.121:0/2008361949 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f515c00ac30 con 0x7f517415ead0 2024-04-20T10:22:15.531 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.530+0000 7f516aa00640 1 --2- 172.21.15.121:0/2008361949 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f515006c8f0 0x7f515006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:15.531 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.530+0000 7f516aa00640 1 -- 172.21.15.121:0/2008361949 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f515c08bee0 con 0x7f517415ead0 2024-04-20T10:22:15.531 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.531+0000 7f5172a00640 1 --2- 172.21.15.121:0/2008361949 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f515006c8f0 0x7f515006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:15.532 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.531+0000 7f5172a00640 1 --2- 172.21.15.121:0/2008361949 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f515006c8f0 0x7f515006edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5164009950 tx=0x7f5164008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:15.535 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.535+0000 7f516aa00640 1 -- 172.21.15.121:0/2008361949 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f51741f40b0 con 0x7f517415ead0 2024-04-20T10:22:15.736 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.735+0000 7f5179600640 1 -- 172.21.15.121:0/2008361949 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f51740d51b0 con 0x7f517415ead0 2024-04-20T10:22:15.737 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.737+0000 7f516aa00640 1 -- 172.21.15.121:0/2008361949 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f515c059e70 con 0x7f517415ead0 2024-04-20T10:22:15.737 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 32 2024-04-20T10:22:15.740 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.739+0000 7f5179600640 1 -- 172.21.15.121:0/2008361949 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f515006c8f0 msgr2=0x7f515006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:15.740 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.739+0000 7f5179600640 1 --2- 172.21.15.121:0/2008361949 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f515006c8f0 0x7f515006edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5164009950 tx=0x7f5164008040 comp rx=0 tx=0).stop 2024-04-20T10:22:15.740 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.739+0000 7f5179600640 1 -- 172.21.15.121:0/2008361949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f517415ead0 msgr2=0x7f51740b1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:15.740 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.739+0000 7f5179600640 1 --2- 172.21.15.121:0/2008361949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f517415ead0 0x7f51740b1350 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f515c00c7a0 tx=0x7f515c00cc60 comp rx=0 tx=0).stop 2024-04-20T10:22:15.740 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.739+0000 7f5179600640 1 -- 172.21.15.121:0/2008361949 shutdown_connections 2024-04-20T10:22:15.741 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.739+0000 7f5179600640 1 --2- 172.21.15.121:0/2008361949 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f515006c8f0 0x7f515006edb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:15.741 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.739+0000 7f5179600640 1 --2- 172.21.15.121:0/2008361949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5174162700 0x7f51740b1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:15.741 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.739+0000 7f5179600640 1 --2- 172.21.15.121:0/2008361949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f517415ead0 0x7f51740b1350 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:15.741 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.739+0000 7f5179600640 1 --2- 172.21.15.121:0/2008361949 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f51740b6c20 0x7f51741f3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:15.741 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.739+0000 7f5179600640 1 -- 172.21.15.121:0/2008361949 >> 172.21.15.121:0/2008361949 conn(0x7f517409d910 msgr2=0x7f51740b4730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:15.741 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.740+0000 7f5179600640 1 -- 172.21.15.121:0/2008361949 shutdown_connections 2024-04-20T10:22:15.741 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.740+0000 7f5179600640 1 -- 172.21.15.121:0/2008361949 wait complete. 2024-04-20T10:22:15.759 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T10:22:15.937 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.937+0000 7ff456000640 1 Processor -- start 2024-04-20T10:22:15.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.937+0000 7ff456000640 1 -- start start 2024-04-20T10:22:15.939 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.938+0000 7ff456000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff450156c20 0x7ff45006b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:15.939 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.938+0000 7ff456000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff45015ca40 0x7ff4500ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:15.939 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.938+0000 7ff456000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff450141c60 con 0x7ff4500ad8e0 2024-04-20T10:22:15.939 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.938+0000 7ff456000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7ff45015cf90 con 0x7ff45015ca40 2024-04-20T10:22:15.939 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.938+0000 7ff456000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ff45015d270 con 0x7ff450156c20 2024-04-20T10:22:15.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.939+0000 7ff44fe00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7ff4500ad8e0 0x7ff4500d0360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:34946/0 (socket says 172.21.15.121:34946) 2024-04-20T10:22:15.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.939+0000 7ff44fe00640 1 -- 172.21.15.121:0/190269880 learned_addr learned my addr 172.21.15.121:0/190269880 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:15.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.939+0000 7ff44ea00640 1 --2- 172.21.15.121:0/190269880 >> v2:172.21.15.121:3300/0 conn(0x7ff45015ca40 0x7ff4500ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:15.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.939+0000 7ff44f400640 1 --2- 172.21.15.121:0/190269880 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff450156c20 0x7ff45006b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:15.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.939+0000 7ff44ea00640 1 -- 172.21.15.121:0/190269880 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff450156c20 msgr2=0x7ff45006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:15.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.939+0000 7ff44ea00640 1 --2- 172.21.15.121:0/190269880 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff450156c20 0x7ff45006b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:15.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.939+0000 7ff44ea00640 1 -- 172.21.15.121:0/190269880 >> v1:172.21.15.121:6789/0 conn(0x7ff4500ad8e0 legacy=0x7ff4500d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:15.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.939+0000 7ff44ea00640 1 -- 172.21.15.121:0/190269880 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff45015d550 con 0x7ff45015ca40 2024-04-20T10:22:15.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.939+0000 7ff44e000640 1 -- 172.21.15.121:0/190269880 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1129676213 0 0) 0x7ff450141c60 con 0x7ff4500ad8e0 2024-04-20T10:22:15.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.940+0000 7ff44ea00640 1 --2- 172.21.15.121:0/190269880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff45015ca40 0x7ff4500ad3a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff43c0097a0 tx=0x7ff43c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd1723ee4016e1b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:15.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.940+0000 7ff44e000640 1 -- 172.21.15.121:0/190269880 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43c035020 con 0x7ff45015ca40 2024-04-20T10:22:15.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.941+0000 7ff44e000640 1 -- 172.21.15.121:0/190269880 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff43c031830 con 0x7ff45015ca40 2024-04-20T10:22:15.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.941+0000 7ff44e000640 1 -- 172.21.15.121:0/190269880 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff43c031c00 con 0x7ff45015ca40 2024-04-20T10:22:15.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.941+0000 7ff456000640 1 -- 172.21.15.121:0/190269880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff45015ca40 msgr2=0x7ff4500ad3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:15.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.941+0000 7ff456000640 1 --2- 172.21.15.121:0/190269880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff45015ca40 0x7ff4500ad3a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff43c0097a0 tx=0x7ff43c02b8d0 comp rx=0 tx=0).stop 2024-04-20T10:22:15.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.941+0000 7ff456000640 1 -- 172.21.15.121:0/190269880 shutdown_connections 2024-04-20T10:22:15.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.941+0000 7ff456000640 1 --2- 172.21.15.121:0/190269880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff45015ca40 0x7ff4500ad3a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:15.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.941+0000 7ff456000640 1 --2- 172.21.15.121:0/190269880 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff450156c20 0x7ff45006b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:15.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.941+0000 7ff456000640 1 -- 172.21.15.121:0/190269880 >> 172.21.15.121:0/190269880 conn(0x7ff4500b3350 msgr2=0x7ff4500b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:15.943 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.941+0000 7ff456000640 1 -- 172.21.15.121:0/190269880 shutdown_connections 2024-04-20T10:22:15.943 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.941+0000 7ff456000640 1 -- 172.21.15.121:0/190269880 wait complete. 2024-04-20T10:22:15.943 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.943+0000 7ff456000640 1 Processor -- start 2024-04-20T10:22:15.943 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.943+0000 7ff456000640 1 -- start start 2024-04-20T10:22:15.945 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.944+0000 7ff456000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4500ad8e0 0x7ff45014fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:15.946 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.944+0000 7ff456000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff450156c20 0x7ff4501f26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:15.946 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.944+0000 7ff456000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff45015ca40 0x7ff450151ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:15.946 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.944+0000 7ff456000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff45006bfc0 con 0x7ff4500ad8e0 2024-04-20T10:22:15.946 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.944+0000 7ff456000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff45006be40 con 0x7ff45015ca40 2024-04-20T10:22:15.946 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.944+0000 7ff456000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ff4500d0a70 con 0x7ff450156c20 2024-04-20T10:22:15.946 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.944+0000 7ff44f400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4500ad8e0 0x7ff45014fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:15.946 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.944+0000 7ff44f400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4500ad8e0 0x7ff45014fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:42694/0 (socket says 172.21.15.121:42694) 2024-04-20T10:22:15.946 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.944+0000 7ff44f400640 1 -- 172.21.15.121:0/1871754537 learned_addr learned my addr 172.21.15.121:0/1871754537 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:15.946 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.945+0000 7ff44f400640 1 -- 172.21.15.121:0/1871754537 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff450156c20 msgr2=0x7ff4501f26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:15.946 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.945+0000 7ff44f400640 1 --2- 172.21.15.121:0/1871754537 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff450156c20 0x7ff4501f26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:15.946 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.945+0000 7ff44f400640 1 -- 172.21.15.121:0/1871754537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff45015ca40 msgr2=0x7ff450151ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:15.946 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.945+0000 7ff44f400640 1 --2- 172.21.15.121:0/1871754537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff45015ca40 0x7ff450151ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:15.947 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.945+0000 7ff44f400640 1 -- 172.21.15.121:0/1871754537 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4501558e0 con 0x7ff4500ad8e0 2024-04-20T10:22:15.947 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.945+0000 7ff44f400640 1 --2- 172.21.15.121:0/1871754537 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4500ad8e0 0x7ff45014fc90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff44000d830 tx=0x7ff44000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:15.947 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.945+0000 7ff447400640 1 -- 172.21.15.121:0/1871754537 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44000b6c0 con 0x7ff4500ad8e0 2024-04-20T10:22:15.947 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.945+0000 7ff447400640 1 -- 172.21.15.121:0/1871754537 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff440019070 con 0x7ff4500ad8e0 2024-04-20T10:22:15.947 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.945+0000 7ff456000640 1 -- 172.21.15.121:0/1871754537 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4500999c0 con 0x7ff4500ad8e0 2024-04-20T10:22:15.947 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.945+0000 7ff447400640 1 -- 172.21.15.121:0/1871754537 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44000ba80 con 0x7ff4500ad8e0 2024-04-20T10:22:15.947 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.945+0000 7ff456000640 1 -- 172.21.15.121:0/1871754537 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4500ccd80 con 0x7ff4500ad8e0 2024-04-20T10:22:15.947 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.946+0000 7ff456000640 1 -- 172.21.15.121:0/1871754537 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff450068980 con 0x7ff4500ad8e0 2024-04-20T10:22:15.948 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.947+0000 7ff447400640 1 -- 172.21.15.121:0/1871754537 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff44001e460 con 0x7ff4500ad8e0 2024-04-20T10:22:15.948 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.948+0000 7ff447400640 1 --2- 172.21.15.121:0/1871754537 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff43006c8f0 0x7ff43006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:15.948 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.948+0000 7ff447400640 1 -- 172.21.15.121:0/1871754537 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ff44008c700 con 0x7ff4500ad8e0 2024-04-20T10:22:15.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.948+0000 7ff44ea00640 1 --2- 172.21.15.121:0/1871754537 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff43006c8f0 0x7ff43006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:15.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.949+0000 7ff44ea00640 1 --2- 172.21.15.121:0/1871754537 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff43006c8f0 0x7ff43006edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff43c036000 tx=0x7ff43c030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:15.953 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:15.952+0000 7ff447400640 1 -- 172.21.15.121:0/1871754537 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff44005a690 con 0x7ff4500ad8e0 2024-04-20T10:22:16.159 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.158+0000 7ff456000640 1 -- 172.21.15.121:0/1871754537 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff4500cfd90 con 0x7ff4500ad8e0 2024-04-20T10:22:16.160 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.159+0000 7ff447400640 1 -- 172.21.15.121:0/1871754537 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7ff440059f90 con 0x7ff4500ad8e0 2024-04-20T10:22:16.160 INFO:teuthology.orchestra.run.smithi121.stderr:listed 0 entries 2024-04-20T10:22:16.163 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.162+0000 7ff456000640 1 -- 172.21.15.121:0/1871754537 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff43006c8f0 msgr2=0x7ff43006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:16.163 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.162+0000 7ff456000640 1 --2- 172.21.15.121:0/1871754537 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff43006c8f0 0x7ff43006edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff43c036000 tx=0x7ff43c030000 comp rx=0 tx=0).stop 2024-04-20T10:22:16.163 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.162+0000 7ff456000640 1 -- 172.21.15.121:0/1871754537 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4500ad8e0 msgr2=0x7ff45014fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:16.163 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.162+0000 7ff456000640 1 --2- 172.21.15.121:0/1871754537 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4500ad8e0 0x7ff45014fc90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff44000d830 tx=0x7ff44000dcf0 comp rx=0 tx=0).stop 2024-04-20T10:22:16.163 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.162+0000 7ff456000640 1 -- 172.21.15.121:0/1871754537 shutdown_connections 2024-04-20T10:22:16.163 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.162+0000 7ff456000640 1 --2- 172.21.15.121:0/1871754537 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff43006c8f0 0x7ff43006edb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:16.163 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.162+0000 7ff456000640 1 --2- 172.21.15.121:0/1871754537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff45015ca40 0x7ff450151ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:16.163 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.162+0000 7ff456000640 1 --2- 172.21.15.121:0/1871754537 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff450156c20 0x7ff4501f26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:16.163 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.162+0000 7ff456000640 1 --2- 172.21.15.121:0/1871754537 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff4500ad8e0 0x7ff45014fc90 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:16.163 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.162+0000 7ff456000640 1 -- 172.21.15.121:0/1871754537 >> 172.21.15.121:0/1871754537 conn(0x7ff4500b3350 msgr2=0x7ff450154f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:16.163 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.162+0000 7ff456000640 1 -- 172.21.15.121:0/1871754537 shutdown_connections 2024-04-20T10:22:16.163 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.162+0000 7ff456000640 1 -- 172.21.15.121:0/1871754537 wait complete. 2024-04-20T10:22:16.180 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi121.front.sepia.ceph.com... 2024-04-20T10:22:16.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:22:16.180 DEBUG:teuthology.orchestra.run.smithi121:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-20T10:22:16.259 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:22:16.259 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2024-04-20T10:22:16.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:22:16.260 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T10:22:16.308 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T10:22:16.308 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:22:16.366 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T10:22:16.367 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:22:16.419 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:22:16.622 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.621+0000 7f8c91800640 1 Processor -- start 2024-04-20T10:22:16.622 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.621+0000 7f8c91800640 1 -- start start 2024-04-20T10:22:16.624 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.623+0000 7f8c91800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8c8c15e8a0 0x7f8c8c06e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:16.625 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.623+0000 7f8c91800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8c8c0d0a70 0x7f8c8c0ad680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:16.625 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.623+0000 7f8c91800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c8c06f060 con 0x7f8c8c06dbb0 2024-04-20T10:22:16.625 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.623+0000 7f8c91800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f8c8c06e7c0 con 0x7f8c8c15e8a0 2024-04-20T10:22:16.625 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.623+0000 7f8c91800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f8c8c06eaa0 con 0x7f8c8c0d0a70 2024-04-20T10:22:16.625 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.623+0000 7f8c8aa00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8c8c15e8a0 0x7f8c8c06e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:16.625 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.623+0000 7f8c8aa00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8c8c15e8a0 0x7f8c8c06e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54892/0 (socket says 172.21.15.121:54892) 2024-04-20T10:22:16.625 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.623+0000 7f8c8aa00640 1 -- 172.21.15.121:0/4194286464 learned_addr learned my addr 172.21.15.121:0/4194286464 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:16.625 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.624+0000 7f8c8b400640 1 --2- 172.21.15.121:0/4194286464 >> v2:172.21.15.123:3300/0 conn(0x7f8c8c0d0a70 0x7f8c8c0ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:16.625 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.624+0000 7f8c8aa00640 1 -- 172.21.15.121:0/4194286464 >> v2:172.21.15.123:3300/0 conn(0x7f8c8c0d0a70 msgr2=0x7f8c8c0ad680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:16.625 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.624+0000 7f8c8aa00640 1 --2- 172.21.15.121:0/4194286464 >> v2:172.21.15.123:3300/0 conn(0x7f8c8c0d0a70 0x7f8c8c0ad680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:16.625 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.624+0000 7f8c8aa00640 1 -- 172.21.15.121:0/4194286464 >> v1:172.21.15.123:6789/0 conn(0x7f8c8c06dbb0 legacy=0x7f8c8c0d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:16.626 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.624+0000 7f8c8aa00640 1 -- 172.21.15.121:0/4194286464 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c8c06ed80 con 0x7f8c8c15e8a0 2024-04-20T10:22:16.626 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.625+0000 7f8c8aa00640 1 --2- 172.21.15.121:0/4194286464 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c8c15e8a0 0x7f8c8c06e1e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c0d0f70 tx=0x7f8c7c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e5cc588ffe6f7de server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:16.626 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.626+0000 7f8c89600640 1 -- 172.21.15.121:0/4194286464 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c02ca80 con 0x7f8c8c15e8a0 2024-04-20T10:22:16.626 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.626+0000 7f8c89600640 1 -- 172.21.15.121:0/4194286464 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c7c02cc20 con 0x7f8c8c15e8a0 2024-04-20T10:22:16.627 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.626+0000 7f8c89600640 1 -- 172.21.15.121:0/4194286464 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7c0343f0 con 0x7f8c8c15e8a0 2024-04-20T10:22:16.627 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.627+0000 7f8c91800640 1 -- 172.21.15.121:0/4194286464 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c8c15e8a0 msgr2=0x7f8c8c06e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:16.627 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.627+0000 7f8c91800640 1 --2- 172.21.15.121:0/4194286464 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c8c15e8a0 0x7f8c8c06e1e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c0d0f70 tx=0x7f8c7c02bee0 comp rx=0 tx=0).stop 2024-04-20T10:22:16.627 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.627+0000 7f8c91800640 1 -- 172.21.15.121:0/4194286464 shutdown_connections 2024-04-20T10:22:16.628 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.627+0000 7f8c91800640 1 --2- 172.21.15.121:0/4194286464 >> v2:172.21.15.123:3300/0 conn(0x7f8c8c0d0a70 0x7f8c8c0ad680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:16.628 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.627+0000 7f8c91800640 1 --2- 172.21.15.121:0/4194286464 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c8c15e8a0 0x7f8c8c06e1e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:16.628 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.627+0000 7f8c91800640 1 -- 172.21.15.121:0/4194286464 >> 172.21.15.121:0/4194286464 conn(0x7f8c8c0b3350 msgr2=0x7f8c8c0b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:16.628 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.627+0000 7f8c91800640 1 -- 172.21.15.121:0/4194286464 shutdown_connections 2024-04-20T10:22:16.628 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.627+0000 7f8c91800640 1 -- 172.21.15.121:0/4194286464 wait complete. 2024-04-20T10:22:16.629 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.629+0000 7f8c91800640 1 Processor -- start 2024-04-20T10:22:16.629 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.629+0000 7f8c91800640 1 -- start start 2024-04-20T10:22:16.631 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.630+0000 7f8c91800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c8c06dbb0 0x7f8c8c144660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:16.631 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.630+0000 7f8c91800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8c8c0d0a70 0x7f8c8c14f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:16.631 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.630+0000 7f8c91800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c8c15e8a0 0x7f8c8c14fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:16.631 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.630+0000 7f8c91800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8c06fef0 con 0x7f8c8c15e8a0 2024-04-20T10:22:16.631 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.630+0000 7f8c91800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8c06fd70 con 0x7f8c8c06dbb0 2024-04-20T10:22:16.631 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.630+0000 7f8c91800640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f8c8c0adbc0 con 0x7f8c8c0d0a70 2024-04-20T10:22:16.631 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.631+0000 7f8c8b400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c8c15e8a0 0x7f8c8c14fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:16.632 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.631+0000 7f8c8b400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c8c15e8a0 0x7f8c8c14fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54902/0 (socket says 172.21.15.121:54902) 2024-04-20T10:22:16.632 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.631+0000 7f8c8b400640 1 -- 172.21.15.121:0/3693133673 learned_addr learned my addr 172.21.15.121:0/3693133673 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:16.633 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.631+0000 7f8c8a000640 1 --2- 172.21.15.121:0/3693133673 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8c8c0d0a70 0x7f8c8c14f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:16.633 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.631+0000 7f8c8b400640 1 -- 172.21.15.121:0/3693133673 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8c8c0d0a70 msgr2=0x7f8c8c14f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:16.633 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.631+0000 7f8c8aa00640 1 --2- 172.21.15.121:0/3693133673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c8c06dbb0 0x7f8c8c144660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:16.633 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.631+0000 7f8c8b400640 1 --2- 172.21.15.121:0/3693133673 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8c8c0d0a70 0x7f8c8c14f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:16.633 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.631+0000 7f8c8b400640 1 -- 172.21.15.121:0/3693133673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c8c06dbb0 msgr2=0x7f8c8c144660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:16.633 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.631+0000 7f8c8b400640 1 --2- 172.21.15.121:0/3693133673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c8c06dbb0 0x7f8c8c144660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:16.633 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.631+0000 7f8c8b400640 1 -- 172.21.15.121:0/3693133673 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c8c153080 con 0x7f8c8c15e8a0 2024-04-20T10:22:16.633 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.631+0000 7f8c8b400640 1 --2- 172.21.15.121:0/3693133673 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c8c15e8a0 0x7f8c8c14fc00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8c8400d4f0 tx=0x7f8c8400d9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:16.634 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.632+0000 7f8c82a00640 1 -- 172.21.15.121:0/3693133673 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c840125f0 con 0x7f8c8c15e8a0 2024-04-20T10:22:16.634 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.632+0000 7f8c82a00640 1 -- 172.21.15.121:0/3693133673 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c84019070 con 0x7f8c8c15e8a0 2024-04-20T10:22:16.634 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.632+0000 7f8c91800640 1 -- 172.21.15.121:0/3693133673 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c8c153360 con 0x7f8c8c15e8a0 2024-04-20T10:22:16.634 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.632+0000 7f8c82a00640 1 -- 172.21.15.121:0/3693133673 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c840129f0 con 0x7f8c8c15e8a0 2024-04-20T10:22:16.634 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.632+0000 7f8c91800640 1 -- 172.21.15.121:0/3693133673 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c8c153920 con 0x7f8c8c15e8a0 2024-04-20T10:22:16.635 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.633+0000 7f8c91800640 1 -- 172.21.15.121:0/3693133673 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c8c068980 con 0x7f8c8c15e8a0 2024-04-20T10:22:16.635 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.634+0000 7f8c82a00640 1 -- 172.21.15.121:0/3693133673 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8c84012b90 con 0x7f8c8c15e8a0 2024-04-20T10:22:16.635 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.635+0000 7f8c82a00640 1 --2- 172.21.15.121:0/3693133673 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f8c6806c8f0 0x7f8c6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:16.635 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.635+0000 7f8c82a00640 1 -- 172.21.15.121:0/3693133673 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f8c8408c2c0 con 0x7f8c8c15e8a0 2024-04-20T10:22:16.636 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.635+0000 7f8c8aa00640 1 --2- 172.21.15.121:0/3693133673 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f8c6806c8f0 0x7f8c6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:16.636 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.636+0000 7f8c8aa00640 1 --2- 172.21.15.121:0/3693133673 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f8c6806c8f0 0x7f8c6806edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c035000 tx=0x7f8c7c030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:16.639 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.639+0000 7f8c82a00640 1 -- 172.21.15.121:0/3693133673 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8c8405a2e0 con 0x7f8c8c15e8a0 2024-04-20T10:22:16.844 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.844+0000 7f8c91800640 1 -- 172.21.15.121:0/3693133673 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8c8c06d160 con 0x7f8c8c15e8a0 2024-04-20T10:22:16.847 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.847+0000 7f8c82a00640 1 -- 172.21.15.121:0/3693133673 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18940 (secure 0 0 0) 0x7f8c84059be0 con 0x7f8c8c15e8a0 2024-04-20T10:22:16.848 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:22:16.848 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":27,"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","created":"2024-04-20T10:20:38.076076+0000","modified":"2024-04-20T10:21:44.715898+0000","last_up_change":"2024-04-20T10:20:50.162700+0000","last_in_change":"2024-04-20T10:20:41.510382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:20:51.141365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:21:00.436937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:21:01.026957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:21:04.862693+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"629a7b8c-a40a-4a2a-a3cc-d9f357774310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6809","nonce":1044588835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6811","nonce":1044588835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6815","nonce":1044588835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6813","nonce":1044588835}]},"public_addr":"172.21.15.121:6809/1044588835","cluster_addr":"172.21.15.121:6811/1044588835","heartbeat_back_addr":"172.21.15.121:6815/1044588835","heartbeat_front_addr":"172.21.15.121:6813/1044588835","state":["exists","up"]},{"osd":1,"uuid":"f2570399-aa50-4ff3-bf5d-3d09207290db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6801","nonce":3505199624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6803","nonce":3505199624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6807","nonce":3505199624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6805","nonce":3505199624}]},"public_addr":"172.21.15.121:6801/3505199624","cluster_addr":"172.21.15.121:6803/3505199624","heartbeat_back_addr":"172.21.15.121:6807/3505199624","heartbeat_front_addr":"172.21.15.121:6805/3505199624","state":["exists","up"]},{"osd":2,"uuid":"50893807-dc30-49f4-b41f-739366626658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6817","nonce":3656724207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6819","nonce":3656724207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6827","nonce":3656724207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6822","nonce":3656724207}]},"public_addr":"172.21.15.121:6817/3656724207","cluster_addr":"172.21.15.121:6819/3656724207","heartbeat_back_addr":"172.21.15.121:6827/3656724207","heartbeat_front_addr":"172.21.15.121:6822/3656724207","state":["exists","up"]},{"osd":3,"uuid":"dadb01bc-5491-4ffc-8824-d48a057240b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6823","nonce":2232342832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6826","nonce":2232342832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6831","nonce":2232342832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6829","nonce":2232342832}]},"public_addr":"172.21.15.121:6823/2232342832","cluster_addr":"172.21.15.121:6826/2232342832","heartbeat_back_addr":"172.21.15.121:6831/2232342832","heartbeat_front_addr":"172.21.15.121:6829/2232342832","state":["exists","up"]},{"osd":4,"uuid":"b3f3b4a1-b375-4526-b69e-344ab24c2da1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6801","nonce":99353111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6803","nonce":99353111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6807","nonce":99353111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6805","nonce":99353111}]},"public_addr":"172.21.15.123:6801/99353111","cluster_addr":"172.21.15.123:6803/99353111","heartbeat_back_addr":"172.21.15.123:6807/99353111","heartbeat_front_addr":"172.21.15.123:6805/99353111","state":["exists","up"]},{"osd":5,"uuid":"2ee24a08-1624-4961-9642-d7b510282484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6818","nonce":2504374688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6822","nonce":2504374688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6830","nonce":2504374688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6826","nonce":2504374688}]},"public_addr":"172.21.15.123:6818/2504374688","cluster_addr":"172.21.15.123:6822/2504374688","heartbeat_back_addr":"172.21.15.123:6830/2504374688","heartbeat_front_addr":"172.21.15.123:6826/2504374688","state":["exists","up"]},{"osd":6,"uuid":"186118cc-6128-49de-acca-55260945c1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6819","nonce":4139299599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6823","nonce":4139299599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6829","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6831","nonce":4139299599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6827","nonce":4139299599}]},"public_addr":"172.21.15.123:6819/4139299599","cluster_addr":"172.21.15.123:6823/4139299599","heartbeat_back_addr":"172.21.15.123:6831/4139299599","heartbeat_front_addr":"172.21.15.123:6827/4139299599","state":["exists","up"]},{"osd":7,"uuid":"a00aac6e-4bd9-415a-a93d-ae85210865ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6809","nonce":4043981069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6811","nonce":4043981069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6815","nonce":4043981069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6813","nonce":4043981069}]},"public_addr":"172.21.15.123:6809/4043981069","cluster_addr":"172.21.15.123:6811/4043981069","heartbeat_back_addr":"172.21.15.123:6815/4043981069","heartbeat_front_addr":"172.21.15.123:6813/4043981069","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.904540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.958585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.616367+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.885244+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.103127+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.167248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.884348+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.862356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.23","mappings":[{"from":3,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:22:16.850 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.850+0000 7f8c91800640 1 -- 172.21.15.121:0/3693133673 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f8c6806c8f0 msgr2=0x7f8c6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:16.850 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.850+0000 7f8c91800640 1 --2- 172.21.15.121:0/3693133673 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f8c6806c8f0 0x7f8c6806edb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c035000 tx=0x7f8c7c030000 comp rx=0 tx=0).stop 2024-04-20T10:22:16.850 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.850+0000 7f8c91800640 1 -- 172.21.15.121:0/3693133673 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c8c15e8a0 msgr2=0x7f8c8c14fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:16.850 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.850+0000 7f8c91800640 1 --2- 172.21.15.121:0/3693133673 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c8c15e8a0 0x7f8c8c14fc00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8c8400d4f0 tx=0x7f8c8400d9b0 comp rx=0 tx=0).stop 2024-04-20T10:22:16.851 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.850+0000 7f8c91800640 1 -- 172.21.15.121:0/3693133673 shutdown_connections 2024-04-20T10:22:16.851 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.850+0000 7f8c91800640 1 --2- 172.21.15.121:0/3693133673 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8c8c15e8a0 0x7f8c8c14fc00 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:16.851 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.850+0000 7f8c91800640 1 --2- 172.21.15.121:0/3693133673 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8c8c0d0a70 0x7f8c8c14f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:16.851 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.850+0000 7f8c91800640 1 --2- 172.21.15.121:0/3693133673 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f8c6806c8f0 0x7f8c6806edb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:16.851 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.850+0000 7f8c91800640 1 --2- 172.21.15.121:0/3693133673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c8c06dbb0 0x7f8c8c144660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:16.851 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.850+0000 7f8c91800640 1 -- 172.21.15.121:0/3693133673 >> 172.21.15.121:0/3693133673 conn(0x7f8c8c0b3350 msgr2=0x7f8c8c160560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:16.851 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.850+0000 7f8c91800640 1 -- 172.21.15.121:0/3693133673 shutdown_connections 2024-04-20T10:22:16.851 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:16.850+0000 7f8c91800640 1 -- 172.21.15.121:0/3693133673 wait complete. 2024-04-20T10:22:16.870 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:20:51.141365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:21:00.436937+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:21:01.026957+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T10:21:04.862693+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T10:22:16.871 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:22:17.075 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.074+0000 7f41f6000640 1 Processor -- start 2024-04-20T10:22:17.075 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.074+0000 7f41f6000640 1 -- start start 2024-04-20T10:22:17.076 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.075+0000 7f41f6000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f41f0072a20 0x7f41f0072e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:17.076 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.075+0000 7f41f6000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41f0160690 con 0x7f41f00b49f0 2024-04-20T10:22:17.076 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.075+0000 7f41f6000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41f0160970 con 0x7f41f00b4610 2024-04-20T10:22:17.076 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.075+0000 7f41f6000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f41f00733d0 con 0x7f41f0072a20 2024-04-20T10:22:17.077 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.076+0000 7f41f5600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f41f0072a20 0x7f41f0072e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:17.077 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.076+0000 7f41efe00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f41f00b49f0 0x7f41f015d080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:53364/0 (socket says 172.21.15.121:53364) 2024-04-20T10:22:17.077 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.076+0000 7f41efe00640 1 -- 172.21.15.121:0/3407288851 learned_addr learned my addr 172.21.15.121:0/3407288851 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:17.077 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.076+0000 7f41f5600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f41f0072a20 0x7f41f0072e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54908/0 (socket says 172.21.15.121:54908) 2024-04-20T10:22:17.077 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.076+0000 7f41f5600640 1 -- 172.21.15.121:0/3407288851 >> v1:172.21.15.123:6789/0 conn(0x7f41f00b4610 legacy=0x7f41f015fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:17.077 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.076+0000 7f41f5600640 1 -- 172.21.15.121:0/3407288851 >> v1:172.21.15.121:6789/0 conn(0x7f41f00b49f0 legacy=0x7f41f015d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:17.077 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.076+0000 7f41f5600640 1 -- 172.21.15.121:0/3407288851 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f0160c50 con 0x7f41f0072a20 2024-04-20T10:22:17.077 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.076+0000 7f41ef400640 1 -- 172.21.15.121:0/3407288851 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2114806257 0 0) 0x7f41f0160690 con 0x7f41f00b49f0 2024-04-20T10:22:17.078 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.078+0000 7f41f5600640 1 --2- 172.21.15.121:0/3407288851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41f0072a20 0x7f41f0072e90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f41e8009d90 tx=0x7f41e802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d242be8ceed0a3ec server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:17.078 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.078+0000 7f41ef400640 1 -- 172.21.15.121:0/3407288851 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e8039070 con 0x7f41f0072a20 2024-04-20T10:22:17.079 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.078+0000 7f41ef400640 1 -- 172.21.15.121:0/3407288851 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41e802cc60 con 0x7f41f0072a20 2024-04-20T10:22:17.079 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.078+0000 7f41ef400640 1 -- 172.21.15.121:0/3407288851 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e8034660 con 0x7f41f0072a20 2024-04-20T10:22:17.080 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.079+0000 7f41f6000640 1 -- 172.21.15.121:0/3407288851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41f0072a20 msgr2=0x7f41f0072e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:17.080 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.079+0000 7f41f6000640 1 --2- 172.21.15.121:0/3407288851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41f0072a20 0x7f41f0072e90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f41e8009d90 tx=0x7f41e802beb0 comp rx=0 tx=0).stop 2024-04-20T10:22:17.080 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.079+0000 7f41f6000640 1 -- 172.21.15.121:0/3407288851 shutdown_connections 2024-04-20T10:22:17.080 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.079+0000 7f41f6000640 1 --2- 172.21.15.121:0/3407288851 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41f0072a20 0x7f41f0072e90 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:17.080 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.079+0000 7f41f6000640 1 -- 172.21.15.121:0/3407288851 >> 172.21.15.121:0/3407288851 conn(0x7f41f00cdbc0 msgr2=0x7f41f00cdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:17.080 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.079+0000 7f41f6000640 1 -- 172.21.15.121:0/3407288851 shutdown_connections 2024-04-20T10:22:17.080 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.079+0000 7f41f6000640 1 -- 172.21.15.121:0/3407288851 wait complete. 2024-04-20T10:22:17.081 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.081+0000 7f41f6000640 1 Processor -- start 2024-04-20T10:22:17.081 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.081+0000 7f41f6000640 1 -- start start 2024-04-20T10:22:17.084 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.082+0000 7f41f6000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41f0072a20 0x7f41f01f3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:17.084 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.082+0000 7f41f6000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41f00b4610 0x7f41f01552f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:17.084 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.082+0000 7f41f6000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f41f00b49f0 0x7f41f015b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:17.084 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.082+0000 7f41f6000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f41f015c850 con 0x7f41f0072a20 2024-04-20T10:22:17.084 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.082+0000 7f41f6000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f41f015d790 con 0x7f41f00b4610 2024-04-20T10:22:17.084 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.082+0000 7f41f6000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f41f00b4ea0 con 0x7f41f00b49f0 2024-04-20T10:22:17.084 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.082+0000 7f41f4c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41f0072a20 0x7f41f01f3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:17.085 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.082+0000 7f41f4c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41f0072a20 0x7f41f01f3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54912/0 (socket says 172.21.15.121:54912) 2024-04-20T10:22:17.085 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.082+0000 7f41f4c00640 1 -- 172.21.15.121:0/399557713 learned_addr learned my addr 172.21.15.121:0/399557713 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:17.085 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.082+0000 7f41efe00640 1 --2- 172.21.15.121:0/399557713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41f00b4610 0x7f41f01552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:17.085 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.082+0000 7f41f5600640 1 --2- 172.21.15.121:0/399557713 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f41f00b49f0 0x7f41f015b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:17.085 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.082+0000 7f41f4c00640 1 -- 172.21.15.121:0/399557713 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f41f00b49f0 msgr2=0x7f41f015b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:17.085 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.082+0000 7f41f4c00640 1 --2- 172.21.15.121:0/399557713 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f41f00b49f0 0x7f41f015b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:17.085 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.082+0000 7f41f4c00640 1 -- 172.21.15.121:0/399557713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41f00b4610 msgr2=0x7f41f01552f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:17.085 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.082+0000 7f41f4c00640 1 --2- 172.21.15.121:0/399557713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41f00b4610 0x7f41f01552f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:17.085 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.082+0000 7f41f4c00640 1 -- 172.21.15.121:0/399557713 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f0155d50 con 0x7f41f0072a20 2024-04-20T10:22:17.085 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.083+0000 7f41efe00640 1 --2- 172.21.15.121:0/399557713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41f00b4610 0x7f41f01552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:17.085 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.083+0000 7f41f4c00640 1 --2- 172.21.15.121:0/399557713 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41f0072a20 0x7f41f01f3b80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f41e400b3f0 tx=0x7f41e400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:17.085 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.083+0000 7f41ecc00640 1 -- 172.21.15.121:0/399557713 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e4015020 con 0x7f41f0072a20 2024-04-20T10:22:17.086 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.083+0000 7f41ecc00640 1 -- 172.21.15.121:0/399557713 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41e4011830 con 0x7f41f0072a20 2024-04-20T10:22:17.086 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.083+0000 7f41f6000640 1 -- 172.21.15.121:0/399557713 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41f0156030 con 0x7f41f0072a20 2024-04-20T10:22:17.086 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.083+0000 7f41ecc00640 1 -- 172.21.15.121:0/399557713 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e4011c20 con 0x7f41f0072a20 2024-04-20T10:22:17.086 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.083+0000 7f41f6000640 1 -- 172.21.15.121:0/399557713 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41f00ced90 con 0x7f41f0072a20 2024-04-20T10:22:17.087 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.085+0000 7f41f6000640 1 -- 172.21.15.121:0/399557713 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41f0075b30 con 0x7f41f0072a20 2024-04-20T10:22:17.087 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.086+0000 7f41ecc00640 1 -- 172.21.15.121:0/399557713 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f41e4018020 con 0x7f41f0072a20 2024-04-20T10:22:17.087 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.087+0000 7f41ecc00640 1 --2- 172.21.15.121:0/399557713 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f41d006c8f0 0x7f41d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:17.087 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.087+0000 7f41ecc00640 1 -- 172.21.15.121:0/399557713 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f41e408bcd0 con 0x7f41f0072a20 2024-04-20T10:22:17.087 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.087+0000 7f41efe00640 1 --2- 172.21.15.121:0/399557713 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f41d006c8f0 0x7f41d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:17.088 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.087+0000 7f41efe00640 1 --2- 172.21.15.121:0/399557713 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f41d006c8f0 0x7f41d006edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f41f01f7850 tx=0x7f41dc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:17.091 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.091+0000 7f41ecc00640 1 -- 172.21.15.121:0/399557713 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f41e401f030 con 0x7f41f0072a20 2024-04-20T10:22:17.290 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.290+0000 7f41f6000640 1 -- 172.21.15.121:0/399557713 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f41f00b5280 con 0x7f41f0072a20 2024-04-20T10:22:17.291 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.290+0000 7f41ecc00640 1 -- 172.21.15.121:0/399557713 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f41e40172c0 con 0x7f41f0072a20 2024-04-20T10:22:17.291 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2024-04-20T10:22:17.293 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.293+0000 7f41f6000640 1 -- 172.21.15.121:0/399557713 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f41d006c8f0 msgr2=0x7f41d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:17.293 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.293+0000 7f41f6000640 1 --2- 172.21.15.121:0/399557713 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f41d006c8f0 0x7f41d006edb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f41f01f7850 tx=0x7f41dc009040 comp rx=0 tx=0).stop 2024-04-20T10:22:17.294 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.293+0000 7f41f6000640 1 -- 172.21.15.121:0/399557713 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41f0072a20 msgr2=0x7f41f01f3b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:17.294 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.293+0000 7f41f6000640 1 --2- 172.21.15.121:0/399557713 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41f0072a20 0x7f41f01f3b80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f41e400b3f0 tx=0x7f41e400b8b0 comp rx=0 tx=0).stop 2024-04-20T10:22:17.294 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.293+0000 7f41f6000640 1 -- 172.21.15.121:0/399557713 shutdown_connections 2024-04-20T10:22:17.294 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.293+0000 7f41f6000640 1 --2- 172.21.15.121:0/399557713 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f41f00b49f0 0x7f41f015b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:17.294 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.293+0000 7f41f6000640 1 --2- 172.21.15.121:0/399557713 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f41d006c8f0 0x7f41d006edb0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:17.294 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.293+0000 7f41f6000640 1 --2- 172.21.15.121:0/399557713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41f00b4610 0x7f41f01552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:17.294 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.293+0000 7f41f6000640 1 --2- 172.21.15.121:0/399557713 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f41f0072a20 0x7f41f01f3b80 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:17.294 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.293+0000 7f41f6000640 1 -- 172.21.15.121:0/399557713 >> 172.21.15.121:0/399557713 conn(0x7f41f00cdbc0 msgr2=0x7f41f006ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:17.294 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.293+0000 7f41f6000640 1 -- 172.21.15.121:0/399557713 shutdown_connections 2024-04-20T10:22:17.294 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.293+0000 7f41f6000640 1 -- 172.21.15.121:0/399557713 wait complete. 2024-04-20T10:22:17.313 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:22:17.504 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.503+0000 7f5b3e600640 1 Processor -- start 2024-04-20T10:22:17.504 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.503+0000 7f5b3e600640 1 -- start start 2024-04-20T10:22:17.505 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.505+0000 7f5b3e600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5b38071b50 0x7f5b3815abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:17.505 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.505+0000 7f5b3e600640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b380ae9b0 con 0x7f5b380af800 2024-04-20T10:22:17.506 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.505+0000 7f5b3e600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b3806e840 con 0x7f5b380725b0 2024-04-20T10:22:17.506 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.505+0000 7f5b3e600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f5b380d0a70 con 0x7f5b38071b50 2024-04-20T10:22:17.506 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.505+0000 7f5b36a00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f5b380af800 0x7f5b380d0360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:53372/0 (socket says 172.21.15.121:53372) 2024-04-20T10:22:17.506 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.505+0000 7f5b36a00640 1 -- 172.21.15.121:0/289380994 learned_addr learned my addr 172.21.15.121:0/289380994 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:17.508 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.506+0000 7f5b37400640 1 --2- 172.21.15.121:0/289380994 >> v2:172.21.15.123:3300/0 conn(0x7f5b38071b50 0x7f5b3815abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:17.508 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.506+0000 7f5b36000640 1 -- 172.21.15.121:0/289380994 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 228393691 0 0) 0x7f5b380ae9b0 con 0x7f5b380af800 2024-04-20T10:22:17.508 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.506+0000 7f5b36000640 1 -- 172.21.15.121:0/289380994 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b1c003520 con 0x7f5b380af800 2024-04-20T10:22:17.508 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.506+0000 7f5b36000640 1 -- 172.21.15.121:0/289380994 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1026026780 0 0) 0x7f5b1c003520 con 0x7f5b380af800 2024-04-20T10:22:17.508 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.506+0000 7f5b36000640 1 -- 172.21.15.121:0/289380994 >> v2:172.21.15.123:3300/0 conn(0x7f5b38071b50 msgr2=0x7f5b3815abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:17.508 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.506+0000 7f5b36000640 1 --2- 172.21.15.121:0/289380994 >> v2:172.21.15.123:3300/0 conn(0x7f5b38071b50 0x7f5b3815abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:17.508 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.506+0000 7f5b36000640 1 -- 172.21.15.121:0/289380994 >> v1:172.21.15.123:6789/0 conn(0x7f5b380725b0 legacy=0x7f5b380729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:17.508 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.506+0000 7f5b36000640 1 -- 172.21.15.121:0/289380994 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b3815b350 con 0x7f5b380af800 2024-04-20T10:22:17.508 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.506+0000 7f5b36000640 1 -- 172.21.15.121:0/289380994 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f5b20001670 con 0x7f5b380af800 2024-04-20T10:22:17.509 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.506+0000 7f5b36000640 1 -- 172.21.15.121:0/289380994 >> v1:172.21.15.121:6789/0 conn(0x7f5b380af800 legacy=0x7f5b380d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:17.509 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.506+0000 7f5b36000640 1 --2- 172.21.15.121:0/289380994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b1c003a00 0x7f5b1c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:17.509 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.506+0000 7f5b36000640 1 --2- 172.21.15.121:0/289380994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b1c004800 0x7f5b1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:17.509 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.506+0000 7f5b36000640 1 --2- 172.21.15.121:0/289380994 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b1c0051c0 0x7f5b1c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:17.509 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.507+0000 7f5b36000640 1 -- 172.21.15.121:0/289380994 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5b380d0a70 con 0x7f5b1c003a00 2024-04-20T10:22:17.509 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.507+0000 7f5b36000640 1 -- 172.21.15.121:0/289380994 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1c00c0c0 con 0x7f5b1c004800 2024-04-20T10:22:17.509 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.507+0000 7f5b36000640 1 -- 172.21.15.121:0/289380994 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f5b1c00c2c0 con 0x7f5b1c0051c0 2024-04-20T10:22:17.509 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.507+0000 7f5b37400640 1 --2- 172.21.15.121:0/289380994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b1c003a00 0x7f5b1c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:17.509 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.507+0000 7f5b36a00640 1 --2- 172.21.15.121:0/289380994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b1c004800 0x7f5b1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:17.509 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.507+0000 7f5b37e00640 1 --2- 172.21.15.121:0/289380994 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b1c0051c0 0x7f5b1c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:17.509 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.507+0000 7f5b37400640 1 -- 172.21.15.121:0/289380994 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b1c0051c0 msgr2=0x7f5b1c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:17.509 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.507+0000 7f5b37400640 1 --2- 172.21.15.121:0/289380994 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b1c0051c0 0x7f5b1c00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:17.509 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.507+0000 7f5b37400640 1 -- 172.21.15.121:0/289380994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b1c004800 msgr2=0x7f5b1c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:17.509 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.507+0000 7f5b37400640 1 --2- 172.21.15.121:0/289380994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b1c004800 0x7f5b1c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:17.509 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.507+0000 7f5b37400640 1 -- 172.21.15.121:0/289380994 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b1c003520 con 0x7f5b1c003a00 2024-04-20T10:22:17.510 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.507+0000 7f5b37e00640 1 -- 172.21.15.121:0/289380994 reap_dead start 2024-04-20T10:22:17.511 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.508+0000 7f5b37400640 1 --2- 172.21.15.121:0/289380994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b1c003a00 0x7f5b1c003e20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5b28007400 tx=0x7f5b1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbf2b90488d996c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:17.511 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.509+0000 7f5b36000640 1 -- 172.21.15.121:0/289380994 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2800bda0 con 0x7f5b1c003a00 2024-04-20T10:22:17.511 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.509+0000 7f5b36000640 1 -- 172.21.15.121:0/289380994 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b2800b0d0 con 0x7f5b1c003a00 2024-04-20T10:22:17.511 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.509+0000 7f5b36000640 1 -- 172.21.15.121:0/289380994 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b28009540 con 0x7f5b1c003a00 2024-04-20T10:22:17.511 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.509+0000 7f5b3e600640 1 -- 172.21.15.121:0/289380994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b1c003a00 msgr2=0x7f5b1c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:17.511 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.509+0000 7f5b3e600640 1 --2- 172.21.15.121:0/289380994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b1c003a00 0x7f5b1c003e20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5b28007400 tx=0x7f5b1c005830 comp rx=0 tx=0).stop 2024-04-20T10:22:17.511 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.509+0000 7f5b3e600640 1 -- 172.21.15.121:0/289380994 shutdown_connections 2024-04-20T10:22:17.511 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.509+0000 7f5b3e600640 1 --2- 172.21.15.121:0/289380994 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b1c003a00 0x7f5b1c003e20 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:17.511 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.509+0000 7f5b3e600640 1 -- 172.21.15.121:0/289380994 >> 172.21.15.121:0/289380994 conn(0x7f5b380b3350 msgr2=0x7f5b380b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:17.511 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.509+0000 7f5b3e600640 1 -- 172.21.15.121:0/289380994 shutdown_connections 2024-04-20T10:22:17.511 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.509+0000 7f5b3e600640 1 -- 172.21.15.121:0/289380994 wait complete. 2024-04-20T10:22:17.512 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.511+0000 7f5b3e600640 1 Processor -- start 2024-04-20T10:22:17.512 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.511+0000 7f5b3e600640 1 -- start start 2024-04-20T10:22:17.513 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.512+0000 7f5b3e600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b1c003a00 0x7f5b380b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:17.514 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.512+0000 7f5b3e600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b3809caf0 0x7f5b3809cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:17.514 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.512+0000 7f5b3e600640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b381566d0 0x7f5b38156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:17.514 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.512+0000 7f5b3e600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3815bf60 con 0x7f5b3809caf0 2024-04-20T10:22:17.514 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.512+0000 7f5b3e600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3815b910 con 0x7f5b1c003a00 2024-04-20T10:22:17.514 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.512+0000 7f5b3e600640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f5b3815bbf0 con 0x7f5b381566d0 2024-04-20T10:22:17.514 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.512+0000 7f5b36a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b3809caf0 0x7f5b3809cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:17.514 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.513+0000 7f5b36a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b3809caf0 0x7f5b3809cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54930/0 (socket says 172.21.15.121:54930) 2024-04-20T10:22:17.514 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.513+0000 7f5b36a00640 1 -- 172.21.15.121:0/3120043894 learned_addr learned my addr 172.21.15.121:0/3120043894 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:17.514 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.513+0000 7f5b37400640 1 --2- 172.21.15.121:0/3120043894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b1c003a00 0x7f5b380b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:17.514 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.513+0000 7f5b37e00640 1 --2- 172.21.15.121:0/3120043894 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b381566d0 0x7f5b38156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:17.514 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.513+0000 7f5b36a00640 1 -- 172.21.15.121:0/3120043894 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b381566d0 msgr2=0x7f5b38156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:17.514 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.513+0000 7f5b36a00640 1 --2- 172.21.15.121:0/3120043894 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b381566d0 0x7f5b38156b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:17.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.513+0000 7f5b36a00640 1 -- 172.21.15.121:0/3120043894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b1c003a00 msgr2=0x7f5b380b1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:17.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.513+0000 7f5b36a00640 1 --2- 172.21.15.121:0/3120043894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b1c003a00 0x7f5b380b1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:17.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.513+0000 7f5b36a00640 1 -- 172.21.15.121:0/3120043894 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b3809d690 con 0x7f5b3809caf0 2024-04-20T10:22:17.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.513+0000 7f5b37400640 1 --2- 172.21.15.121:0/3120043894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b1c003a00 0x7f5b380b1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:22:17.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.513+0000 7f5b36a00640 1 --2- 172.21.15.121:0/3120043894 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b3809caf0 0x7f5b3809cf70 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5b2000e700 tx=0x7f5b2000ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:17.516 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.514+0000 7f5b2f400640 1 -- 172.21.15.121:0/3120043894 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2000ee10 con 0x7f5b3809caf0 2024-04-20T10:22:17.516 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.514+0000 7f5b2f400640 1 -- 172.21.15.121:0/3120043894 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b20018070 con 0x7f5b3809caf0 2024-04-20T10:22:17.516 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.514+0000 7f5b2f400640 1 -- 172.21.15.121:0/3120043894 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2000cb10 con 0x7f5b3809caf0 2024-04-20T10:22:17.516 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.514+0000 7f5b3e600640 1 -- 172.21.15.121:0/3120043894 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b3809d970 con 0x7f5b3809caf0 2024-04-20T10:22:17.516 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.514+0000 7f5b3e600640 1 -- 172.21.15.121:0/3120043894 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b3809df30 con 0x7f5b3809caf0 2024-04-20T10:22:17.516 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.515+0000 7f5b3e600640 1 -- 172.21.15.121:0/3120043894 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b38068980 con 0x7f5b3809caf0 2024-04-20T10:22:17.516 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.516+0000 7f5b2f400640 1 -- 172.21.15.121:0/3120043894 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5b2000ccb0 con 0x7f5b3809caf0 2024-04-20T10:22:17.517 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.517+0000 7f5b2f400640 1 --2- 172.21.15.121:0/3120043894 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f5b1406c920 0x7f5b1406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:17.517 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.517+0000 7f5b2f400640 1 -- 172.21.15.121:0/3120043894 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f5b2008bdb0 con 0x7f5b3809caf0 2024-04-20T10:22:17.518 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.517+0000 7f5b37400640 1 --2- 172.21.15.121:0/3120043894 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f5b1406c920 0x7f5b1406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:17.518 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.518+0000 7f5b37400640 1 --2- 172.21.15.121:0/3120043894 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f5b1406c920 0x7f5b1406ede0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5b28035000 tx=0x7f5b2802e000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:17.521 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.521+0000 7f5b2f400640 1 -- 172.21.15.121:0/3120043894 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5b20059de0 con 0x7f5b3809caf0 2024-04-20T10:22:17.719 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.718+0000 7f5b3e600640 1 -- 172.21.15.121:0/3120043894 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5b380cfe90 con 0x7f5b3809caf0 2024-04-20T10:22:17.720 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.719+0000 7f5b2f400640 1 -- 172.21.15.121:0/3120043894 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f5b200596e0 con 0x7f5b3809caf0 2024-04-20T10:22:17.720 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 32 2024-04-20T10:22:17.722 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.722+0000 7f5b3e600640 1 -- 172.21.15.121:0/3120043894 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f5b1406c920 msgr2=0x7f5b1406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:17.723 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.722+0000 7f5b3e600640 1 --2- 172.21.15.121:0/3120043894 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f5b1406c920 0x7f5b1406ede0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5b28035000 tx=0x7f5b2802e000 comp rx=0 tx=0).stop 2024-04-20T10:22:17.723 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.722+0000 7f5b3e600640 1 -- 172.21.15.121:0/3120043894 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b3809caf0 msgr2=0x7f5b3809cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:17.723 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.722+0000 7f5b3e600640 1 --2- 172.21.15.121:0/3120043894 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b3809caf0 0x7f5b3809cf70 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5b2000e700 tx=0x7f5b2000ebc0 comp rx=0 tx=0).stop 2024-04-20T10:22:17.723 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.722+0000 7f5b3e600640 1 -- 172.21.15.121:0/3120043894 shutdown_connections 2024-04-20T10:22:17.723 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.722+0000 7f5b3e600640 1 --2- 172.21.15.121:0/3120043894 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b381566d0 0x7f5b38156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:17.723 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.722+0000 7f5b3e600640 1 --2- 172.21.15.121:0/3120043894 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b3809caf0 0x7f5b3809cf70 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:17.723 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.722+0000 7f5b3e600640 1 --2- 172.21.15.121:0/3120043894 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f5b1406c920 0x7f5b1406ede0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:17.723 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.722+0000 7f5b3e600640 1 --2- 172.21.15.121:0/3120043894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b1c003a00 0x7f5b380b1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:17.723 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.722+0000 7f5b3e600640 1 -- 172.21.15.121:0/3120043894 >> 172.21.15.121:0/3120043894 conn(0x7f5b380b3350 msgr2=0x7f5b3806b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:17.723 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.722+0000 7f5b3e600640 1 -- 172.21.15.121:0/3120043894 shutdown_connections 2024-04-20T10:22:17.723 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.722+0000 7f5b3e600640 1 -- 172.21.15.121:0/3120043894 wait complete. 2024-04-20T10:22:17.739 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:22:17.935 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.934+0000 7fe1e9000640 1 Processor -- start 2024-04-20T10:22:17.935 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.934+0000 7fe1e9000640 1 -- start start 2024-04-20T10:22:17.936 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.936+0000 7fe1e9000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe1e415e8a0 0x7fe1e406e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:17.936 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.936+0000 7fe1e9000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe1e40d0a70 0x7fe1e40ad680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:17.936 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.936+0000 7fe1e9000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1e406f060 con 0x7fe1e406dbb0 2024-04-20T10:22:17.936 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.936+0000 7fe1e9000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fe1e406e7c0 con 0x7fe1e415e8a0 2024-04-20T10:22:17.937 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.936+0000 7fe1e9000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fe1e406eaa0 con 0x7fe1e40d0a70 2024-04-20T10:22:17.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.936+0000 7fe1d9400640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fe1e406dbb0 0x7fe1e40d0360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:53380/0 (socket says 172.21.15.121:53380) 2024-04-20T10:22:17.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.936+0000 7fe1d9400640 1 -- 172.21.15.121:0/1601216979 learned_addr learned my addr 172.21.15.121:0/1601216979 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:17.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.936+0000 7fe1e1e00640 1 --2- 172.21.15.121:0/1601216979 >> v2:172.21.15.121:3300/0 conn(0x7fe1e415e8a0 0x7fe1e406e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:17.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.937+0000 7fe1e2800640 1 --2- 172.21.15.121:0/1601216979 >> v2:172.21.15.123:3300/0 conn(0x7fe1e40d0a70 0x7fe1e40ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:17.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.937+0000 7fe1e1400640 1 -- 172.21.15.121:0/1601216979 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3510460927 0 0) 0x7fe1e406f060 con 0x7fe1e406dbb0 2024-04-20T10:22:17.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.937+0000 7fe1e1400640 1 -- 172.21.15.121:0/1601216979 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1c8003520 con 0x7fe1e406dbb0 2024-04-20T10:22:17.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.937+0000 7fe1e1e00640 1 -- 172.21.15.121:0/1601216979 >> v2:172.21.15.123:3300/0 conn(0x7fe1e40d0a70 msgr2=0x7fe1e40ad680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:17.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.937+0000 7fe1e1e00640 1 --2- 172.21.15.121:0/1601216979 >> v2:172.21.15.123:3300/0 conn(0x7fe1e40d0a70 0x7fe1e40ad680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:17.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.937+0000 7fe1e1e00640 1 -- 172.21.15.121:0/1601216979 >> v1:172.21.15.121:6789/0 conn(0x7fe1e406dbb0 legacy=0x7fe1e40d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:17.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.937+0000 7fe1e1e00640 1 -- 172.21.15.121:0/1601216979 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e406ed80 con 0x7fe1e415e8a0 2024-04-20T10:22:17.939 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.939+0000 7fe1e1e00640 1 --2- 172.21.15.121:0/1601216979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1e415e8a0 0x7fe1e406e1e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe1e40d0f70 tx=0x7fe1d402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f2773abf1b6b56 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:17.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.939+0000 7fe1e1400640 1 -- 172.21.15.121:0/1601216979 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d4035020 con 0x7fe1e415e8a0 2024-04-20T10:22:17.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.939+0000 7fe1e1400640 1 -- 172.21.15.121:0/1601216979 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1d4031830 con 0x7fe1e415e8a0 2024-04-20T10:22:17.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.940+0000 7fe1e1400640 1 -- 172.21.15.121:0/1601216979 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d4031c00 con 0x7fe1e415e8a0 2024-04-20T10:22:17.941 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.940+0000 7fe1e9000640 1 -- 172.21.15.121:0/1601216979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1e415e8a0 msgr2=0x7fe1e406e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:17.941 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.940+0000 7fe1e9000640 1 --2- 172.21.15.121:0/1601216979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1e415e8a0 0x7fe1e406e1e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe1e40d0f70 tx=0x7fe1d402b8d0 comp rx=0 tx=0).stop 2024-04-20T10:22:17.941 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.940+0000 7fe1e9000640 1 -- 172.21.15.121:0/1601216979 shutdown_connections 2024-04-20T10:22:17.941 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.940+0000 7fe1e9000640 1 --2- 172.21.15.121:0/1601216979 >> v2:172.21.15.123:3300/0 conn(0x7fe1e40d0a70 0x7fe1e40ad680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:17.941 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.940+0000 7fe1e9000640 1 --2- 172.21.15.121:0/1601216979 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1e415e8a0 0x7fe1e406e1e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:17.941 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.940+0000 7fe1e9000640 1 -- 172.21.15.121:0/1601216979 >> 172.21.15.121:0/1601216979 conn(0x7fe1e40b3350 msgr2=0x7fe1e40b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:17.941 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.940+0000 7fe1e9000640 1 -- 172.21.15.121:0/1601216979 shutdown_connections 2024-04-20T10:22:17.941 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.940+0000 7fe1e9000640 1 -- 172.21.15.121:0/1601216979 wait complete. 2024-04-20T10:22:17.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.942+0000 7fe1e9000640 1 Processor -- start 2024-04-20T10:22:17.943 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.942+0000 7fe1e9000640 1 -- start start 2024-04-20T10:22:17.944 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.943+0000 7fe1e9000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1e406dbb0 0x7fe1e4146c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:17.944 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.943+0000 7fe1e9000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1e40d0a70 0x7fe1e409a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:17.944 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.943+0000 7fe1e9000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1e415e8a0 0x7fe1e4099940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:17.945 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.943+0000 7fe1e9000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e406fef0 con 0x7fe1e406dbb0 2024-04-20T10:22:17.945 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.943+0000 7fe1e9000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e406fd70 con 0x7fe1e40d0a70 2024-04-20T10:22:17.945 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.943+0000 7fe1e9000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe1e40adbc0 con 0x7fe1e415e8a0 2024-04-20T10:22:17.945 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.943+0000 7fe1e1e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1e406dbb0 0x7fe1e4146c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:17.945 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.944+0000 7fe1e1e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1e406dbb0 0x7fe1e4146c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54942/0 (socket says 172.21.15.121:54942) 2024-04-20T10:22:17.945 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.944+0000 7fe1e1e00640 1 -- 172.21.15.121:0/1739981693 learned_addr learned my addr 172.21.15.121:0/1739981693 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:17.945 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.944+0000 7fe1e2800640 1 --2- 172.21.15.121:0/1739981693 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1e415e8a0 0x7fe1e4099940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:17.945 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.944+0000 7fe1e1e00640 1 -- 172.21.15.121:0/1739981693 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1e415e8a0 msgr2=0x7fe1e4099940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:17.945 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.944+0000 7fe1e1e00640 1 --2- 172.21.15.121:0/1739981693 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1e415e8a0 0x7fe1e4099940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:17.945 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.944+0000 7fe1e1e00640 1 -- 172.21.15.121:0/1739981693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1e40d0a70 msgr2=0x7fe1e409a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:17.945 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.944+0000 7fe1d9400640 1 --2- 172.21.15.121:0/1739981693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1e40d0a70 0x7fe1e409a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:17.945 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.944+0000 7fe1e1e00640 1 --2- 172.21.15.121:0/1739981693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1e40d0a70 0x7fe1e409a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:17.946 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.944+0000 7fe1e1e00640 1 -- 172.21.15.121:0/1739981693 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e4099e80 con 0x7fe1e406dbb0 2024-04-20T10:22:17.947 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.944+0000 7fe1e1e00640 1 --2- 172.21.15.121:0/1739981693 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1e406dbb0 0x7fe1e4146c00 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe1d4000c00 tx=0x7fe1d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:17.947 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.945+0000 7fe1daa00640 1 -- 172.21.15.121:0/1739981693 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d4040070 con 0x7fe1e406dbb0 2024-04-20T10:22:17.947 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.945+0000 7fe1daa00640 1 -- 172.21.15.121:0/1739981693 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1d403a910 con 0x7fe1e406dbb0 2024-04-20T10:22:17.947 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.945+0000 7fe1e9000640 1 -- 172.21.15.121:0/1739981693 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1e409c900 con 0x7fe1e406dbb0 2024-04-20T10:22:17.947 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.945+0000 7fe1daa00640 1 -- 172.21.15.121:0/1739981693 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d4035040 con 0x7fe1e406dbb0 2024-04-20T10:22:17.947 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.945+0000 7fe1e9000640 1 -- 172.21.15.121:0/1739981693 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1e409aba0 con 0x7fe1e406dbb0 2024-04-20T10:22:17.948 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.946+0000 7fe1e9000640 1 -- 172.21.15.121:0/1739981693 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1e4068980 con 0x7fe1e406dbb0 2024-04-20T10:22:17.948 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.947+0000 7fe1daa00640 1 -- 172.21.15.121:0/1739981693 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe1d404b060 con 0x7fe1e406dbb0 2024-04-20T10:22:17.948 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.948+0000 7fe1daa00640 1 --2- 172.21.15.121:0/1739981693 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe1c006c8f0 0x7fe1c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:17.948 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.948+0000 7fe1daa00640 1 -- 172.21.15.121:0/1739981693 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fe1d40ad6f0 con 0x7fe1e406dbb0 2024-04-20T10:22:17.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.948+0000 7fe1d9400640 1 --2- 172.21.15.121:0/1739981693 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe1c006c8f0 0x7fe1c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:17.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.949+0000 7fe1d9400640 1 --2- 172.21.15.121:0/1739981693 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe1c006c8f0 0x7fe1c006edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe1e40cfaf0 tx=0x7fe1cc009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:17.953 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:17.953+0000 7fe1daa00640 1 -- 172.21.15.121:0/1739981693 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe1e4099e80 con 0x7fe1e406dbb0 2024-04-20T10:22:18.140 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.139+0000 7fe1e9000640 1 -- 172.21.15.121:0/1739981693 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe1e406d160 con 0x7fe1e406dbb0 2024-04-20T10:22:18.141 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.140+0000 7fe1daa00640 1 -- 172.21.15.121:0/1739981693 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fe1d407b720 con 0x7fe1e406dbb0 2024-04-20T10:22:18.141 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2024-04-20T10:22:18.144 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.143+0000 7fe1e9000640 1 -- 172.21.15.121:0/1739981693 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe1c006c8f0 msgr2=0x7fe1c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:18.144 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.143+0000 7fe1e9000640 1 --2- 172.21.15.121:0/1739981693 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe1c006c8f0 0x7fe1c006edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe1e40cfaf0 tx=0x7fe1cc009040 comp rx=0 tx=0).stop 2024-04-20T10:22:18.144 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.143+0000 7fe1e9000640 1 -- 172.21.15.121:0/1739981693 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1e406dbb0 msgr2=0x7fe1e4146c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:18.144 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.143+0000 7fe1e9000640 1 --2- 172.21.15.121:0/1739981693 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1e406dbb0 0x7fe1e4146c00 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fe1d4000c00 tx=0x7fe1d40046c0 comp rx=0 tx=0).stop 2024-04-20T10:22:18.144 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.143+0000 7fe1e9000640 1 -- 172.21.15.121:0/1739981693 shutdown_connections 2024-04-20T10:22:18.144 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.143+0000 7fe1e9000640 1 --2- 172.21.15.121:0/1739981693 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe1e415e8a0 0x7fe1e4099940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:18.144 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.143+0000 7fe1e9000640 1 --2- 172.21.15.121:0/1739981693 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe1c006c8f0 0x7fe1c006edb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:18.144 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.143+0000 7fe1e9000640 1 --2- 172.21.15.121:0/1739981693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1e40d0a70 0x7fe1e409a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:18.144 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.143+0000 7fe1e9000640 1 --2- 172.21.15.121:0/1739981693 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe1e406dbb0 0x7fe1e4146c00 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:18.144 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.143+0000 7fe1e9000640 1 -- 172.21.15.121:0/1739981693 >> 172.21.15.121:0/1739981693 conn(0x7fe1e40b3350 msgr2=0x7fe1e415d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:18.144 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.143+0000 7fe1e9000640 1 -- 172.21.15.121:0/1739981693 shutdown_connections 2024-04-20T10:22:18.144 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.143+0000 7fe1e9000640 1 -- 172.21.15.121:0/1739981693 wait complete. 2024-04-20T10:22:18.163 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T10:22:18.354 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.353+0000 7fc02f200640 1 Processor -- start 2024-04-20T10:22:18.354 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.354+0000 7fc02f200640 1 -- start start 2024-04-20T10:22:18.355 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.355+0000 7fc02f200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc02815b1f0 0x7fc02815f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:18.355 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.355+0000 7fc02f200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc0280d0010 0x7fc0280adc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:18.356 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.355+0000 7fc02f200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc02815b5d0 con 0x7fc0280ae140 2024-04-20T10:22:18.356 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.355+0000 7fc02f200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fc02806e730 con 0x7fc02815b1f0 2024-04-20T10:22:18.356 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.355+0000 7fc02f200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fc0280d05d0 con 0x7fc0280d0010 2024-04-20T10:22:18.358 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.355+0000 7fc02e800640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fc0280ae140 0x7fc02806ca10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:53384/0 (socket says 172.21.15.121:53384) 2024-04-20T10:22:18.358 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.355+0000 7fc02e800640 1 -- 172.21.15.121:0/3015435445 learned_addr learned my addr 172.21.15.121:0/3015435445 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:18.358 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.356+0000 7fc02d400640 1 --2- 172.21.15.121:0/3015435445 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc0280d0010 0x7fc0280adc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:18.358 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.356+0000 7fc02de00640 1 --2- 172.21.15.121:0/3015435445 >> v2:172.21.15.123:3300/0 conn(0x7fc02815b1f0 0x7fc02815f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:18.358 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.356+0000 7fc02ca00640 1 -- 172.21.15.121:0/3015435445 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1997321460 0 0) 0x7fc02815b5d0 con 0x7fc0280ae140 2024-04-20T10:22:18.358 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.356+0000 7fc02ca00640 1 -- 172.21.15.121:0/3015435445 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc010005730 con 0x7fc0280ae140 2024-04-20T10:22:18.358 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.356+0000 7fc02ca00640 1 -- 172.21.15.121:0/3015435445 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1990721403 0 0) 0x7fc010005730 con 0x7fc0280ae140 2024-04-20T10:22:18.358 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.356+0000 7fc02ca00640 1 -- 172.21.15.121:0/3015435445 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc0280d0010 msgr2=0x7fc0280adc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:18.358 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.356+0000 7fc02ca00640 1 --2- 172.21.15.121:0/3015435445 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc0280d0010 0x7fc0280adc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:18.358 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.356+0000 7fc02ca00640 1 -- 172.21.15.121:0/3015435445 >> v2:172.21.15.123:3300/0 conn(0x7fc02815b1f0 msgr2=0x7fc02815f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:18.359 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.356+0000 7fc02ca00640 1 --2- 172.21.15.121:0/3015435445 >> v2:172.21.15.123:3300/0 conn(0x7fc02815b1f0 0x7fc02815f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:18.359 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.356+0000 7fc02ca00640 1 -- 172.21.15.121:0/3015435445 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc02815fc70 con 0x7fc0280ae140 2024-04-20T10:22:18.359 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.356+0000 7fc02ca00640 1 -- 172.21.15.121:0/3015435445 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7fc0240019f0 con 0x7fc0280ae140 2024-04-20T10:22:18.359 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.356+0000 7fc02d400640 1 --2- 172.21.15.121:0/3015435445 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc0280d0010 0x7fc0280adc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:22:18.359 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.356+0000 7fc02ca00640 1 -- 172.21.15.121:0/3015435445 >> v1:172.21.15.121:6789/0 conn(0x7fc0280ae140 legacy=0x7fc02806ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:18.359 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.356+0000 7fc02ca00640 1 --2- 172.21.15.121:0/3015435445 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc0280d0010 0x7fc010005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:18.359 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.356+0000 7fc02ca00640 1 --2- 172.21.15.121:0/3015435445 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc010006100 0x7fc010006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:18.359 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.356+0000 7fc02ca00640 1 --2- 172.21.15.121:0/3015435445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc010006f50 0x7fc0100073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:18.359 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.356+0000 7fc02ca00640 1 -- 172.21.15.121:0/3015435445 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc02806e730 con 0x7fc010006100 2024-04-20T10:22:18.359 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.356+0000 7fc02ca00640 1 -- 172.21.15.121:0/3015435445 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc0280d05d0 con 0x7fc010006f50 2024-04-20T10:22:18.359 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.356+0000 7fc02ca00640 1 -- 172.21.15.121:0/3015435445 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fc01000dc00 con 0x7fc0280d0010 2024-04-20T10:22:18.359 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.357+0000 7fc02d400640 1 --2- 172.21.15.121:0/3015435445 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc010006100 0x7fc010006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:18.359 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.357+0000 7fc02de00640 1 --2- 172.21.15.121:0/3015435445 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc0280d0010 0x7fc010005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:18.359 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.357+0000 7fc02e800640 1 --2- 172.21.15.121:0/3015435445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc010006f50 0x7fc0100073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:18.360 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.357+0000 7fc02d400640 1 -- 172.21.15.121:0/3015435445 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc0280d0010 msgr2=0x7fc010005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:18.360 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.357+0000 7fc02d400640 1 --2- 172.21.15.121:0/3015435445 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc0280d0010 0x7fc010005bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:18.360 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.357+0000 7fc02d400640 1 -- 172.21.15.121:0/3015435445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc010006f50 msgr2=0x7fc0100073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:18.360 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.357+0000 7fc02d400640 1 --2- 172.21.15.121:0/3015435445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc010006f50 0x7fc0100073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:18.360 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.357+0000 7fc02d400640 1 -- 172.21.15.121:0/3015435445 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc010005730 con 0x7fc010006100 2024-04-20T10:22:18.361 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.358+0000 7fc02d400640 1 --2- 172.21.15.121:0/3015435445 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc010006100 0x7fc010006a10 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc0180098a0 tx=0x7fc010007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee51356b71d33a2a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:18.361 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.359+0000 7fc02ca00640 1 -- 172.21.15.121:0/3015435445 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc01800f040 con 0x7fc010006100 2024-04-20T10:22:18.361 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.359+0000 7fc02ca00640 1 -- 172.21.15.121:0/3015435445 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc018004230 con 0x7fc010006100 2024-04-20T10:22:18.361 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.359+0000 7fc02ca00640 1 -- 172.21.15.121:0/3015435445 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc018003610 con 0x7fc010006100 2024-04-20T10:22:18.361 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.359+0000 7fc02f200640 1 -- 172.21.15.121:0/3015435445 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc010006100 msgr2=0x7fc010006a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:18.361 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.359+0000 7fc02f200640 1 --2- 172.21.15.121:0/3015435445 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc010006100 0x7fc010006a10 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc0180098a0 tx=0x7fc010007a40 comp rx=0 tx=0).stop 2024-04-20T10:22:18.361 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.359+0000 7fc02e800640 1 -- 172.21.15.121:0/3015435445 reap_dead start 2024-04-20T10:22:18.361 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.359+0000 7fc02f200640 1 -- 172.21.15.121:0/3015435445 shutdown_connections 2024-04-20T10:22:18.361 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.359+0000 7fc02f200640 1 -- 172.21.15.121:0/3015435445 >> 172.21.15.121:0/3015435445 conn(0x7fc0280b5270 msgr2=0x7fc0280b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:18.362 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.359+0000 7fc02f200640 1 -- 172.21.15.121:0/3015435445 shutdown_connections 2024-04-20T10:22:18.362 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.359+0000 7fc02f200640 1 -- 172.21.15.121:0/3015435445 wait complete. 2024-04-20T10:22:18.364 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.361+0000 7fc02f200640 1 Processor -- start 2024-04-20T10:22:18.364 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.361+0000 7fc02f200640 1 -- start start 2024-04-20T10:22:18.364 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.362+0000 7fc02f200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc010006100 0x7fc02809c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:18.364 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.362+0000 7fc02f200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc0281f4bb0 0x7fc028159380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:18.364 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.362+0000 7fc02f200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0281598c0 0x7fc02815e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:18.364 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.362+0000 7fc02f200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc0280d0e70 con 0x7fc0281598c0 2024-04-20T10:22:18.364 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.362+0000 7fc02f200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc0280d0b90 con 0x7fc010006100 2024-04-20T10:22:18.364 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.362+0000 7fc02f200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fc02806d120 con 0x7fc0281f4bb0 2024-04-20T10:22:18.364 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.362+0000 7fc02e800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0281598c0 0x7fc02815e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:18.364 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.362+0000 7fc02e800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0281598c0 0x7fc02815e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54968/0 (socket says 172.21.15.121:54968) 2024-04-20T10:22:18.364 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.362+0000 7fc02e800640 1 -- 172.21.15.121:0/4150454147 learned_addr learned my addr 172.21.15.121:0/4150454147 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:18.365 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.363+0000 7fc02de00640 1 --2- 172.21.15.121:0/4150454147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc010006100 0x7fc02809c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:18.365 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.363+0000 7fc02d400640 1 --2- 172.21.15.121:0/4150454147 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc0281f4bb0 0x7fc028159380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:18.365 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.363+0000 7fc02e800640 1 -- 172.21.15.121:0/4150454147 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc0281f4bb0 msgr2=0x7fc028159380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:18.365 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.363+0000 7fc02e800640 1 --2- 172.21.15.121:0/4150454147 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc0281f4bb0 0x7fc028159380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:18.365 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.363+0000 7fc02e800640 1 -- 172.21.15.121:0/4150454147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc010006100 msgr2=0x7fc02809c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:18.365 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.363+0000 7fc02e800640 1 --2- 172.21.15.121:0/4150454147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc010006100 0x7fc02809c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:18.365 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.363+0000 7fc02e800640 1 -- 172.21.15.121:0/4150454147 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc02809c280 con 0x7fc0281598c0 2024-04-20T10:22:18.365 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.363+0000 7fc02de00640 1 --2- 172.21.15.121:0/4150454147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc010006100 0x7fc02809c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:18.365 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.363+0000 7fc02e800640 1 --2- 172.21.15.121:0/4150454147 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0281598c0 0x7fc02815e600 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc024000f80 tx=0x7fc02400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:18.365 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.363+0000 7fc022000640 1 -- 172.21.15.121:0/4150454147 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc02400ec10 con 0x7fc0281598c0 2024-04-20T10:22:18.365 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.363+0000 7fc022000640 1 -- 172.21.15.121:0/4150454147 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc024018070 con 0x7fc0281598c0 2024-04-20T10:22:18.365 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.363+0000 7fc02f200640 1 -- 172.21.15.121:0/4150454147 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0281f5030 con 0x7fc0281598c0 2024-04-20T10:22:18.365 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.363+0000 7fc022000640 1 -- 172.21.15.121:0/4150454147 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc024013420 con 0x7fc0281598c0 2024-04-20T10:22:18.365 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.363+0000 7fc02f200640 1 -- 172.21.15.121:0/4150454147 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc02809d0b0 con 0x7fc0281598c0 2024-04-20T10:22:18.366 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.364+0000 7fc02f200640 1 -- 172.21.15.121:0/4150454147 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc02809de50 con 0x7fc0281598c0 2024-04-20T10:22:18.366 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.365+0000 7fc022000640 1 -- 172.21.15.121:0/4150454147 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc0240135c0 con 0x7fc0281598c0 2024-04-20T10:22:18.367 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.366+0000 7fc022000640 1 --2- 172.21.15.121:0/4150454147 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fc00c06c920 0x7fc00c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:18.367 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.366+0000 7fc022000640 1 -- 172.21.15.121:0/4150454147 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc02408c3b0 con 0x7fc0281598c0 2024-04-20T10:22:18.367 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.367+0000 7fc02de00640 1 --2- 172.21.15.121:0/4150454147 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fc00c06c920 0x7fc00c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:18.367 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.367+0000 7fc02de00640 1 --2- 172.21.15.121:0/4150454147 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fc00c06c920 0x7fc00c06ede0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc02815f480 tx=0x7fc01c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:18.371 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.371+0000 7fc022000640 1 -- 172.21.15.121:0/4150454147 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc02809de50 con 0x7fc0281598c0 2024-04-20T10:22:18.567 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.566+0000 7fc02f200640 1 -- 172.21.15.121:0/4150454147 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc028075b30 con 0x7fc0281598c0 2024-04-20T10:22:18.568 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.567+0000 7fc022000640 1 -- 172.21.15.121:0/4150454147 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fc02405a340 con 0x7fc0281598c0 2024-04-20T10:22:18.568 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 32 2024-04-20T10:22:18.571 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.570+0000 7fc02f200640 1 -- 172.21.15.121:0/4150454147 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fc00c06c920 msgr2=0x7fc00c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:18.571 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.570+0000 7fc02f200640 1 --2- 172.21.15.121:0/4150454147 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fc00c06c920 0x7fc00c06ede0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc02815f480 tx=0x7fc01c008040 comp rx=0 tx=0).stop 2024-04-20T10:22:18.571 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.570+0000 7fc02f200640 1 -- 172.21.15.121:0/4150454147 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0281598c0 msgr2=0x7fc02815e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:18.571 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.570+0000 7fc02f200640 1 --2- 172.21.15.121:0/4150454147 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0281598c0 0x7fc02815e600 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc024000f80 tx=0x7fc02400e530 comp rx=0 tx=0).stop 2024-04-20T10:22:18.571 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.570+0000 7fc02f200640 1 -- 172.21.15.121:0/4150454147 shutdown_connections 2024-04-20T10:22:18.571 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.570+0000 7fc02f200640 1 --2- 172.21.15.121:0/4150454147 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc0281598c0 0x7fc02815e600 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:18.571 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.570+0000 7fc02f200640 1 --2- 172.21.15.121:0/4150454147 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc0281f4bb0 0x7fc028159380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:18.571 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.570+0000 7fc02f200640 1 --2- 172.21.15.121:0/4150454147 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fc00c06c920 0x7fc00c06ede0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:18.571 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.570+0000 7fc02f200640 1 --2- 172.21.15.121:0/4150454147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc010006100 0x7fc02809c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:18.571 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.570+0000 7fc02f200640 1 -- 172.21.15.121:0/4150454147 >> 172.21.15.121:0/4150454147 conn(0x7fc0280b5270 msgr2=0x7fc02806c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:18.571 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.570+0000 7fc02f200640 1 -- 172.21.15.121:0/4150454147 shutdown_connections 2024-04-20T10:22:18.572 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.570+0000 7fc02f200640 1 -- 172.21.15.121:0/4150454147 wait complete. 2024-04-20T10:22:18.589 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T10:22:18.784 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.783+0000 7f717ec00640 1 Processor -- start 2024-04-20T10:22:18.784 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.784+0000 7f717ec00640 1 -- start start 2024-04-20T10:22:18.786 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.785+0000 7f717ec00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7178072190 0x7f71780735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:18.786 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.785+0000 7f717ec00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f717815cc90 con 0x7f717815a1f0 2024-04-20T10:22:18.786 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.785+0000 7f717ec00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f717815cf70 con 0x7f7178159e10 2024-04-20T10:22:18.786 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.785+0000 7f717ec00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f7178072900 con 0x7f7178072190 2024-04-20T10:22:18.786 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.785+0000 7f717ce00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f717815a1f0 0x7f7178072ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:53400/0 (socket says 172.21.15.121:53400) 2024-04-20T10:22:18.786 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.785+0000 7f717ce00640 1 -- 172.21.15.121:0/3085101288 learned_addr learned my addr 172.21.15.121:0/3085101288 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:18.788 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.786+0000 7f717e200640 1 --2- 172.21.15.121:0/3085101288 >> v2:172.21.15.123:3300/0 conn(0x7f7178072190 0x7f71780735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:18.788 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.786+0000 7f7177e00640 1 -- 172.21.15.121:0/3085101288 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 423110925 0 0) 0x7f717815cc90 con 0x7f717815a1f0 2024-04-20T10:22:18.788 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.786+0000 7f7177e00640 1 -- 172.21.15.121:0/3085101288 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f715c003520 con 0x7f717815a1f0 2024-04-20T10:22:18.788 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.786+0000 7f7177e00640 1 -- 172.21.15.121:0/3085101288 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 938346222 0 0) 0x7f715c003520 con 0x7f717815a1f0 2024-04-20T10:22:18.788 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.786+0000 7f7177e00640 1 -- 172.21.15.121:0/3085101288 >> v2:172.21.15.123:3300/0 conn(0x7f7178072190 msgr2=0x7f71780735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:18.788 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.786+0000 7f7177e00640 1 --2- 172.21.15.121:0/3085101288 >> v2:172.21.15.123:3300/0 conn(0x7f7178072190 0x7f71780735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:18.788 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.786+0000 7f7177e00640 1 -- 172.21.15.121:0/3085101288 >> v1:172.21.15.123:6789/0 conn(0x7f7178159e10 legacy=0x7f7178071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:18.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.786+0000 7f7177e00640 1 -- 172.21.15.121:0/3085101288 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f717815d250 con 0x7f717815a1f0 2024-04-20T10:22:18.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.786+0000 7f7177e00640 1 -- 172.21.15.121:0/3085101288 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f7164001670 con 0x7f717815a1f0 2024-04-20T10:22:18.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.786+0000 7f7177e00640 1 -- 172.21.15.121:0/3085101288 >> v1:172.21.15.121:6789/0 conn(0x7f717815a1f0 legacy=0x7f7178072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:18.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.786+0000 7f7177e00640 1 --2- 172.21.15.121:0/3085101288 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f715c0039b0 0x7f715c003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:18.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.786+0000 7f7177e00640 1 --2- 172.21.15.121:0/3085101288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f715c004800 0x7f715c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:18.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.786+0000 7f7177e00640 1 --2- 172.21.15.121:0/3085101288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f715c0051c0 0x7f715c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:18.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.787+0000 7f7177e00640 1 -- 172.21.15.121:0/3085101288 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f715c00c010 con 0x7f715c004800 2024-04-20T10:22:18.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.787+0000 7f7177e00640 1 -- 172.21.15.121:0/3085101288 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f715c00c1c0 con 0x7f715c0051c0 2024-04-20T10:22:18.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.787+0000 7f7177e00640 1 -- 172.21.15.121:0/3085101288 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f715c00c3c0 con 0x7f715c0039b0 2024-04-20T10:22:18.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.787+0000 7f717ce00640 1 --2- 172.21.15.121:0/3085101288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f715c004800 0x7f715c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:18.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.787+0000 7f717d800640 1 --2- 172.21.15.121:0/3085101288 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f715c0039b0 0x7f715c003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:18.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.787+0000 7f717e200640 1 --2- 172.21.15.121:0/3085101288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f715c0051c0 0x7f715c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:18.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.787+0000 7f717ce00640 1 -- 172.21.15.121:0/3085101288 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f715c0039b0 msgr2=0x7f715c003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:18.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.787+0000 7f717ce00640 1 --2- 172.21.15.121:0/3085101288 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f715c0039b0 0x7f715c003db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:18.790 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.787+0000 7f717ce00640 1 -- 172.21.15.121:0/3085101288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f715c0051c0 msgr2=0x7f715c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:18.790 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.787+0000 7f717ce00640 1 --2- 172.21.15.121:0/3085101288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f715c0051c0 0x7f715c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:18.790 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.787+0000 7f717ce00640 1 -- 172.21.15.121:0/3085101288 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f715c003520 con 0x7f715c004800 2024-04-20T10:22:18.790 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.787+0000 7f717e200640 1 --2- 172.21.15.121:0/3085101288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f715c0051c0 0x7f715c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:18.790 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.787+0000 7f717e200640 1 -- 172.21.15.121:0/3085101288 reap_dead start 2024-04-20T10:22:18.791 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.788+0000 7f717ce00640 1 --2- 172.21.15.121:0/3085101288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f715c004800 0x7f715c004c80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f716400c210 tx=0x7f715c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e24e85439c6be5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:18.791 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.789+0000 7f7177e00640 1 -- 172.21.15.121:0/3085101288 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7164007d30 con 0x7f715c004800 2024-04-20T10:22:18.791 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.789+0000 7f7177e00640 1 -- 172.21.15.121:0/3085101288 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7164030070 con 0x7f715c004800 2024-04-20T10:22:18.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.789+0000 7f7177e00640 1 -- 172.21.15.121:0/3085101288 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7164005d70 con 0x7f715c004800 2024-04-20T10:22:18.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.789+0000 7f717ec00640 1 -- 172.21.15.121:0/3085101288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f715c004800 msgr2=0x7f715c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:18.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.789+0000 7f717ec00640 1 --2- 172.21.15.121:0/3085101288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f715c004800 0x7f715c004c80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f716400c210 tx=0x7f715c005830 comp rx=0 tx=0).stop 2024-04-20T10:22:18.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.789+0000 7f717ec00640 1 -- 172.21.15.121:0/3085101288 shutdown_connections 2024-04-20T10:22:18.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.789+0000 7f717ec00640 1 --2- 172.21.15.121:0/3085101288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f715c004800 0x7f715c004c80 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:18.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.789+0000 7f717ec00640 1 -- 172.21.15.121:0/3085101288 >> 172.21.15.121:0/3085101288 conn(0x7f71780d0040 msgr2=0x7f71780d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:18.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.789+0000 7f717ec00640 1 -- 172.21.15.121:0/3085101288 shutdown_connections 2024-04-20T10:22:18.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.789+0000 7f717ec00640 1 -- 172.21.15.121:0/3085101288 wait complete. 2024-04-20T10:22:18.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.791+0000 7f717ec00640 1 Processor -- start 2024-04-20T10:22:18.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.791+0000 7f717ec00640 1 -- start start 2024-04-20T10:22:18.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.792+0000 7f717ec00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f715c004800 0x7f71781f5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:18.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.792+0000 7f717ec00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f717815e8a0 0x7f71781586c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:18.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.792+0000 7f717ec00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f717814a660 0x7f717814aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:18.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.792+0000 7f717ec00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f717815d810 con 0x7f717814a660 2024-04-20T10:22:18.795 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.792+0000 7f717ec00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f717815ca40 con 0x7f715c004800 2024-04-20T10:22:18.795 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.792+0000 7f717ec00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f717806da90 con 0x7f717815e8a0 2024-04-20T10:22:18.795 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.792+0000 7f717e200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f717814a660 0x7f717814aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:18.795 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.792+0000 7f717e200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f717814a660 0x7f717814aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:54998/0 (socket says 172.21.15.121:54998) 2024-04-20T10:22:18.795 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.792+0000 7f717e200640 1 -- 172.21.15.121:0/932683810 learned_addr learned my addr 172.21.15.121:0/932683810 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:18.795 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.793+0000 7f717ce00640 1 --2- 172.21.15.121:0/932683810 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f717815e8a0 0x7f71781586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:18.795 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.793+0000 7f717e200640 1 -- 172.21.15.121:0/932683810 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f717815e8a0 msgr2=0x7f71781586c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:18.795 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.793+0000 7f717e200640 1 --2- 172.21.15.121:0/932683810 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f717815e8a0 0x7f71781586c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:18.795 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.793+0000 7f717e200640 1 -- 172.21.15.121:0/932683810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f715c004800 msgr2=0x7f71781f5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:18.795 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.793+0000 7f717e200640 1 --2- 172.21.15.121:0/932683810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f715c004800 0x7f71781f5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:18.795 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.793+0000 7f717e200640 1 -- 172.21.15.121:0/932683810 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f717814b020 con 0x7f717814a660 2024-04-20T10:22:18.795 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.793+0000 7f717e200640 1 --2- 172.21.15.121:0/932683810 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f717814a660 0x7f717814aae0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f715c004780 tx=0x7f717000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:18.795 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.793+0000 7f7175600640 1 -- 172.21.15.121:0/932683810 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f717000fe50 con 0x7f717814a660 2024-04-20T10:22:18.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.793+0000 7f7175600640 1 -- 172.21.15.121:0/932683810 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7170003990 con 0x7f717814a660 2024-04-20T10:22:18.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.793+0000 7f717ec00640 1 -- 172.21.15.121:0/932683810 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f717809ac00 con 0x7f717814a660 2024-04-20T10:22:18.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.793+0000 7f7175600640 1 -- 172.21.15.121:0/932683810 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7170003d80 con 0x7f717814a660 2024-04-20T10:22:18.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.794+0000 7f717ec00640 1 -- 172.21.15.121:0/932683810 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f717809b1c0 con 0x7f717814a660 2024-04-20T10:22:18.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.795+0000 7f717ec00640 1 -- 172.21.15.121:0/932683810 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7178075b60 con 0x7f717814a660 2024-04-20T10:22:18.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.796+0000 7f7175600640 1 -- 172.21.15.121:0/932683810 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f717001d030 con 0x7f717814a660 2024-04-20T10:22:18.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.796+0000 7f7175600640 1 --2- 172.21.15.121:0/932683810 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f715806c8f0 0x7f715806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:18.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.797+0000 7f7175600640 1 -- 172.21.15.121:0/932683810 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f717008c8b0 con 0x7f717814a660 2024-04-20T10:22:18.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.797+0000 7f717d800640 1 --2- 172.21.15.121:0/932683810 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f715806c8f0 0x7f715806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:18.798 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.797+0000 7f717d800640 1 --2- 172.21.15.121:0/932683810 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f715806c8f0 0x7f715806edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7168003d10 tx=0x7f7168009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:18.801 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:18.801+0000 7f7175600640 1 -- 172.21.15.121:0/932683810 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7170024020 con 0x7f717814a660 2024-04-20T10:22:19.020 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.019+0000 7f717ec00640 1 -- 172.21.15.121:0/932683810 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7178072830 con 0x7f717814a660 2024-04-20T10:22:19.020 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.020+0000 7f7175600640 1 -- 172.21.15.121:0/932683810 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f717005a970 con 0x7f717814a660 2024-04-20T10:22:19.021 INFO:teuthology.orchestra.run.smithi121.stderr:listed 0 entries 2024-04-20T10:22:19.023 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.022+0000 7f717ec00640 1 -- 172.21.15.121:0/932683810 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f715806c8f0 msgr2=0x7f715806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:19.023 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.022+0000 7f717ec00640 1 --2- 172.21.15.121:0/932683810 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f715806c8f0 0x7f715806edb0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7168003d10 tx=0x7f7168009040 comp rx=0 tx=0).stop 2024-04-20T10:22:19.023 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.022+0000 7f717ec00640 1 -- 172.21.15.121:0/932683810 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f717814a660 msgr2=0x7f717814aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:19.023 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.022+0000 7f717ec00640 1 --2- 172.21.15.121:0/932683810 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f717814a660 0x7f717814aae0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f715c004780 tx=0x7f717000f520 comp rx=0 tx=0).stop 2024-04-20T10:22:19.024 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.023+0000 7f717ec00640 1 -- 172.21.15.121:0/932683810 shutdown_connections 2024-04-20T10:22:19.024 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.023+0000 7f717ec00640 1 --2- 172.21.15.121:0/932683810 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f717814a660 0x7f717814aae0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:19.024 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.023+0000 7f717ec00640 1 --2- 172.21.15.121:0/932683810 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f717815e8a0 0x7f71781586c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:19.024 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.023+0000 7f717ec00640 1 --2- 172.21.15.121:0/932683810 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f715806c8f0 0x7f715806edb0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:19.024 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.023+0000 7f717ec00640 1 --2- 172.21.15.121:0/932683810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f715c004800 0x7f71781f5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:19.024 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.023+0000 7f717ec00640 1 -- 172.21.15.121:0/932683810 >> 172.21.15.121:0/932683810 conn(0x7f71780d0040 msgr2=0x7f7178071a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:19.024 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.023+0000 7f717ec00640 1 -- 172.21.15.121:0/932683810 shutdown_connections 2024-04-20T10:22:19.024 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.023+0000 7f717ec00640 1 -- 172.21.15.121:0/932683810 wait complete. 2024-04-20T10:22:19.042 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T10:22:19.042 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:22:19.101 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:22:19.301 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.301+0000 7fbcfd200640 1 Processor -- start 2024-04-20T10:22:19.302 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.301+0000 7fbcfd200640 1 -- start start 2024-04-20T10:22:19.303 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.302+0000 7fbcfd200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbcf806dcb0 0x7fbcf8071b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:19.304 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.302+0000 7fbcfd200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fbcf8072090 0x7fbcf80d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:19.304 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.302+0000 7fbcfd200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcf80ade40 con 0x7fbcf815b3b0 2024-04-20T10:22:19.304 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.302+0000 7fbcfd200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fbcf806f1b0 con 0x7fbcf8072090 2024-04-20T10:22:19.304 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.302+0000 7fbcfd200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fbcf80ae0e0 con 0x7fbcf806dcb0 2024-04-20T10:22:19.304 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.303+0000 7fbcf6800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fbcf8072090 0x7fbcf80d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:19.304 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.303+0000 7fbcf6800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fbcf8072090 0x7fbcf80d0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55000/0 (socket says 172.21.15.121:55000) 2024-04-20T10:22:19.304 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.303+0000 7fbcf6800640 1 -- 172.21.15.121:0/1831332856 learned_addr learned my addr 172.21.15.121:0/1831332856 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:19.304 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.303+0000 7fbcf5400640 1 --2- 172.21.15.121:0/1831332856 >> v2:172.21.15.123:3300/0 conn(0x7fbcf806dcb0 0x7fbcf8071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:19.304 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.303+0000 7fbcf6800640 1 -- 172.21.15.121:0/1831332856 >> v2:172.21.15.123:3300/0 conn(0x7fbcf806dcb0 msgr2=0x7fbcf8071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:19.304 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.303+0000 7fbcf6800640 1 --2- 172.21.15.121:0/1831332856 >> v2:172.21.15.123:3300/0 conn(0x7fbcf806dcb0 0x7fbcf8071b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:19.304 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.303+0000 7fbcf6800640 1 -- 172.21.15.121:0/1831332856 >> v1:172.21.15.123:6789/0 conn(0x7fbcf815b3b0 legacy=0x7fbcf806eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:19.304 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.303+0000 7fbcf6800640 1 -- 172.21.15.121:0/1831332856 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf815b790 con 0x7fbcf8072090 2024-04-20T10:22:19.305 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.304+0000 7fbcf6800640 1 --2- 172.21.15.121:0/1831332856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbcf8072090 0x7fbcf80d0360 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fbcf80ad8d0 tx=0x7fbcf002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3bce826ff6acc03 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:19.305 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.305+0000 7fbcf4a00640 1 -- 172.21.15.121:0/1831332856 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf002ca80 con 0x7fbcf8072090 2024-04-20T10:22:19.305 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.305+0000 7fbcf4a00640 1 -- 172.21.15.121:0/1831332856 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcf002cc20 con 0x7fbcf8072090 2024-04-20T10:22:19.305 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.305+0000 7fbcf4a00640 1 -- 172.21.15.121:0/1831332856 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf00343f0 con 0x7fbcf8072090 2024-04-20T10:22:19.306 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.305+0000 7fbcfd200640 1 -- 172.21.15.121:0/1831332856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbcf8072090 msgr2=0x7fbcf80d0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:19.306 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.305+0000 7fbcfd200640 1 --2- 172.21.15.121:0/1831332856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbcf8072090 0x7fbcf80d0360 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fbcf80ad8d0 tx=0x7fbcf002bee0 comp rx=0 tx=0).stop 2024-04-20T10:22:19.306 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.305+0000 7fbcfd200640 1 -- 172.21.15.121:0/1831332856 shutdown_connections 2024-04-20T10:22:19.306 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.305+0000 7fbcfd200640 1 --2- 172.21.15.121:0/1831332856 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbcf8072090 0x7fbcf80d0360 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:19.306 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.305+0000 7fbcfd200640 1 --2- 172.21.15.121:0/1831332856 >> v2:172.21.15.123:3300/0 conn(0x7fbcf806dcb0 0x7fbcf8071b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:19.306 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.305+0000 7fbcfd200640 1 -- 172.21.15.121:0/1831332856 >> 172.21.15.121:0/1831332856 conn(0x7fbcf80b3350 msgr2=0x7fbcf80b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:19.306 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.305+0000 7fbcfd200640 1 -- 172.21.15.121:0/1831332856 shutdown_connections 2024-04-20T10:22:19.307 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.305+0000 7fbcfd200640 1 -- 172.21.15.121:0/1831332856 wait complete. 2024-04-20T10:22:19.307 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.307+0000 7fbcfd200640 1 Processor -- start 2024-04-20T10:22:19.308 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.307+0000 7fbcfd200640 1 -- start start 2024-04-20T10:22:19.308 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.308+0000 7fbcfd200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcf806dcb0 0x7fbcf815b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:19.309 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.308+0000 7fbcfd200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbcf815b3b0 0x7fbcf8152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:19.309 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.308+0000 7fbcfd200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbcf81f8420 0x7fbcf815efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:19.309 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.308+0000 7fbcfd200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf80ae940 con 0x7fbcf81f8420 2024-04-20T10:22:19.309 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.308+0000 7fbcfd200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf80ae6e0 con 0x7fbcf806dcb0 2024-04-20T10:22:19.309 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.308+0000 7fbcfd200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fbcf80aec20 con 0x7fbcf815b3b0 2024-04-20T10:22:19.309 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.308+0000 7fbcf6800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbcf81f8420 0x7fbcf815efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:19.309 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.308+0000 7fbcf6800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbcf81f8420 0x7fbcf815efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55004/0 (socket says 172.21.15.121:55004) 2024-04-20T10:22:19.309 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.308+0000 7fbcf6800640 1 -- 172.21.15.121:0/1316017284 learned_addr learned my addr 172.21.15.121:0/1316017284 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:19.310 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.309+0000 7fbcf5e00640 1 --2- 172.21.15.121:0/1316017284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcf806dcb0 0x7fbcf815b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:19.310 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.309+0000 7fbcf6800640 1 -- 172.21.15.121:0/1316017284 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbcf815b3b0 msgr2=0x7fbcf8152570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:19.310 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.309+0000 7fbcf6800640 1 --2- 172.21.15.121:0/1316017284 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbcf815b3b0 0x7fbcf8152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:19.311 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.309+0000 7fbcf6800640 1 -- 172.21.15.121:0/1316017284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcf806dcb0 msgr2=0x7fbcf815b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:19.311 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.309+0000 7fbcf6800640 1 --2- 172.21.15.121:0/1316017284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcf806dcb0 0x7fbcf815b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:19.311 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.309+0000 7fbcf6800640 1 -- 172.21.15.121:0/1316017284 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf81548e0 con 0x7fbcf81f8420 2024-04-20T10:22:19.311 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.309+0000 7fbcf6800640 1 --2- 172.21.15.121:0/1316017284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbcf81f8420 0x7fbcf815efa0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0036040 tx=0x7fbcf0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:19.311 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.309+0000 7fbcee000640 1 -- 172.21.15.121:0/1316017284 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0043020 con 0x7fbcf81f8420 2024-04-20T10:22:19.311 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.309+0000 7fbcee000640 1 -- 172.21.15.121:0/1316017284 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcf003e450 con 0x7fbcf81f8420 2024-04-20T10:22:19.311 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.309+0000 7fbcee000640 1 -- 172.21.15.121:0/1316017284 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf0009510 con 0x7fbcf81f8420 2024-04-20T10:22:19.311 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.309+0000 7fbcfd200640 1 -- 172.21.15.121:0/1316017284 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcf815d460 con 0x7fbcf81f8420 2024-04-20T10:22:19.311 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.310+0000 7fbcfd200640 1 -- 172.21.15.121:0/1316017284 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcf8154b80 con 0x7fbcf81f8420 2024-04-20T10:22:19.312 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.311+0000 7fbcfd200640 1 -- 172.21.15.121:0/1316017284 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcf8068980 con 0x7fbcf81f8420 2024-04-20T10:22:19.312 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.312+0000 7fbcee000640 1 -- 172.21.15.121:0/1316017284 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbcf0046020 con 0x7fbcf81f8420 2024-04-20T10:22:19.313 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.312+0000 7fbcee000640 1 --2- 172.21.15.121:0/1316017284 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fbcd806c8f0 0x7fbcd806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:19.313 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.313+0000 7fbcee000640 1 -- 172.21.15.121:0/1316017284 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fbcf00ae7f0 con 0x7fbcf81f8420 2024-04-20T10:22:19.313 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.313+0000 7fbcf5e00640 1 --2- 172.21.15.121:0/1316017284 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fbcd806c8f0 0x7fbcd806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:19.314 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.314+0000 7fbcf5e00640 1 --2- 172.21.15.121:0/1316017284 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fbcd806c8f0 0x7fbcd806edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbce8006fd0 tx=0x7fbce8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:19.318 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.317+0000 7fbcee000640 1 -- 172.21.15.121:0/1316017284 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbcf004d030 con 0x7fbcf81f8420 2024-04-20T10:22:19.504 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.503+0000 7fbcfd200640 1 -- 172.21.15.121:0/1316017284 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbcf806e210 con 0x7fbcf81f8420 2024-04-20T10:22:19.507 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.506+0000 7fbcee000640 1 -- 172.21.15.121:0/1316017284 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18940 (secure 0 0 0) 0x7fbcf007c820 con 0x7fbcf81f8420 2024-04-20T10:22:19.507 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:22:19.507 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":27,"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","created":"2024-04-20T10:20:38.076076+0000","modified":"2024-04-20T10:21:44.715898+0000","last_up_change":"2024-04-20T10:20:50.162700+0000","last_in_change":"2024-04-20T10:20:41.510382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:20:51.141365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:21:00.436937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:21:01.026957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:21:04.862693+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"629a7b8c-a40a-4a2a-a3cc-d9f357774310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6809","nonce":1044588835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6811","nonce":1044588835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6815","nonce":1044588835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6813","nonce":1044588835}]},"public_addr":"172.21.15.121:6809/1044588835","cluster_addr":"172.21.15.121:6811/1044588835","heartbeat_back_addr":"172.21.15.121:6815/1044588835","heartbeat_front_addr":"172.21.15.121:6813/1044588835","state":["exists","up"]},{"osd":1,"uuid":"f2570399-aa50-4ff3-bf5d-3d09207290db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6801","nonce":3505199624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6803","nonce":3505199624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6807","nonce":3505199624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6805","nonce":3505199624}]},"public_addr":"172.21.15.121:6801/3505199624","cluster_addr":"172.21.15.121:6803/3505199624","heartbeat_back_addr":"172.21.15.121:6807/3505199624","heartbeat_front_addr":"172.21.15.121:6805/3505199624","state":["exists","up"]},{"osd":2,"uuid":"50893807-dc30-49f4-b41f-739366626658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6817","nonce":3656724207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6819","nonce":3656724207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6827","nonce":3656724207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6822","nonce":3656724207}]},"public_addr":"172.21.15.121:6817/3656724207","cluster_addr":"172.21.15.121:6819/3656724207","heartbeat_back_addr":"172.21.15.121:6827/3656724207","heartbeat_front_addr":"172.21.15.121:6822/3656724207","state":["exists","up"]},{"osd":3,"uuid":"dadb01bc-5491-4ffc-8824-d48a057240b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6823","nonce":2232342832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6826","nonce":2232342832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6831","nonce":2232342832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6829","nonce":2232342832}]},"public_addr":"172.21.15.121:6823/2232342832","cluster_addr":"172.21.15.121:6826/2232342832","heartbeat_back_addr":"172.21.15.121:6831/2232342832","heartbeat_front_addr":"172.21.15.121:6829/2232342832","state":["exists","up"]},{"osd":4,"uuid":"b3f3b4a1-b375-4526-b69e-344ab24c2da1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6801","nonce":99353111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6803","nonce":99353111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6807","nonce":99353111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6805","nonce":99353111}]},"public_addr":"172.21.15.123:6801/99353111","cluster_addr":"172.21.15.123:6803/99353111","heartbeat_back_addr":"172.21.15.123:6807/99353111","heartbeat_front_addr":"172.21.15.123:6805/99353111","state":["exists","up"]},{"osd":5,"uuid":"2ee24a08-1624-4961-9642-d7b510282484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6818","nonce":2504374688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6822","nonce":2504374688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6830","nonce":2504374688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6826","nonce":2504374688}]},"public_addr":"172.21.15.123:6818/2504374688","cluster_addr":"172.21.15.123:6822/2504374688","heartbeat_back_addr":"172.21.15.123:6830/2504374688","heartbeat_front_addr":"172.21.15.123:6826/2504374688","state":["exists","up"]},{"osd":6,"uuid":"186118cc-6128-49de-acca-55260945c1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6819","nonce":4139299599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6823","nonce":4139299599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6829","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6831","nonce":4139299599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6827","nonce":4139299599}]},"public_addr":"172.21.15.123:6819/4139299599","cluster_addr":"172.21.15.123:6823/4139299599","heartbeat_back_addr":"172.21.15.123:6831/4139299599","heartbeat_front_addr":"172.21.15.123:6827/4139299599","state":["exists","up"]},{"osd":7,"uuid":"a00aac6e-4bd9-415a-a93d-ae85210865ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6809","nonce":4043981069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6811","nonce":4043981069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6815","nonce":4043981069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6813","nonce":4043981069}]},"public_addr":"172.21.15.123:6809/4043981069","cluster_addr":"172.21.15.123:6811/4043981069","heartbeat_back_addr":"172.21.15.123:6815/4043981069","heartbeat_front_addr":"172.21.15.123:6813/4043981069","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.904540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.958585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.616367+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.885244+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.103127+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.167248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.884348+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.862356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.23","mappings":[{"from":3,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:22:19.509 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.509+0000 7fbcfd200640 1 -- 172.21.15.121:0/1316017284 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fbcd806c8f0 msgr2=0x7fbcd806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:19.510 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.509+0000 7fbcfd200640 1 --2- 172.21.15.121:0/1316017284 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fbcd806c8f0 0x7fbcd806edb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbce8006fd0 tx=0x7fbce8008040 comp rx=0 tx=0).stop 2024-04-20T10:22:19.510 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.509+0000 7fbcfd200640 1 -- 172.21.15.121:0/1316017284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbcf81f8420 msgr2=0x7fbcf815efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:19.510 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.509+0000 7fbcfd200640 1 --2- 172.21.15.121:0/1316017284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbcf81f8420 0x7fbcf815efa0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0036040 tx=0x7fbcf0004a40 comp rx=0 tx=0).stop 2024-04-20T10:22:19.510 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.509+0000 7fbcfd200640 1 -- 172.21.15.121:0/1316017284 shutdown_connections 2024-04-20T10:22:19.510 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.509+0000 7fbcfd200640 1 --2- 172.21.15.121:0/1316017284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbcf81f8420 0x7fbcf815efa0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:19.510 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.509+0000 7fbcfd200640 1 --2- 172.21.15.121:0/1316017284 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbcf815b3b0 0x7fbcf8152570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:19.510 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.509+0000 7fbcfd200640 1 --2- 172.21.15.121:0/1316017284 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fbcd806c8f0 0x7fbcd806edb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:19.510 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.509+0000 7fbcfd200640 1 --2- 172.21.15.121:0/1316017284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcf806dcb0 0x7fbcf815b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:19.510 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.509+0000 7fbcfd200640 1 -- 172.21.15.121:0/1316017284 >> 172.21.15.121:0/1316017284 conn(0x7fbcf80b3350 msgr2=0x7fbcf806c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:19.510 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.509+0000 7fbcfd200640 1 -- 172.21.15.121:0/1316017284 shutdown_connections 2024-04-20T10:22:19.510 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.509+0000 7fbcfd200640 1 -- 172.21.15.121:0/1316017284 wait complete. 2024-04-20T10:22:19.528 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:20:51.141365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:21:00.436937+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:21:01.026957+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T10:21:04.862693+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T10:22:19.529 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:22:19.733 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.732+0000 7fe950800640 1 Processor -- start 2024-04-20T10:22:19.733 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.733+0000 7fe950800640 1 -- start start 2024-04-20T10:22:19.735 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.734+0000 7fe950800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe94815e9c0 0x7fe9480d1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:19.735 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.734+0000 7fe950800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9480d23b0 con 0x7fe9480d4930 2024-04-20T10:22:19.735 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.734+0000 7fe950800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9480d2690 con 0x7fe9480b11c0 2024-04-20T10:22:19.735 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.734+0000 7fe950800640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe9480d0f20 con 0x7fe94815e9c0 2024-04-20T10:22:19.735 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.734+0000 7fe94e800640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fe9480d4930 0x7fe9480d3170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:53404/0 (socket says 172.21.15.121:53404) 2024-04-20T10:22:19.735 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.734+0000 7fe94e800640 1 -- 172.21.15.121:0/737728444 learned_addr learned my addr 172.21.15.121:0/737728444 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:19.737 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.735+0000 7fe94de00640 1 --2- 172.21.15.121:0/737728444 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe94815e9c0 0x7fe9480d1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:19.737 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.735+0000 7fe94ca00640 1 -- 172.21.15.121:0/737728444 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3671715405 0 0) 0x7fe9480d23b0 con 0x7fe9480d4930 2024-04-20T10:22:19.737 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.735+0000 7fe94ca00640 1 -- 172.21.15.121:0/737728444 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe934003520 con 0x7fe9480d4930 2024-04-20T10:22:19.737 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.735+0000 7fe94ca00640 1 -- 172.21.15.121:0/737728444 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2767829831 0 0) 0x7fe934003520 con 0x7fe9480d4930 2024-04-20T10:22:19.737 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.735+0000 7fe94ca00640 1 -- 172.21.15.121:0/737728444 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe94815e9c0 msgr2=0x7fe9480d1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:19.737 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.735+0000 7fe94ca00640 1 --2- 172.21.15.121:0/737728444 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe94815e9c0 0x7fe9480d1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:19.737 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.735+0000 7fe94ca00640 1 -- 172.21.15.121:0/737728444 >> v1:172.21.15.123:6789/0 conn(0x7fe9480b11c0 legacy=0x7fe9480d0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:19.737 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.735+0000 7fe94ca00640 1 -- 172.21.15.121:0/737728444 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9480d2970 con 0x7fe9480d4930 2024-04-20T10:22:19.737 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.735+0000 7fe94ca00640 1 -- 172.21.15.121:0/737728444 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7fe944001670 con 0x7fe9480d4930 2024-04-20T10:22:19.738 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.735+0000 7fe94ca00640 1 -- 172.21.15.121:0/737728444 >> v1:172.21.15.121:6789/0 conn(0x7fe9480d4930 legacy=0x7fe9480d3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:19.738 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.735+0000 7fe94ca00640 1 --2- 172.21.15.121:0/737728444 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe94815e9c0 0x7fe9340039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:19.738 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.735+0000 7fe94ca00640 1 --2- 172.21.15.121:0/737728444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe934003f60 0x7fe934004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:19.738 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.735+0000 7fe94ca00640 1 --2- 172.21.15.121:0/737728444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe934004d40 0x7fe9340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:19.738 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.735+0000 7fe94ca00640 1 -- 172.21.15.121:0/737728444 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe9480d0f20 con 0x7fe934004d40 2024-04-20T10:22:19.738 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.735+0000 7fe94ca00640 1 -- 172.21.15.121:0/737728444 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe93400bb50 con 0x7fe934003f60 2024-04-20T10:22:19.738 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.735+0000 7fe94ca00640 1 -- 172.21.15.121:0/737728444 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe93400bd50 con 0x7fe94815e9c0 2024-04-20T10:22:19.738 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.736+0000 7fe94e800640 1 --2- 172.21.15.121:0/737728444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe934004d40 0x7fe9340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:19.738 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.736+0000 7fe94d400640 1 --2- 172.21.15.121:0/737728444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe934003f60 0x7fe934004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:19.738 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.736+0000 7fe94de00640 1 --2- 172.21.15.121:0/737728444 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe94815e9c0 0x7fe9340039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:19.738 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.736+0000 7fe94e800640 1 -- 172.21.15.121:0/737728444 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe94815e9c0 msgr2=0x7fe9340039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:19.738 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.736+0000 7fe94e800640 1 --2- 172.21.15.121:0/737728444 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe94815e9c0 0x7fe9340039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:19.738 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.736+0000 7fe94e800640 1 -- 172.21.15.121:0/737728444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe934003f60 msgr2=0x7fe934004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:19.739 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.736+0000 7fe94e800640 1 --2- 172.21.15.121:0/737728444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe934003f60 0x7fe934004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:19.739 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.736+0000 7fe94e800640 1 -- 172.21.15.121:0/737728444 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe934003520 con 0x7fe934004d40 2024-04-20T10:22:19.740 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.737+0000 7fe94e800640 1 --2- 172.21.15.121:0/737728444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe934004d40 0x7fe9340051c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe94400c240 tx=0x7fe934005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db4eac48a6fc09b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:19.740 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.738+0000 7fe94ca00640 1 -- 172.21.15.121:0/737728444 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe944007cf0 con 0x7fe934004d40 2024-04-20T10:22:19.740 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.738+0000 7fe94ca00640 1 -- 172.21.15.121:0/737728444 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe944030070 con 0x7fe934004d40 2024-04-20T10:22:19.740 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.738+0000 7fe94ca00640 1 -- 172.21.15.121:0/737728444 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe944005d30 con 0x7fe934004d40 2024-04-20T10:22:19.740 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.738+0000 7fe950800640 1 -- 172.21.15.121:0/737728444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe934004d40 msgr2=0x7fe9340051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:19.740 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.738+0000 7fe950800640 1 --2- 172.21.15.121:0/737728444 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe934004d40 0x7fe9340051c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe94400c240 tx=0x7fe934005830 comp rx=0 tx=0).stop 2024-04-20T10:22:19.740 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.738+0000 7fe94e800640 1 -- 172.21.15.121:0/737728444 reap_dead start 2024-04-20T10:22:19.740 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.738+0000 7fe950800640 1 -- 172.21.15.121:0/737728444 shutdown_connections 2024-04-20T10:22:19.740 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.738+0000 7fe950800640 1 -- 172.21.15.121:0/737728444 >> 172.21.15.121:0/737728444 conn(0x7fe9480b9600 msgr2=0x7fe9480b9a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:19.740 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.738+0000 7fe950800640 1 -- 172.21.15.121:0/737728444 shutdown_connections 2024-04-20T10:22:19.741 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.738+0000 7fe950800640 1 -- 172.21.15.121:0/737728444 wait complete. 2024-04-20T10:22:19.741 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.740+0000 7fe950800640 1 Processor -- start 2024-04-20T10:22:19.741 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.740+0000 7fe950800640 1 -- start start 2024-04-20T10:22:19.743 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.741+0000 7fe950800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe934004d40 0x7fe94815d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:19.743 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.741+0000 7fe950800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe9480ce0c0 0x7fe9480b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:19.743 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.741+0000 7fe950800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9480ce520 0x7fe9480b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:19.743 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.741+0000 7fe950800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe9480d1f30 con 0x7fe9480ce0c0 2024-04-20T10:22:19.743 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.741+0000 7fe950800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe9480d1db0 con 0x7fe9480ce520 2024-04-20T10:22:19.743 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.741+0000 7fe950800640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe9480d2ce0 con 0x7fe934004d40 2024-04-20T10:22:19.743 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.741+0000 7fe94d400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe9480ce0c0 0x7fe9480b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:19.744 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.741+0000 7fe94d400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe9480ce0c0 0x7fe9480b4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55024/0 (socket says 172.21.15.121:55024) 2024-04-20T10:22:19.744 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.741+0000 7fe94d400640 1 -- 172.21.15.121:0/1810401551 learned_addr learned my addr 172.21.15.121:0/1810401551 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:19.744 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.742+0000 7fe94e800640 1 --2- 172.21.15.121:0/1810401551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9480ce520 0x7fe9480b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:19.744 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.742+0000 7fe94de00640 1 --2- 172.21.15.121:0/1810401551 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe934004d40 0x7fe94815d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:19.744 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.742+0000 7fe94d400640 1 -- 172.21.15.121:0/1810401551 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe934004d40 msgr2=0x7fe94815d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:19.744 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.742+0000 7fe94d400640 1 --2- 172.21.15.121:0/1810401551 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe934004d40 0x7fe94815d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:19.744 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.742+0000 7fe94d400640 1 -- 172.21.15.121:0/1810401551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9480ce520 msgr2=0x7fe9480b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:19.744 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.742+0000 7fe94d400640 1 --2- 172.21.15.121:0/1810401551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9480ce520 0x7fe9480b54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:19.744 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.742+0000 7fe94d400640 1 -- 172.21.15.121:0/1810401551 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9480b43a0 con 0x7fe9480ce0c0 2024-04-20T10:22:19.744 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.742+0000 7fe94d400640 1 --2- 172.21.15.121:0/1810401551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe9480ce0c0 0x7fe9480b4f60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe93800e5b0 tx=0x7fe93800ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:19.744 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.742+0000 7fe93e000640 1 -- 172.21.15.121:0/1810401551 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9380052e0 con 0x7fe9480ce0c0 2024-04-20T10:22:19.744 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.743+0000 7fe93e000640 1 -- 172.21.15.121:0/1810401551 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe938018070 con 0x7fe9480ce0c0 2024-04-20T10:22:19.744 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.743+0000 7fe93e000640 1 -- 172.21.15.121:0/1810401551 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe938003810 con 0x7fe9480ce0c0 2024-04-20T10:22:19.744 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.743+0000 7fe950800640 1 -- 172.21.15.121:0/1810401551 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9480cd020 con 0x7fe9480ce0c0 2024-04-20T10:22:19.745 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.743+0000 7fe950800640 1 -- 172.21.15.121:0/1810401551 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9480cd5e0 con 0x7fe9480ce0c0 2024-04-20T10:22:19.745 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.744+0000 7fe950800640 1 -- 172.21.15.121:0/1810401551 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe948068980 con 0x7fe9480ce0c0 2024-04-20T10:22:19.745 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.745+0000 7fe93e000640 1 -- 172.21.15.121:0/1810401551 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe9380039b0 con 0x7fe9480ce0c0 2024-04-20T10:22:19.746 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.746+0000 7fe93e000640 1 --2- 172.21.15.121:0/1810401551 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe92c06c920 0x7fe92c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:19.746 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.746+0000 7fe93e000640 1 -- 172.21.15.121:0/1810401551 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fe93808bd20 con 0x7fe9480ce0c0 2024-04-20T10:22:19.747 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.746+0000 7fe94de00640 1 --2- 172.21.15.121:0/1810401551 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe92c06c920 0x7fe92c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:19.747 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.747+0000 7fe94de00640 1 --2- 172.21.15.121:0/1810401551 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe92c06c920 0x7fe92c06ede0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe940007520 tx=0x7fe94000b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:19.750 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.750+0000 7fe93e000640 1 -- 172.21.15.121:0/1810401551 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe938059cc0 con 0x7fe9480ce0c0 2024-04-20T10:22:19.950 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.950+0000 7fe950800640 1 -- 172.21.15.121:0/1810401551 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe9480b8a70 con 0x7fe9480ce0c0 2024-04-20T10:22:19.951 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.950+0000 7fe93e000640 1 -- 172.21.15.121:0/1810401551 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fe9380595c0 con 0x7fe9480ce0c0 2024-04-20T10:22:19.951 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2024-04-20T10:22:19.954 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.953+0000 7fe950800640 1 -- 172.21.15.121:0/1810401551 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe92c06c920 msgr2=0x7fe92c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:19.954 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.953+0000 7fe950800640 1 --2- 172.21.15.121:0/1810401551 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe92c06c920 0x7fe92c06ede0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe940007520 tx=0x7fe94000b040 comp rx=0 tx=0).stop 2024-04-20T10:22:19.954 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.953+0000 7fe950800640 1 -- 172.21.15.121:0/1810401551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe9480ce0c0 msgr2=0x7fe9480b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:19.955 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.953+0000 7fe950800640 1 --2- 172.21.15.121:0/1810401551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe9480ce0c0 0x7fe9480b4f60 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe93800e5b0 tx=0x7fe93800ea70 comp rx=0 tx=0).stop 2024-04-20T10:22:19.955 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.954+0000 7fe950800640 1 -- 172.21.15.121:0/1810401551 shutdown_connections 2024-04-20T10:22:19.955 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.954+0000 7fe950800640 1 --2- 172.21.15.121:0/1810401551 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe92c06c920 0x7fe92c06ede0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:19.955 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.954+0000 7fe950800640 1 --2- 172.21.15.121:0/1810401551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9480ce520 0x7fe9480b54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:19.955 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.954+0000 7fe950800640 1 --2- 172.21.15.121:0/1810401551 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe9480ce0c0 0x7fe9480b4f60 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:19.955 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.954+0000 7fe950800640 1 --2- 172.21.15.121:0/1810401551 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe934004d40 0x7fe94815d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:19.955 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.954+0000 7fe950800640 1 -- 172.21.15.121:0/1810401551 >> 172.21.15.121:0/1810401551 conn(0x7fe9480b9600 msgr2=0x7fe94806b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:19.955 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.954+0000 7fe950800640 1 -- 172.21.15.121:0/1810401551 shutdown_connections 2024-04-20T10:22:19.955 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:19.954+0000 7fe950800640 1 -- 172.21.15.121:0/1810401551 wait complete. 2024-04-20T10:22:19.972 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:22:20.161 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.160+0000 7f75cbe00640 1 Processor -- start 2024-04-20T10:22:20.161 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.161+0000 7f75cbe00640 1 -- start start 2024-04-20T10:22:20.163 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.162+0000 7f75cbe00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f75c40d1a00 0x7f75c40b1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:20.163 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.162+0000 7f75cbe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f75c40b17d0 0x7f75c40d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:20.163 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.162+0000 7f75cbe00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f75c40d1e90 con 0x7f75c40d4270 2024-04-20T10:22:20.163 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.162+0000 7f75cbe00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f75c40d40c0 con 0x7f75c40d1a00 2024-04-20T10:22:20.163 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.162+0000 7f75cbe00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f75c40d3820 con 0x7f75c40b17d0 2024-04-20T10:22:20.163 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.162+0000 7f75c8a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f75c40d1a00 0x7f75c40b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:20.163 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.162+0000 7f75c8a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f75c40d1a00 0x7f75c40b1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55026/0 (socket says 172.21.15.121:55026) 2024-04-20T10:22:20.163 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.162+0000 7f75c8a00640 1 -- 172.21.15.121:0/385404627 learned_addr learned my addr 172.21.15.121:0/385404627 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:20.164 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.163+0000 7f75c9e00640 1 --2- 172.21.15.121:0/385404627 >> v2:172.21.15.123:3300/0 conn(0x7f75c40b17d0 0x7f75c40d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:20.164 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.163+0000 7f75c8a00640 1 -- 172.21.15.121:0/385404627 >> v2:172.21.15.123:3300/0 conn(0x7f75c40b17d0 msgr2=0x7f75c40d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:20.164 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.163+0000 7f75c8a00640 1 --2- 172.21.15.121:0/385404627 >> v2:172.21.15.123:3300/0 conn(0x7f75c40b17d0 0x7f75c40d0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:20.164 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.163+0000 7f75c8a00640 1 -- 172.21.15.121:0/385404627 >> v1:172.21.15.121:6789/0 conn(0x7f75c40d4270 legacy=0x7f75c40d4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:20.164 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.163+0000 7f75c8a00640 1 -- 172.21.15.121:0/385404627 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75c40d3b00 con 0x7f75c40d1a00 2024-04-20T10:22:20.165 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.164+0000 7f75c8a00640 1 --2- 172.21.15.121:0/385404627 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75c40d1a00 0x7f75c40b1290 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f75c4068550 tx=0x7f75b402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d42649b46acc15c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:20.165 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.164+0000 7f75c3e00640 1 -- 172.21.15.121:0/385404627 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b4034070 con 0x7f75c40d1a00 2024-04-20T10:22:20.165 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.164+0000 7f75c3e00640 1 -- 172.21.15.121:0/385404627 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75b4030860 con 0x7f75c40d1a00 2024-04-20T10:22:20.165 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.164+0000 7f75c3e00640 1 -- 172.21.15.121:0/385404627 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b4030c30 con 0x7f75c40d1a00 2024-04-20T10:22:20.166 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.165+0000 7f75cbe00640 1 -- 172.21.15.121:0/385404627 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75c40d1a00 msgr2=0x7f75c40b1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:20.166 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.165+0000 7f75cbe00640 1 --2- 172.21.15.121:0/385404627 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75c40d1a00 0x7f75c40b1290 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f75c4068550 tx=0x7f75b402b9a0 comp rx=0 tx=0).stop 2024-04-20T10:22:20.166 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.165+0000 7f75cbe00640 1 -- 172.21.15.121:0/385404627 shutdown_connections 2024-04-20T10:22:20.166 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.165+0000 7f75cbe00640 1 --2- 172.21.15.121:0/385404627 >> v2:172.21.15.123:3300/0 conn(0x7f75c40b17d0 0x7f75c40d0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:20.166 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.165+0000 7f75cbe00640 1 --2- 172.21.15.121:0/385404627 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75c40d1a00 0x7f75c40b1290 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:20.166 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.165+0000 7f75cbe00640 1 -- 172.21.15.121:0/385404627 >> 172.21.15.121:0/385404627 conn(0x7f75c40b99c0 msgr2=0x7f75c40b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:20.166 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.165+0000 7f75cbe00640 1 -- 172.21.15.121:0/385404627 shutdown_connections 2024-04-20T10:22:20.166 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.165+0000 7f75cbe00640 1 -- 172.21.15.121:0/385404627 wait complete. 2024-04-20T10:22:20.167 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.167+0000 7f75cbe00640 1 Processor -- start 2024-04-20T10:22:20.167 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.167+0000 7f75cbe00640 1 -- start start 2024-04-20T10:22:20.168 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.168+0000 7f75cbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75c40b17d0 0x7f75c41f89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:20.169 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.168+0000 7f75cbe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75c40d1a00 0x7f75c40cc9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:20.169 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.168+0000 7f75cbe00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f75c40d4270 0x7f75c41f7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:20.169 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.168+0000 7f75cbe00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f75c40d53b0 con 0x7f75c40d1a00 2024-04-20T10:22:20.169 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.168+0000 7f75cbe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f75c40cfe60 con 0x7f75c40b17d0 2024-04-20T10:22:20.169 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.168+0000 7f75cbe00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f75c4161f70 con 0x7f75c40d4270 2024-04-20T10:22:20.169 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.168+0000 7f75c8a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75c40d1a00 0x7f75c40cc9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:20.169 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.168+0000 7f75c8a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75c40d1a00 0x7f75c40cc9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55036/0 (socket says 172.21.15.121:55036) 2024-04-20T10:22:20.169 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.168+0000 7f75c8a00640 1 -- 172.21.15.121:0/1677865680 learned_addr learned my addr 172.21.15.121:0/1677865680 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:20.169 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.168+0000 7f75c8a00640 1 -- 172.21.15.121:0/1677865680 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f75c40d4270 msgr2=0x7f75c41f7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:20.169 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.168+0000 7f75c8a00640 1 --2- 172.21.15.121:0/1677865680 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f75c40d4270 0x7f75c41f7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:20.169 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.168+0000 7f75c8a00640 1 -- 172.21.15.121:0/1677865680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75c40b17d0 msgr2=0x7f75c41f89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:20.169 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.168+0000 7f75c8a00640 1 --2- 172.21.15.121:0/1677865680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75c40b17d0 0x7f75c41f89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:20.169 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.168+0000 7f75c8a00640 1 -- 172.21.15.121:0/1677865680 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75c40ce140 con 0x7f75c40d1a00 2024-04-20T10:22:20.170 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.169+0000 7f75c8a00640 1 --2- 172.21.15.121:0/1677865680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75c40d1a00 0x7f75c40cc9f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f75b402b9e0 tx=0x7f75b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:20.170 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.169+0000 7f75c1600640 1 -- 172.21.15.121:0/1677865680 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b4048070 con 0x7f75c40d1a00 2024-04-20T10:22:20.171 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.169+0000 7f75c1600640 1 -- 172.21.15.121:0/1677865680 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75b4039d90 con 0x7f75c40d1a00 2024-04-20T10:22:20.171 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.169+0000 7f75cbe00640 1 -- 172.21.15.121:0/1677865680 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75c40c88d0 con 0x7f75c40d1a00 2024-04-20T10:22:20.171 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.169+0000 7f75c1600640 1 -- 172.21.15.121:0/1677865680 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b4034040 con 0x7f75c40d1a00 2024-04-20T10:22:20.171 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.169+0000 7f75cbe00640 1 -- 172.21.15.121:0/1677865680 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75c40c8e90 con 0x7f75c40d1a00 2024-04-20T10:22:20.171 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.170+0000 7f75cbe00640 1 -- 172.21.15.121:0/1677865680 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75c4068980 con 0x7f75c40d1a00 2024-04-20T10:22:20.171 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.171+0000 7f75c1600640 1 -- 172.21.15.121:0/1677865680 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f75b404b070 con 0x7f75c40d1a00 2024-04-20T10:22:20.173 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.172+0000 7f75c1600640 1 --2- 172.21.15.121:0/1677865680 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f75a806c8f0 0x7f75a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:20.173 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.172+0000 7f75c1600640 1 -- 172.21.15.121:0/1677865680 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f75b40ae140 con 0x7f75c40d1a00 2024-04-20T10:22:20.173 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.173+0000 7f75c9400640 1 --2- 172.21.15.121:0/1677865680 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f75a806c8f0 0x7f75a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:20.173 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.173+0000 7f75c9400640 1 --2- 172.21.15.121:0/1677865680 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f75a806c8f0 0x7f75a806edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f75b8000f60 tx=0x7f75b80079d0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:20.178 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.176+0000 7f75c1600640 1 -- 172.21.15.121:0/1677865680 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f75b407c160 con 0x7f75c40d1a00 2024-04-20T10:22:20.379 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.378+0000 7f75cbe00640 1 -- 172.21.15.121:0/1677865680 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f75c40b6690 con 0x7f75c40d1a00 2024-04-20T10:22:20.380 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.379+0000 7f75c1600640 1 -- 172.21.15.121:0/1677865680 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f75b407ba60 con 0x7f75c40d1a00 2024-04-20T10:22:20.380 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 32 2024-04-20T10:22:20.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.382+0000 7f75cbe00640 1 -- 172.21.15.121:0/1677865680 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f75a806c8f0 msgr2=0x7f75a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:20.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.382+0000 7f75cbe00640 1 --2- 172.21.15.121:0/1677865680 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f75a806c8f0 0x7f75a806edb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f75b8000f60 tx=0x7f75b80079d0 comp rx=0 tx=0).stop 2024-04-20T10:22:20.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.382+0000 7f75cbe00640 1 -- 172.21.15.121:0/1677865680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75c40d1a00 msgr2=0x7f75c40cc9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:20.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.382+0000 7f75cbe00640 1 --2- 172.21.15.121:0/1677865680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75c40d1a00 0x7f75c40cc9f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f75b402b9e0 tx=0x7f75b4004730 comp rx=0 tx=0).stop 2024-04-20T10:22:20.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.382+0000 7f75cbe00640 1 -- 172.21.15.121:0/1677865680 shutdown_connections 2024-04-20T10:22:20.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.382+0000 7f75cbe00640 1 --2- 172.21.15.121:0/1677865680 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f75c40d4270 0x7f75c41f7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:20.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.382+0000 7f75cbe00640 1 --2- 172.21.15.121:0/1677865680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f75c40d1a00 0x7f75c40cc9f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:20.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.382+0000 7f75cbe00640 1 --2- 172.21.15.121:0/1677865680 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f75a806c8f0 0x7f75a806edb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:20.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.382+0000 7f75cbe00640 1 --2- 172.21.15.121:0/1677865680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75c40b17d0 0x7f75c41f89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:20.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.382+0000 7f75cbe00640 1 -- 172.21.15.121:0/1677865680 >> 172.21.15.121:0/1677865680 conn(0x7f75c40b99c0 msgr2=0x7f75c40b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:20.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.382+0000 7f75cbe00640 1 -- 172.21.15.121:0/1677865680 shutdown_connections 2024-04-20T10:22:20.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.382+0000 7f75cbe00640 1 -- 172.21.15.121:0/1677865680 wait complete. 2024-04-20T10:22:20.401 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:22:20.606 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.605+0000 7fea89a00640 1 Processor -- start 2024-04-20T10:22:20.606 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.605+0000 7fea89a00640 1 -- start start 2024-04-20T10:22:20.607 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.607+0000 7fea89a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fea84156c20 0x7fea8406b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:20.608 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.607+0000 7fea89a00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea840d0360 0x7fea840d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:20.608 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.607+0000 7fea89a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea84141c60 con 0x7fea8415ca40 2024-04-20T10:22:20.608 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.607+0000 7fea89a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fea840adab0 con 0x7fea84156c20 2024-04-20T10:22:20.608 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.607+0000 7fea89a00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fea8415cf90 con 0x7fea840d0360 2024-04-20T10:22:20.608 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.607+0000 7fea7be00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fea8415ca40 0x7fea840ad3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:53422/0 (socket says 172.21.15.121:53422) 2024-04-20T10:22:20.608 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.607+0000 7fea7be00640 1 -- 172.21.15.121:0/3485121554 learned_addr learned my addr 172.21.15.121:0/3485121554 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:20.610 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.608+0000 7fea83400640 1 --2- 172.21.15.121:0/3485121554 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea840d0360 0x7fea840d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:20.610 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.608+0000 7fea82a00640 1 --2- 172.21.15.121:0/3485121554 >> v2:172.21.15.123:3300/0 conn(0x7fea84156c20 0x7fea8406b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:20.610 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.608+0000 7fea82000640 1 -- 172.21.15.121:0/3485121554 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1427898876 0 0) 0x7fea84141c60 con 0x7fea8415ca40 2024-04-20T10:22:20.610 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.608+0000 7fea82000640 1 -- 172.21.15.121:0/3485121554 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea68003520 con 0x7fea8415ca40 2024-04-20T10:22:20.610 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.608+0000 7fea82000640 1 -- 172.21.15.121:0/3485121554 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3462637006 0 0) 0x7fea68003520 con 0x7fea8415ca40 2024-04-20T10:22:20.610 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.608+0000 7fea82000640 1 -- 172.21.15.121:0/3485121554 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea840d0360 msgr2=0x7fea840d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:20.611 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.608+0000 7fea82000640 1 --2- 172.21.15.121:0/3485121554 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea840d0360 0x7fea840d07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:20.611 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.608+0000 7fea82000640 1 -- 172.21.15.121:0/3485121554 >> v2:172.21.15.123:3300/0 conn(0x7fea84156c20 msgr2=0x7fea8406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:20.611 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.608+0000 7fea82000640 1 --2- 172.21.15.121:0/3485121554 >> v2:172.21.15.123:3300/0 conn(0x7fea84156c20 0x7fea8406b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:20.611 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.608+0000 7fea82000640 1 -- 172.21.15.121:0/3485121554 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8415d270 con 0x7fea8415ca40 2024-04-20T10:22:20.611 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.608+0000 7fea82000640 1 -- 172.21.15.121:0/3485121554 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7fea6c001670 con 0x7fea8415ca40 2024-04-20T10:22:20.611 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.608+0000 7fea82000640 1 -- 172.21.15.121:0/3485121554 >> v1:172.21.15.121:6789/0 conn(0x7fea8415ca40 legacy=0x7fea840ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:20.611 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.608+0000 7fea82000640 1 --2- 172.21.15.121:0/3485121554 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea680039b0 0x7fea68003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:20.611 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.608+0000 7fea82000640 1 --2- 172.21.15.121:0/3485121554 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea840d0360 0x7fea68004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:20.611 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.608+0000 7fea82000640 1 --2- 172.21.15.121:0/3485121554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea68004d40 0x7fea680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:20.611 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.608+0000 7fea82000640 1 -- 172.21.15.121:0/3485121554 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fea840adab0 con 0x7fea680039b0 2024-04-20T10:22:20.611 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.608+0000 7fea82000640 1 -- 172.21.15.121:0/3485121554 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fea8415cf90 con 0x7fea68004d40 2024-04-20T10:22:20.611 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.608+0000 7fea82000640 1 -- 172.21.15.121:0/3485121554 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fea6800b9c0 con 0x7fea840d0360 2024-04-20T10:22:20.611 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.609+0000 7fea82a00640 1 --2- 172.21.15.121:0/3485121554 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea680039b0 0x7fea68003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:20.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.609+0000 7fea83400640 1 --2- 172.21.15.121:0/3485121554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea68004d40 0x7fea680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:20.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.609+0000 7fea7be00640 1 --2- 172.21.15.121:0/3485121554 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea840d0360 0x7fea68004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:20.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.609+0000 7fea82a00640 1 -- 172.21.15.121:0/3485121554 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea840d0360 msgr2=0x7fea68004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:20.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.609+0000 7fea82a00640 1 --2- 172.21.15.121:0/3485121554 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea840d0360 0x7fea68004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:20.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.609+0000 7fea82a00640 1 -- 172.21.15.121:0/3485121554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea68004d40 msgr2=0x7fea680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:20.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.609+0000 7fea82a00640 1 --2- 172.21.15.121:0/3485121554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea68004d40 0x7fea680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:20.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.609+0000 7fea82a00640 1 -- 172.21.15.121:0/3485121554 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea68003520 con 0x7fea680039b0 2024-04-20T10:22:20.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.609+0000 7fea83400640 1 --2- 172.21.15.121:0/3485121554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea68004d40 0x7fea680051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:20.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.609+0000 7fea7be00640 1 --2- 172.21.15.121:0/3485121554 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea840d0360 0x7fea68004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:20.613 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.610+0000 7fea82a00640 1 --2- 172.21.15.121:0/3485121554 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea680039b0 0x7fea68003dd0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fea7400ba60 tx=0x7fea68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a23d6aee53dde0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:20.614 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.610+0000 7fea82000640 1 -- 172.21.15.121:0/3485121554 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea74036020 con 0x7fea680039b0 2024-04-20T10:22:20.614 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.610+0000 7fea82000640 1 -- 172.21.15.121:0/3485121554 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea74032660 con 0x7fea680039b0 2024-04-20T10:22:20.614 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.610+0000 7fea82000640 1 -- 172.21.15.121:0/3485121554 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7400b600 con 0x7fea680039b0 2024-04-20T10:22:20.614 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.611+0000 7fea89a00640 1 -- 172.21.15.121:0/3485121554 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea680039b0 msgr2=0x7fea68003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:20.614 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.611+0000 7fea89a00640 1 --2- 172.21.15.121:0/3485121554 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea680039b0 0x7fea68003dd0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fea7400ba60 tx=0x7fea68005830 comp rx=0 tx=0).stop 2024-04-20T10:22:20.614 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.611+0000 7fea83400640 1 -- 172.21.15.121:0/3485121554 reap_dead start 2024-04-20T10:22:20.614 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.611+0000 7fea89a00640 1 -- 172.21.15.121:0/3485121554 shutdown_connections 2024-04-20T10:22:20.614 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.611+0000 7fea89a00640 1 -- 172.21.15.121:0/3485121554 >> 172.21.15.121:0/3485121554 conn(0x7fea840b3350 msgr2=0x7fea840b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:20.614 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.611+0000 7fea89a00640 1 -- 172.21.15.121:0/3485121554 shutdown_connections 2024-04-20T10:22:20.614 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.611+0000 7fea89a00640 1 -- 172.21.15.121:0/3485121554 wait complete. 2024-04-20T10:22:20.614 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.613+0000 7fea89a00640 1 Processor -- start 2024-04-20T10:22:20.614 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.613+0000 7fea89a00640 1 -- start start 2024-04-20T10:22:20.616 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.614+0000 7fea89a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea680039b0 0x7fea841f4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:20.616 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.614+0000 7fea89a00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea840ceb80 0x7fea840cf000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:20.616 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.614+0000 7fea89a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea84073250 0x7fea840736d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:20.616 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.614+0000 7fea89a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fea8406be40 con 0x7fea680039b0 2024-04-20T10:22:20.616 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.614+0000 7fea89a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fea8415d830 con 0x7fea84073250 2024-04-20T10:22:20.617 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.614+0000 7fea89a00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fea8406bfc0 con 0x7fea840ceb80 2024-04-20T10:22:20.617 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.614+0000 7fea82a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea680039b0 0x7fea841f4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:20.617 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.614+0000 7fea82a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea680039b0 0x7fea841f4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55050/0 (socket says 172.21.15.121:55050) 2024-04-20T10:22:20.617 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.615+0000 7fea82a00640 1 -- 172.21.15.121:0/3027495906 learned_addr learned my addr 172.21.15.121:0/3027495906 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:20.617 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.615+0000 7fea7be00640 1 --2- 172.21.15.121:0/3027495906 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea840ceb80 0x7fea840cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:20.617 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.615+0000 7fea82a00640 1 -- 172.21.15.121:0/3027495906 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea840ceb80 msgr2=0x7fea840cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:20.617 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.615+0000 7fea82a00640 1 --2- 172.21.15.121:0/3027495906 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea840ceb80 0x7fea840cf000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:20.617 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.615+0000 7fea82a00640 1 -- 172.21.15.121:0/3027495906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea84073250 msgr2=0x7fea840736d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:20.617 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.615+0000 7fea82a00640 1 --2- 172.21.15.121:0/3027495906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea84073250 0x7fea840736d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:20.617 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.615+0000 7fea82a00640 1 -- 172.21.15.121:0/3027495906 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea840cf6f0 con 0x7fea680039b0 2024-04-20T10:22:20.617 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.615+0000 7fea82a00640 1 --2- 172.21.15.121:0/3027495906 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea680039b0 0x7fea841f4a20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fea7400bb80 tx=0x7fea74004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:20.617 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.615+0000 7fea7aa00640 1 -- 172.21.15.121:0/3027495906 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea74036040 con 0x7fea680039b0 2024-04-20T10:22:20.617 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.616+0000 7fea7aa00640 1 -- 172.21.15.121:0/3027495906 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea740309f0 con 0x7fea680039b0 2024-04-20T10:22:20.618 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.616+0000 7fea7aa00640 1 -- 172.21.15.121:0/3027495906 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7400b600 con 0x7fea680039b0 2024-04-20T10:22:20.618 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.616+0000 7fea89a00640 1 -- 172.21.15.121:0/3027495906 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea8414d4b0 con 0x7fea680039b0 2024-04-20T10:22:20.618 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.616+0000 7fea89a00640 1 -- 172.21.15.121:0/3027495906 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea8414da70 con 0x7fea680039b0 2024-04-20T10:22:20.618 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.617+0000 7fea89a00640 1 -- 172.21.15.121:0/3027495906 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea84068980 con 0x7fea680039b0 2024-04-20T10:22:20.618 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.618+0000 7fea7aa00640 1 -- 172.21.15.121:0/3027495906 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fea74039020 con 0x7fea680039b0 2024-04-20T10:22:20.619 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.618+0000 7fea7aa00640 1 --2- 172.21.15.121:0/3027495906 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fea6006c920 0x7fea6006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:20.619 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.619+0000 7fea7aa00640 1 -- 172.21.15.121:0/3027495906 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fea740ae640 con 0x7fea680039b0 2024-04-20T10:22:20.619 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.619+0000 7fea7be00640 1 --2- 172.21.15.121:0/3027495906 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fea6006c920 0x7fea6006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:20.620 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.619+0000 7fea7be00640 1 --2- 172.21.15.121:0/3027495906 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fea6006c920 0x7fea6006ede0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fea6c000c00 tx=0x7fea6c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:20.623 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.623+0000 7fea7aa00640 1 -- 172.21.15.121:0/3027495906 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fea7407c5d0 con 0x7fea680039b0 2024-04-20T10:22:20.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.821+0000 7fea89a00640 1 -- 172.21.15.121:0/3027495906 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fea8415df60 con 0x7fea680039b0 2024-04-20T10:22:20.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.822+0000 7fea7aa00640 1 -- 172.21.15.121:0/3027495906 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fea7407bed0 con 0x7fea680039b0 2024-04-20T10:22:20.823 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2024-04-20T10:22:20.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.826+0000 7fea89a00640 1 -- 172.21.15.121:0/3027495906 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fea6006c920 msgr2=0x7fea6006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:20.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.826+0000 7fea89a00640 1 --2- 172.21.15.121:0/3027495906 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fea6006c920 0x7fea6006ede0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fea6c000c00 tx=0x7fea6c009040 comp rx=0 tx=0).stop 2024-04-20T10:22:20.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.826+0000 7fea89a00640 1 -- 172.21.15.121:0/3027495906 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea680039b0 msgr2=0x7fea841f4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:20.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.826+0000 7fea89a00640 1 --2- 172.21.15.121:0/3027495906 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea680039b0 0x7fea841f4a20 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fea7400bb80 tx=0x7fea74004730 comp rx=0 tx=0).stop 2024-04-20T10:22:20.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.826+0000 7fea89a00640 1 -- 172.21.15.121:0/3027495906 shutdown_connections 2024-04-20T10:22:20.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.826+0000 7fea89a00640 1 --2- 172.21.15.121:0/3027495906 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fea6006c920 0x7fea6006ede0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:20.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.826+0000 7fea89a00640 1 --2- 172.21.15.121:0/3027495906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea84073250 0x7fea840736d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:20.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.826+0000 7fea89a00640 1 --2- 172.21.15.121:0/3027495906 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fea840ceb80 0x7fea840cf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:20.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.826+0000 7fea89a00640 1 --2- 172.21.15.121:0/3027495906 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fea680039b0 0x7fea841f4a20 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:20.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.826+0000 7fea89a00640 1 -- 172.21.15.121:0/3027495906 >> 172.21.15.121:0/3027495906 conn(0x7fea840b3350 msgr2=0x7fea8415e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:20.828 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.826+0000 7fea89a00640 1 -- 172.21.15.121:0/3027495906 shutdown_connections 2024-04-20T10:22:20.828 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:20.827+0000 7fea89a00640 1 -- 172.21.15.121:0/3027495906 wait complete. 2024-04-20T10:22:20.846 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T10:22:21.049 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.048+0000 7f0403000640 1 Processor -- start 2024-04-20T10:22:21.049 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.049+0000 7f0403000640 1 -- start start 2024-04-20T10:22:21.050 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.050+0000 7f0403000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03fc0d4270 0x7f03fc0d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:21.050 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.050+0000 7f0403000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f03fc0d4650 0x7f03fc0b1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:21.051 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.050+0000 7f0403000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03fc0d3820 con 0x7f03fc0b17d0 2024-04-20T10:22:21.051 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.050+0000 7f0403000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f03fc0d40c0 con 0x7f03fc0d4650 2024-04-20T10:22:21.051 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.050+0000 7f0403000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f03fc0d3b00 con 0x7f03fc0d4270 2024-04-20T10:22:21.053 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.050+0000 7f0401000640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f03fc0b17d0 0x7f03fc0d0380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:53436/0 (socket says 172.21.15.121:53436) 2024-04-20T10:22:21.053 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.050+0000 7f0401000640 1 -- 172.21.15.121:0/4013125299 learned_addr learned my addr 172.21.15.121:0/4013125299 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:21.053 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.051+0000 7f03fb400640 1 --2- 172.21.15.121:0/4013125299 >> v2:172.21.15.123:3300/0 conn(0x7f03fc0d4650 0x7f03fc0b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:21.053 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.051+0000 7f03fbe00640 1 --2- 172.21.15.121:0/4013125299 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03fc0d4270 0x7f03fc0d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:21.053 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.051+0000 7f03faa00640 1 -- 172.21.15.121:0/4013125299 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2012310908 0 0) 0x7f03fc0d3820 con 0x7f03fc0b17d0 2024-04-20T10:22:21.053 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.051+0000 7f03faa00640 1 -- 172.21.15.121:0/4013125299 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f03e0003520 con 0x7f03fc0b17d0 2024-04-20T10:22:21.053 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.051+0000 7f03faa00640 1 -- 172.21.15.121:0/4013125299 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 800715233 0 0) 0x7f03e0003520 con 0x7f03fc0b17d0 2024-04-20T10:22:21.053 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.051+0000 7f03faa00640 1 -- 172.21.15.121:0/4013125299 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03fc0d4270 msgr2=0x7f03fc0d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:21.054 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.051+0000 7f03faa00640 1 --2- 172.21.15.121:0/4013125299 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03fc0d4270 0x7f03fc0d1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:21.054 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.051+0000 7f03faa00640 1 -- 172.21.15.121:0/4013125299 >> v2:172.21.15.123:3300/0 conn(0x7f03fc0d4650 msgr2=0x7f03fc0b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:21.054 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.051+0000 7f03faa00640 1 --2- 172.21.15.121:0/4013125299 >> v2:172.21.15.123:3300/0 conn(0x7f03fc0d4650 0x7f03fc0b1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:21.054 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.051+0000 7f03faa00640 1 -- 172.21.15.121:0/4013125299 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03fc0d3de0 con 0x7f03fc0b17d0 2024-04-20T10:22:21.054 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.051+0000 7f03faa00640 1 -- 172.21.15.121:0/4013125299 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f03f4001670 con 0x7f03fc0b17d0 2024-04-20T10:22:21.054 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.051+0000 7f03faa00640 1 -- 172.21.15.121:0/4013125299 >> v1:172.21.15.121:6789/0 conn(0x7f03fc0b17d0 legacy=0x7f03fc0d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:21.054 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.052+0000 7f03faa00640 1 --2- 172.21.15.121:0/4013125299 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03fc0d4270 0x7f03e00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:21.054 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.052+0000 7f03faa00640 1 --2- 172.21.15.121:0/4013125299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03e0003f30 0x7f03e0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:21.054 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.052+0000 7f03faa00640 1 --2- 172.21.15.121:0/4013125299 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03e0004d40 0x7f03e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:21.054 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.052+0000 7f03faa00640 1 -- 172.21.15.121:0/4013125299 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc0d40c0 con 0x7f03e0004d40 2024-04-20T10:22:21.054 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.052+0000 7f03faa00640 1 -- 172.21.15.121:0/4013125299 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc0d3b00 con 0x7f03e0003f30 2024-04-20T10:22:21.054 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.052+0000 7f03faa00640 1 -- 172.21.15.121:0/4013125299 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f03e000b9c0 con 0x7f03fc0d4270 2024-04-20T10:22:21.055 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.052+0000 7f0401000640 1 --2- 172.21.15.121:0/4013125299 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03e0004d40 0x7f03e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:21.055 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.052+0000 7f03fbe00640 1 --2- 172.21.15.121:0/4013125299 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03fc0d4270 0x7f03e00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:21.055 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.052+0000 7f03fb400640 1 --2- 172.21.15.121:0/4013125299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03e0003f30 0x7f03e0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:21.055 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.052+0000 7f0401000640 1 -- 172.21.15.121:0/4013125299 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03fc0d4270 msgr2=0x7f03e00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:21.055 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.052+0000 7f0401000640 1 --2- 172.21.15.121:0/4013125299 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03fc0d4270 0x7f03e00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:21.055 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.053+0000 7f0401000640 1 -- 172.21.15.121:0/4013125299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03e0003f30 msgr2=0x7f03e0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:21.055 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.053+0000 7f0401000640 1 --2- 172.21.15.121:0/4013125299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03e0003f30 0x7f03e0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:21.055 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.053+0000 7f0401000640 1 -- 172.21.15.121:0/4013125299 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e0003520 con 0x7f03e0004d40 2024-04-20T10:22:21.056 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.054+0000 7f0401000640 1 --2- 172.21.15.121:0/4013125299 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03e0004d40 0x7f03e00051c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f03fc08df30 tx=0x7f03e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46d255719a87168d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:21.057 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.054+0000 7f03faa00640 1 -- 172.21.15.121:0/4013125299 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4007d50 con 0x7f03e0004d40 2024-04-20T10:22:21.057 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.054+0000 7f03faa00640 1 -- 172.21.15.121:0/4013125299 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03f4030070 con 0x7f03e0004d40 2024-04-20T10:22:21.057 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.054+0000 7f03faa00640 1 -- 172.21.15.121:0/4013125299 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4005d70 con 0x7f03e0004d40 2024-04-20T10:22:21.057 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.055+0000 7f0403000640 1 -- 172.21.15.121:0/4013125299 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03e0004d40 msgr2=0x7f03e00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:21.057 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.055+0000 7f0403000640 1 --2- 172.21.15.121:0/4013125299 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03e0004d40 0x7f03e00051c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f03fc08df30 tx=0x7f03e0005830 comp rx=0 tx=0).stop 2024-04-20T10:22:21.057 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.055+0000 7f0401000640 1 -- 172.21.15.121:0/4013125299 reap_dead start 2024-04-20T10:22:21.057 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.055+0000 7f0403000640 1 -- 172.21.15.121:0/4013125299 shutdown_connections 2024-04-20T10:22:21.057 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.055+0000 7f0403000640 1 -- 172.21.15.121:0/4013125299 >> 172.21.15.121:0/4013125299 conn(0x7f03fc0b99c0 msgr2=0x7f03fc0b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:21.057 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.055+0000 7f0403000640 1 -- 172.21.15.121:0/4013125299 shutdown_connections 2024-04-20T10:22:21.057 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.055+0000 7f0403000640 1 -- 172.21.15.121:0/4013125299 wait complete. 2024-04-20T10:22:21.058 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.057+0000 7f0403000640 1 Processor -- start 2024-04-20T10:22:21.058 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.057+0000 7f0403000640 1 -- start start 2024-04-20T10:22:21.059 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.058+0000 7f0403000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03fc0c8bb0 0x7f03fc0b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:21.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.058+0000 7f0403000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03fc0c8f90 0x7f03fc0b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:21.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.058+0000 7f0403000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03fc1fbcd0 0x7f03fc1fc150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:21.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.058+0000 7f0403000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc161f70 con 0x7f03fc1fbcd0 2024-04-20T10:22:21.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.058+0000 7f0403000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc0d53b0 con 0x7f03fc0c8bb0 2024-04-20T10:22:21.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.058+0000 7f0403000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f03fc1620f0 con 0x7f03fc0c8f90 2024-04-20T10:22:21.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.058+0000 7f0401000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03fc1fbcd0 0x7f03fc1fc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:21.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.058+0000 7f0401000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03fc1fbcd0 0x7f03fc1fc150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55062/0 (socket says 172.21.15.121:55062) 2024-04-20T10:22:21.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.058+0000 7f0401000640 1 -- 172.21.15.121:0/2680651251 learned_addr learned my addr 172.21.15.121:0/2680651251 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:21.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.059+0000 7f03fb400640 1 --2- 172.21.15.121:0/2680651251 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03fc0c8f90 0x7f03fc0b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:21.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.059+0000 7f03fbe00640 1 --2- 172.21.15.121:0/2680651251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03fc0c8bb0 0x7f03fc0b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:21.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.059+0000 7f0401000640 1 -- 172.21.15.121:0/2680651251 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03fc0c8f90 msgr2=0x7f03fc0b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:21.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.059+0000 7f0401000640 1 --2- 172.21.15.121:0/2680651251 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03fc0c8f90 0x7f03fc0b55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:21.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.059+0000 7f0401000640 1 -- 172.21.15.121:0/2680651251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03fc0c8bb0 msgr2=0x7f03fc0b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:21.061 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.059+0000 7f0401000640 1 --2- 172.21.15.121:0/2680651251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03fc0c8bb0 0x7f03fc0b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:21.061 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.059+0000 7f0401000640 1 -- 172.21.15.121:0/2680651251 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03fc1f86f0 con 0x7f03fc1fbcd0 2024-04-20T10:22:21.061 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.059+0000 7f0401000640 1 --2- 172.21.15.121:0/2680651251 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03fc1fbcd0 0x7f03fc1fc150 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f03f402c590 tx=0x7f03f4001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:21.061 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.059+0000 7f03f3e00640 1 -- 172.21.15.121:0/2680651251 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4007d50 con 0x7f03fc1fbcd0 2024-04-20T10:22:21.061 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.060+0000 7f03f3e00640 1 -- 172.21.15.121:0/2680651251 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03f4004cb0 con 0x7f03fc1fbcd0 2024-04-20T10:22:21.062 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.060+0000 7f03f3e00640 1 -- 172.21.15.121:0/2680651251 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4001c90 con 0x7f03fc1fbcd0 2024-04-20T10:22:21.062 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.060+0000 7f0403000640 1 -- 172.21.15.121:0/2680651251 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03fc1f8960 con 0x7f03fc1fbcd0 2024-04-20T10:22:21.062 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.060+0000 7f0403000640 1 -- 172.21.15.121:0/2680651251 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03fc1fca50 con 0x7f03fc1fbcd0 2024-04-20T10:22:21.062 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.061+0000 7f0403000640 1 -- 172.21.15.121:0/2680651251 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03fc068980 con 0x7f03fc1fbcd0 2024-04-20T10:22:21.062 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.062+0000 7f03f3e00640 1 -- 172.21.15.121:0/2680651251 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f03f4007290 con 0x7f03fc1fbcd0 2024-04-20T10:22:21.063 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.063+0000 7f03f3e00640 1 --2- 172.21.15.121:0/2680651251 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f03d806c920 0x7f03d806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:21.063 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.063+0000 7f03f3e00640 1 -- 172.21.15.121:0/2680651251 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f03f40accb0 con 0x7f03fc1fbcd0 2024-04-20T10:22:21.063 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.063+0000 7f03fbe00640 1 --2- 172.21.15.121:0/2680651251 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f03d806c920 0x7f03d806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:21.064 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.064+0000 7f03fbe00640 1 --2- 172.21.15.121:0/2680651251 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f03d806c920 0x7f03d806ede0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f03e4009a10 tx=0x7f03e4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:21.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.067+0000 7f03f3e00640 1 -- 172.21.15.121:0/2680651251 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f03f407acd0 con 0x7f03fc1fbcd0 2024-04-20T10:22:21.270 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.269+0000 7f0403000640 1 -- 172.21.15.121:0/2680651251 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f03fc069090 con 0x7f03fc1fbcd0 2024-04-20T10:22:21.270 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.270+0000 7f03f3e00640 1 -- 172.21.15.121:0/2680651251 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f03f407a5d0 con 0x7f03fc1fbcd0 2024-04-20T10:22:21.271 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 32 2024-04-20T10:22:21.273 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.273+0000 7f0403000640 1 -- 172.21.15.121:0/2680651251 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f03d806c920 msgr2=0x7f03d806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:21.273 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.273+0000 7f0403000640 1 --2- 172.21.15.121:0/2680651251 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f03d806c920 0x7f03d806ede0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f03e4009a10 tx=0x7f03e4008040 comp rx=0 tx=0).stop 2024-04-20T10:22:21.273 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.273+0000 7f0403000640 1 -- 172.21.15.121:0/2680651251 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03fc1fbcd0 msgr2=0x7f03fc1fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:21.274 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.273+0000 7f0403000640 1 --2- 172.21.15.121:0/2680651251 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03fc1fbcd0 0x7f03fc1fc150 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f03f402c590 tx=0x7f03f4001a40 comp rx=0 tx=0).stop 2024-04-20T10:22:21.274 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.273+0000 7f0403000640 1 -- 172.21.15.121:0/2680651251 shutdown_connections 2024-04-20T10:22:21.274 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.273+0000 7f0403000640 1 --2- 172.21.15.121:0/2680651251 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f03fc1fbcd0 0x7f03fc1fc150 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:21.274 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.273+0000 7f0403000640 1 --2- 172.21.15.121:0/2680651251 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f03fc0c8f90 0x7f03fc0b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:21.274 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.273+0000 7f0403000640 1 --2- 172.21.15.121:0/2680651251 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f03d806c920 0x7f03d806ede0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:21.274 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.273+0000 7f0403000640 1 --2- 172.21.15.121:0/2680651251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03fc0c8bb0 0x7f03fc0b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:21.274 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.273+0000 7f0403000640 1 -- 172.21.15.121:0/2680651251 >> 172.21.15.121:0/2680651251 conn(0x7f03fc0b99c0 msgr2=0x7f03fc160cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:21.275 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.273+0000 7f0403000640 1 -- 172.21.15.121:0/2680651251 shutdown_connections 2024-04-20T10:22:21.275 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.273+0000 7f0403000640 1 -- 172.21.15.121:0/2680651251 wait complete. 2024-04-20T10:22:21.292 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T10:22:21.492 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.491+0000 7fe3c3200640 1 Processor -- start 2024-04-20T10:22:21.492 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.492+0000 7fe3c3200640 1 -- start start 2024-04-20T10:22:21.494 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.493+0000 7fe3c3200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe3c40b2f70 0x7fe3c40b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:21.494 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.493+0000 7fe3c3200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe3c40adf60 0x7fe3c415f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:21.494 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.493+0000 7fe3c3200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe3c415faf0 0x7fe3c406b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:21.494 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.493+0000 7fe3c3200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fe3c4071a80 con 0x7fe3c40b2f70 2024-04-20T10:22:21.494 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.493+0000 7fe3c3200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fe3c40ae3c0 con 0x7fe3c415faf0 2024-04-20T10:22:21.494 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.493+0000 7fe3c3200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe3c4071cf0 con 0x7fe3c40adf60 2024-04-20T10:22:21.494 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.493+0000 7fe3c1e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe3c40b2f70 0x7fe3c40b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:21.494 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.494+0000 7fe3c1e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe3c40b2f70 0x7fe3c40b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55068/0 (socket says 172.21.15.121:55068) 2024-04-20T10:22:21.494 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.494+0000 7fe3c1e00640 1 -- 172.21.15.121:0/1567341455 learned_addr learned my addr 172.21.15.121:0/1567341455 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:21.495 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.494+0000 7fe3c1e00640 1 -- 172.21.15.121:0/1567341455 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe3c40adf60 msgr2=0x7fe3c415f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:21.495 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.494+0000 7fe3b9400640 1 --2- 172.21.15.121:0/1567341455 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe3c40adf60 0x7fe3c415f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:21.495 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.494+0000 7fe3c1e00640 1 --2- 172.21.15.121:0/1567341455 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe3c40adf60 0x7fe3c415f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:21.495 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.494+0000 7fe3c2800640 1 --2- 172.21.15.121:0/1567341455 >> v2:172.21.15.123:3300/0 conn(0x7fe3c415faf0 0x7fe3c406b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:21.495 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.494+0000 7fe3c1e00640 1 -- 172.21.15.121:0/1567341455 >> v2:172.21.15.123:3300/0 conn(0x7fe3c415faf0 msgr2=0x7fe3c406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:21.495 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.494+0000 7fe3c1e00640 1 --2- 172.21.15.121:0/1567341455 >> v2:172.21.15.123:3300/0 conn(0x7fe3c415faf0 0x7fe3c406b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:21.495 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.494+0000 7fe3c1e00640 1 -- 172.21.15.121:0/1567341455 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c4071fd0 con 0x7fe3c40b2f70 2024-04-20T10:22:21.496 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.494+0000 7fe3c2800640 1 --2- 172.21.15.121:0/1567341455 >> v2:172.21.15.123:3300/0 conn(0x7fe3c415faf0 0x7fe3c406b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:21.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.495+0000 7fe3c1e00640 1 --2- 172.21.15.121:0/1567341455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3c40b2f70 0x7fe3c40b4320 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe3c40d1120 tx=0x7fe3b402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=561bc376190d4108 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:21.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.496+0000 7fe3c1400640 1 -- 172.21.15.121:0/1567341455 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4035020 con 0x7fe3c40b2f70 2024-04-20T10:22:21.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.496+0000 7fe3c1400640 1 -- 172.21.15.121:0/1567341455 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3b4031870 con 0x7fe3c40b2f70 2024-04-20T10:22:21.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.496+0000 7fe3c1400640 1 -- 172.21.15.121:0/1567341455 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4031c40 con 0x7fe3c40b2f70 2024-04-20T10:22:21.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.496+0000 7fe3c3200640 1 -- 172.21.15.121:0/1567341455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3c40b2f70 msgr2=0x7fe3c40b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:21.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.496+0000 7fe3c3200640 1 --2- 172.21.15.121:0/1567341455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3c40b2f70 0x7fe3c40b4320 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe3c40d1120 tx=0x7fe3b402b910 comp rx=0 tx=0).stop 2024-04-20T10:22:21.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.496+0000 7fe3c3200640 1 -- 172.21.15.121:0/1567341455 shutdown_connections 2024-04-20T10:22:21.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.496+0000 7fe3c3200640 1 --2- 172.21.15.121:0/1567341455 >> v2:172.21.15.123:3300/0 conn(0x7fe3c415faf0 0x7fe3c406b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:21.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.496+0000 7fe3c3200640 1 --2- 172.21.15.121:0/1567341455 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe3c40adf60 0x7fe3c415f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:21.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.496+0000 7fe3c3200640 1 --2- 172.21.15.121:0/1567341455 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3c40b2f70 0x7fe3c40b4320 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:21.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.496+0000 7fe3c3200640 1 -- 172.21.15.121:0/1567341455 >> 172.21.15.121:0/1567341455 conn(0x7fe3c40d0040 msgr2=0x7fe3c40d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:21.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.496+0000 7fe3c3200640 1 -- 172.21.15.121:0/1567341455 shutdown_connections 2024-04-20T10:22:21.498 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.497+0000 7fe3c3200640 1 -- 172.21.15.121:0/1567341455 wait complete. 2024-04-20T10:22:21.498 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.498+0000 7fe3c3200640 1 Processor -- start 2024-04-20T10:22:21.499 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.498+0000 7fe3c3200640 1 -- start start 2024-04-20T10:22:21.500 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.499+0000 7fe3c3200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3c40adf60 0x7fe3c41497a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:21.500 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.499+0000 7fe3c3200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe3c40b2f70 0x7fe3c41599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:21.500 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.499+0000 7fe3c3200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3c415faf0 0x7fe3c4154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:21.500 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.499+0000 7fe3c3200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c406be40 con 0x7fe3c40adf60 2024-04-20T10:22:21.501 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.499+0000 7fe3c3200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c40736c0 con 0x7fe3c415faf0 2024-04-20T10:22:21.501 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.499+0000 7fe3c3200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe3c406bfc0 con 0x7fe3c40b2f70 2024-04-20T10:22:21.501 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.499+0000 7fe3c1e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3c40adf60 0x7fe3c41497a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:21.501 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.500+0000 7fe3c1e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3c40adf60 0x7fe3c41497a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55074/0 (socket says 172.21.15.121:55074) 2024-04-20T10:22:21.501 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.500+0000 7fe3c1e00640 1 -- 172.21.15.121:0/3256514345 learned_addr learned my addr 172.21.15.121:0/3256514345 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:21.501 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.500+0000 7fe3c2800640 1 --2- 172.21.15.121:0/3256514345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3c415faf0 0x7fe3c4154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:21.501 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.500+0000 7fe3c1e00640 1 -- 172.21.15.121:0/3256514345 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe3c40b2f70 msgr2=0x7fe3c41599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:21.501 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.500+0000 7fe3c1e00640 1 --2- 172.21.15.121:0/3256514345 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe3c40b2f70 0x7fe3c41599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:21.501 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.500+0000 7fe3c1e00640 1 -- 172.21.15.121:0/3256514345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3c415faf0 msgr2=0x7fe3c4154040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:21.501 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.500+0000 7fe3c1e00640 1 --2- 172.21.15.121:0/3256514345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3c415faf0 0x7fe3c4154040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:21.501 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.500+0000 7fe3c1e00640 1 -- 172.21.15.121:0/3256514345 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c4158680 con 0x7fe3c40adf60 2024-04-20T10:22:21.501 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.500+0000 7fe3c2800640 1 --2- 172.21.15.121:0/3256514345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3c415faf0 0x7fe3c4154040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:21.502 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.500+0000 7fe3c1e00640 1 --2- 172.21.15.121:0/3256514345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3c40adf60 0x7fe3c41497a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe3b402b950 tx=0x7fe3b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:21.502 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.501+0000 7fe3baa00640 1 -- 172.21.15.121:0/3256514345 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4040070 con 0x7fe3c40adf60 2024-04-20T10:22:21.502 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.501+0000 7fe3baa00640 1 -- 172.21.15.121:0/3256514345 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3b403a420 con 0x7fe3c40adf60 2024-04-20T10:22:21.502 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.501+0000 7fe3baa00640 1 -- 172.21.15.121:0/3256514345 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4035040 con 0x7fe3c40adf60 2024-04-20T10:22:21.503 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.501+0000 7fe3c3200640 1 -- 172.21.15.121:0/3256514345 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3c4158920 con 0x7fe3c40adf60 2024-04-20T10:22:21.503 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.501+0000 7fe3c3200640 1 -- 172.21.15.121:0/3256514345 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3c4158ee0 con 0x7fe3c40adf60 2024-04-20T10:22:21.503 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.502+0000 7fe3c3200640 1 -- 172.21.15.121:0/3256514345 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3c40742b0 con 0x7fe3c40adf60 2024-04-20T10:22:21.503 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.503+0000 7fe3baa00640 1 -- 172.21.15.121:0/3256514345 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe3b404b060 con 0x7fe3c40adf60 2024-04-20T10:22:21.504 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.504+0000 7fe3baa00640 1 --2- 172.21.15.121:0/3256514345 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe3a006c920 0x7fe3a006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:21.504 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.504+0000 7fe3baa00640 1 -- 172.21.15.121:0/3256514345 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fe3b40adf50 con 0x7fe3c40adf60 2024-04-20T10:22:21.504 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.504+0000 7fe3b9400640 1 --2- 172.21.15.121:0/3256514345 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe3a006c920 0x7fe3a006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:21.505 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.505+0000 7fe3b9400640 1 --2- 172.21.15.121:0/3256514345 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe3a006c920 0x7fe3a006ede0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe3c40b4180 tx=0x7fe3ac008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:21.508 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.508+0000 7fe3baa00640 1 -- 172.21.15.121:0/3256514345 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe3c40742b0 con 0x7fe3c40adf60 2024-04-20T10:22:21.725 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.724+0000 7fe3c3200640 1 -- 172.21.15.121:0/3256514345 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe3c4073220 con 0x7fe3c40adf60 2024-04-20T10:22:21.725 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.725+0000 7fe3baa00640 1 -- 172.21.15.121:0/3256514345 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7fe3c4073220 con 0x7fe3c40adf60 2024-04-20T10:22:21.726 INFO:teuthology.orchestra.run.smithi121.stderr:listed 0 entries 2024-04-20T10:22:21.728 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.727+0000 7fe3c3200640 1 -- 172.21.15.121:0/3256514345 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe3a006c920 msgr2=0x7fe3a006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:21.728 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.727+0000 7fe3c3200640 1 --2- 172.21.15.121:0/3256514345 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe3a006c920 0x7fe3a006ede0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe3c40b4180 tx=0x7fe3ac008040 comp rx=0 tx=0).stop 2024-04-20T10:22:21.728 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.727+0000 7fe3c3200640 1 -- 172.21.15.121:0/3256514345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3c40adf60 msgr2=0x7fe3c41497a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:21.728 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.727+0000 7fe3c3200640 1 --2- 172.21.15.121:0/3256514345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3c40adf60 0x7fe3c41497a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe3b402b950 tx=0x7fe3b40046c0 comp rx=0 tx=0).stop 2024-04-20T10:22:21.729 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.728+0000 7fe3c3200640 1 -- 172.21.15.121:0/3256514345 shutdown_connections 2024-04-20T10:22:21.729 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.728+0000 7fe3c3200640 1 --2- 172.21.15.121:0/3256514345 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe3a006c920 0x7fe3a006ede0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:21.729 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.728+0000 7fe3c3200640 1 --2- 172.21.15.121:0/3256514345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe3c415faf0 0x7fe3c4154040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:21.729 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.728+0000 7fe3c3200640 1 --2- 172.21.15.121:0/3256514345 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe3c40b2f70 0x7fe3c41599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:21.729 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.728+0000 7fe3c3200640 1 --2- 172.21.15.121:0/3256514345 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe3c40adf60 0x7fe3c41497a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:21.729 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.728+0000 7fe3c3200640 1 -- 172.21.15.121:0/3256514345 >> 172.21.15.121:0/3256514345 conn(0x7fe3c40d0040 msgr2=0x7fe3c40cf640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:21.729 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.728+0000 7fe3c3200640 1 -- 172.21.15.121:0/3256514345 shutdown_connections 2024-04-20T10:22:21.729 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:21.728+0000 7fe3c3200640 1 -- 172.21.15.121:0/3256514345 wait complete. 2024-04-20T10:22:21.745 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi123.front.sepia.ceph.com... 2024-04-20T10:22:21.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:22:21.746 DEBUG:teuthology.orchestra.run.smithi123:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-20T10:22:21.839 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:22:21.839 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2024-04-20T10:22:21.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:22:21.839 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-20T10:22:21.909 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T10:22:21.909 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:22:21.965 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T10:22:21.966 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:22:22.015 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:22:22.195 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.194+0000 7f09a6800640 1 Processor -- start 2024-04-20T10:22:22.195 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.195+0000 7f09a6800640 1 -- start start 2024-04-20T10:22:22.197 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.196+0000 7f09a6800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f09a00b3330 0x7f09a00d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:22.197 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.196+0000 7f09a6800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09a006c190 con 0x7f09a0161d50 2024-04-20T10:22:22.197 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.196+0000 7f09a6800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09a006c470 con 0x7f09a006cde0 2024-04-20T10:22:22.197 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.196+0000 7f09a6800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f09a0162130 con 0x7f09a00b3330 2024-04-20T10:22:22.197 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.196+0000 7f09a5400640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f09a0161d50 0x7f09a006c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:53450/0 (socket says 172.21.15.121:53450) 2024-04-20T10:22:22.197 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.196+0000 7f09a5400640 1 -- 172.21.15.121:0/794635960 learned_addr learned my addr 172.21.15.121:0/794635960 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:22.197 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.196+0000 7f09a5e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f09a00b3330 0x7f09a00d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:22.197 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.197+0000 7f09a5e00640 1 -- 172.21.15.121:0/794635960 >> v1:172.21.15.123:6789/0 conn(0x7f09a006cde0 legacy=0x7f09a00b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:22.198 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.197+0000 7f09a5e00640 1 -- 172.21.15.121:0/794635960 >> v1:172.21.15.121:6789/0 conn(0x7f09a0161d50 legacy=0x7f09a006c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:22.198 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.197+0000 7f09a5e00640 1 -- 172.21.15.121:0/794635960 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a00d2e00 con 0x7f09a00b3330 2024-04-20T10:22:22.198 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.198+0000 7f09a5e00640 1 --2- 172.21.15.121:0/794635960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09a00b3330 0x7f09a00d38e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f09980097a0 tx=0x7f099802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1dd6fa09a24191e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:22.199 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.198+0000 7f099fe00640 1 -- 172.21.15.121:0/794635960 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998035020 con 0x7f09a00b3330 2024-04-20T10:22:22.199 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.198+0000 7f099fe00640 1 -- 172.21.15.121:0/794635960 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0998031830 con 0x7f09a00b3330 2024-04-20T10:22:22.199 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.198+0000 7f099fe00640 1 -- 172.21.15.121:0/794635960 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998031c00 con 0x7f09a00b3330 2024-04-20T10:22:22.200 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.199+0000 7f09a6800640 1 -- 172.21.15.121:0/794635960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09a00b3330 msgr2=0x7f09a00d38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:22.200 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.199+0000 7f09a6800640 1 --2- 172.21.15.121:0/794635960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09a00b3330 0x7f09a00d38e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f09980097a0 tx=0x7f099802b8d0 comp rx=0 tx=0).stop 2024-04-20T10:22:22.200 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.199+0000 7f09a6800640 1 -- 172.21.15.121:0/794635960 shutdown_connections 2024-04-20T10:22:22.200 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.199+0000 7f09a6800640 1 --2- 172.21.15.121:0/794635960 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09a00b3330 0x7f09a00d38e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:22.200 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.199+0000 7f09a6800640 1 -- 172.21.15.121:0/794635960 >> 172.21.15.121:0/794635960 conn(0x7f09a00d4450 msgr2=0x7f09a00d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:22.200 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.199+0000 7f09a6800640 1 -- 172.21.15.121:0/794635960 shutdown_connections 2024-04-20T10:22:22.200 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.199+0000 7f09a6800640 1 -- 172.21.15.121:0/794635960 wait complete. 2024-04-20T10:22:22.201 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.201+0000 7f09a6800640 1 Processor -- start 2024-04-20T10:22:22.201 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.201+0000 7f09a6800640 1 -- start start 2024-04-20T10:22:22.203 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.202+0000 7f09a6800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09a006cde0 0x7f09a015d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:22.203 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.202+0000 7f09a6800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09a00b3330 0x7f09a01e7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:22.203 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.202+0000 7f09a6800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09a0161d50 0x7f09a015f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:22.203 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.202+0000 7f09a6800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f09a00b3880 con 0x7f09a006cde0 2024-04-20T10:22:22.204 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.202+0000 7f09a6800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f09a00d33c0 con 0x7f09a0161d50 2024-04-20T10:22:22.204 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.202+0000 7f09a6800640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f09a00b3a00 con 0x7f09a00b3330 2024-04-20T10:22:22.204 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.202+0000 7f09a5400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09a006cde0 0x7f09a015d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:22.204 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.202+0000 7f09a5400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09a006cde0 0x7f09a015d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55092/0 (socket says 172.21.15.121:55092) 2024-04-20T10:22:22.204 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.202+0000 7f09a5400640 1 -- 172.21.15.121:0/2254702363 learned_addr learned my addr 172.21.15.121:0/2254702363 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:22.204 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.202+0000 7f09a5400640 1 -- 172.21.15.121:0/2254702363 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09a00b3330 msgr2=0x7f09a01e7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:22.204 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.202+0000 7f09a5400640 1 --2- 172.21.15.121:0/2254702363 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09a00b3330 0x7f09a01e7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:22.204 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.202+0000 7f09a5400640 1 -- 172.21.15.121:0/2254702363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09a0161d50 msgr2=0x7f09a015f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:22.204 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.202+0000 7f09a5400640 1 --2- 172.21.15.121:0/2254702363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09a0161d50 0x7f09a015f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:22.204 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.202+0000 7f09a5400640 1 -- 172.21.15.121:0/2254702363 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a01e87f0 con 0x7f09a006cde0 2024-04-20T10:22:22.204 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.203+0000 7f09a5400640 1 --2- 172.21.15.121:0/2254702363 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09a006cde0 0x7f09a015d5a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f099400caa0 tx=0x7f099400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:22.204 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.203+0000 7f099d600640 1 -- 172.21.15.121:0/2254702363 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0994017070 con 0x7f09a006cde0 2024-04-20T10:22:22.204 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.203+0000 7f099d600640 1 -- 172.21.15.121:0/2254702363 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f099400ac00 con 0x7f09a006cde0 2024-04-20T10:22:22.204 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.203+0000 7f099d600640 1 -- 172.21.15.121:0/2254702363 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0994012400 con 0x7f09a006cde0 2024-04-20T10:22:22.205 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.203+0000 7f09a6800640 1 -- 172.21.15.121:0/2254702363 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09a01e8510 con 0x7f09a006cde0 2024-04-20T10:22:22.205 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.203+0000 7f09a6800640 1 -- 172.21.15.121:0/2254702363 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09a00cccf0 con 0x7f09a006cde0 2024-04-20T10:22:22.205 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.204+0000 7f09a6800640 1 -- 172.21.15.121:0/2254702363 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09a0075b60 con 0x7f09a006cde0 2024-04-20T10:22:22.205 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.205+0000 7f099d600640 1 -- 172.21.15.121:0/2254702363 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f099401a070 con 0x7f09a006cde0 2024-04-20T10:22:22.206 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.206+0000 7f099d600640 1 --2- 172.21.15.121:0/2254702363 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f098006c8f0 0x7f098006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:22.206 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.206+0000 7f099d600640 1 -- 172.21.15.121:0/2254702363 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f099408bd30 con 0x7f09a006cde0 2024-04-20T10:22:22.206 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.206+0000 7f09a4a00640 1 --2- 172.21.15.121:0/2254702363 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f098006c8f0 0x7f098006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:22.207 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.207+0000 7f09a4a00640 1 --2- 172.21.15.121:0/2254702363 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f098006c8f0 0x7f098006edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f09a01e7800 tx=0x7f098c0079b0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:22.210 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.210+0000 7f099d600640 1 -- 172.21.15.121:0/2254702363 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0994059cc0 con 0x7f09a006cde0 2024-04-20T10:22:22.393 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.393+0000 7f09a6800640 1 -- 172.21.15.121:0/2254702363 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f09a0161290 con 0x7f09a006cde0 2024-04-20T10:22:22.397 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.396+0000 7f099d600640 1 -- 172.21.15.121:0/2254702363 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18940 (secure 0 0 0) 0x7f09940595c0 con 0x7f09a006cde0 2024-04-20T10:22:22.397 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:22:22.397 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":27,"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","created":"2024-04-20T10:20:38.076076+0000","modified":"2024-04-20T10:21:44.715898+0000","last_up_change":"2024-04-20T10:20:50.162700+0000","last_in_change":"2024-04-20T10:20:41.510382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:20:51.141365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:21:00.436937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:21:01.026957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:21:04.862693+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"629a7b8c-a40a-4a2a-a3cc-d9f357774310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6809","nonce":1044588835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6811","nonce":1044588835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6815","nonce":1044588835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6813","nonce":1044588835}]},"public_addr":"172.21.15.121:6809/1044588835","cluster_addr":"172.21.15.121:6811/1044588835","heartbeat_back_addr":"172.21.15.121:6815/1044588835","heartbeat_front_addr":"172.21.15.121:6813/1044588835","state":["exists","up"]},{"osd":1,"uuid":"f2570399-aa50-4ff3-bf5d-3d09207290db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6801","nonce":3505199624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6803","nonce":3505199624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6807","nonce":3505199624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6805","nonce":3505199624}]},"public_addr":"172.21.15.121:6801/3505199624","cluster_addr":"172.21.15.121:6803/3505199624","heartbeat_back_addr":"172.21.15.121:6807/3505199624","heartbeat_front_addr":"172.21.15.121:6805/3505199624","state":["exists","up"]},{"osd":2,"uuid":"50893807-dc30-49f4-b41f-739366626658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6817","nonce":3656724207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6819","nonce":3656724207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6827","nonce":3656724207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6822","nonce":3656724207}]},"public_addr":"172.21.15.121:6817/3656724207","cluster_addr":"172.21.15.121:6819/3656724207","heartbeat_back_addr":"172.21.15.121:6827/3656724207","heartbeat_front_addr":"172.21.15.121:6822/3656724207","state":["exists","up"]},{"osd":3,"uuid":"dadb01bc-5491-4ffc-8824-d48a057240b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6823","nonce":2232342832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6826","nonce":2232342832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6831","nonce":2232342832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6829","nonce":2232342832}]},"public_addr":"172.21.15.121:6823/2232342832","cluster_addr":"172.21.15.121:6826/2232342832","heartbeat_back_addr":"172.21.15.121:6831/2232342832","heartbeat_front_addr":"172.21.15.121:6829/2232342832","state":["exists","up"]},{"osd":4,"uuid":"b3f3b4a1-b375-4526-b69e-344ab24c2da1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6801","nonce":99353111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6803","nonce":99353111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6807","nonce":99353111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6805","nonce":99353111}]},"public_addr":"172.21.15.123:6801/99353111","cluster_addr":"172.21.15.123:6803/99353111","heartbeat_back_addr":"172.21.15.123:6807/99353111","heartbeat_front_addr":"172.21.15.123:6805/99353111","state":["exists","up"]},{"osd":5,"uuid":"2ee24a08-1624-4961-9642-d7b510282484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6818","nonce":2504374688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6822","nonce":2504374688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6830","nonce":2504374688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6826","nonce":2504374688}]},"public_addr":"172.21.15.123:6818/2504374688","cluster_addr":"172.21.15.123:6822/2504374688","heartbeat_back_addr":"172.21.15.123:6830/2504374688","heartbeat_front_addr":"172.21.15.123:6826/2504374688","state":["exists","up"]},{"osd":6,"uuid":"186118cc-6128-49de-acca-55260945c1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6819","nonce":4139299599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6823","nonce":4139299599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6829","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6831","nonce":4139299599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6827","nonce":4139299599}]},"public_addr":"172.21.15.123:6819/4139299599","cluster_addr":"172.21.15.123:6823/4139299599","heartbeat_back_addr":"172.21.15.123:6831/4139299599","heartbeat_front_addr":"172.21.15.123:6827/4139299599","state":["exists","up"]},{"osd":7,"uuid":"a00aac6e-4bd9-415a-a93d-ae85210865ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6809","nonce":4043981069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6811","nonce":4043981069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6815","nonce":4043981069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6813","nonce":4043981069}]},"public_addr":"172.21.15.123:6809/4043981069","cluster_addr":"172.21.15.123:6811/4043981069","heartbeat_back_addr":"172.21.15.123:6815/4043981069","heartbeat_front_addr":"172.21.15.123:6813/4043981069","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.904540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.958585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.616367+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.885244+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.103127+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.167248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.884348+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.862356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.23","mappings":[{"from":3,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:22:22.399 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.398+0000 7f09a6800640 1 -- 172.21.15.121:0/2254702363 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f098006c8f0 msgr2=0x7f098006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:22.399 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.398+0000 7f09a6800640 1 --2- 172.21.15.121:0/2254702363 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f098006c8f0 0x7f098006edb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f09a01e7800 tx=0x7f098c0079b0 comp rx=0 tx=0).stop 2024-04-20T10:22:22.399 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.398+0000 7f09a6800640 1 -- 172.21.15.121:0/2254702363 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09a006cde0 msgr2=0x7f09a015d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:22.399 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.398+0000 7f09a6800640 1 --2- 172.21.15.121:0/2254702363 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09a006cde0 0x7f09a015d5a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f099400caa0 tx=0x7f099400cf60 comp rx=0 tx=0).stop 2024-04-20T10:22:22.399 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.399+0000 7f09a6800640 1 -- 172.21.15.121:0/2254702363 shutdown_connections 2024-04-20T10:22:22.400 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.399+0000 7f09a6800640 1 --2- 172.21.15.121:0/2254702363 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f098006c8f0 0x7f098006edb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:22.400 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.399+0000 7f09a6800640 1 --2- 172.21.15.121:0/2254702363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09a0161d50 0x7f09a015f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:22.400 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.399+0000 7f09a6800640 1 --2- 172.21.15.121:0/2254702363 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f09a00b3330 0x7f09a01e7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:22.400 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.399+0000 7f09a6800640 1 --2- 172.21.15.121:0/2254702363 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f09a006cde0 0x7f09a015d5a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:22.400 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.399+0000 7f09a6800640 1 -- 172.21.15.121:0/2254702363 >> 172.21.15.121:0/2254702363 conn(0x7f09a00d4450 msgr2=0x7f09a0160350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:22.400 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.399+0000 7f09a6800640 1 -- 172.21.15.121:0/2254702363 shutdown_connections 2024-04-20T10:22:22.400 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.399+0000 7f09a6800640 1 -- 172.21.15.121:0/2254702363 wait complete. 2024-04-20T10:22:22.416 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:20:51.141365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:21:00.436937+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:21:01.026957+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T10:21:04.862693+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T10:22:22.417 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:22:22.604 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.604+0000 7f5293c00640 1 Processor -- start 2024-04-20T10:22:22.605 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.604+0000 7f5293c00640 1 -- start start 2024-04-20T10:22:22.606 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.605+0000 7f5293c00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f528c156c20 0x7f528c06b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:22.606 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.605+0000 7f5293c00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f528c141c60 con 0x7f528c0d0360 2024-04-20T10:22:22.606 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.605+0000 7f5293c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f528c0adab0 con 0x7f528c15ca40 2024-04-20T10:22:22.606 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.605+0000 7f5293c00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f528c15cf90 con 0x7f528c156c20 2024-04-20T10:22:22.606 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.606+0000 7f5291c00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f528c0d0360 0x7f528c142140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:53464/0 (socket says 172.21.15.121:53464) 2024-04-20T10:22:22.606 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.606+0000 7f5291c00640 1 -- 172.21.15.121:0/3184693419 learned_addr learned my addr 172.21.15.121:0/3184693419 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:22.608 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.606+0000 7f5291200640 1 --2- 172.21.15.121:0/3184693419 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f528c156c20 0x7f528c06b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:22.608 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.606+0000 7f528b400640 1 -- 172.21.15.121:0/3184693419 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2243287408 0 0) 0x7f528c141c60 con 0x7f528c0d0360 2024-04-20T10:22:22.608 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.606+0000 7f528b400640 1 -- 172.21.15.121:0/3184693419 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5274003520 con 0x7f528c0d0360 2024-04-20T10:22:22.608 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.607+0000 7f528b400640 1 -- 172.21.15.121:0/3184693419 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1874994790 0 0) 0x7f5274003520 con 0x7f528c0d0360 2024-04-20T10:22:22.608 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.607+0000 7f528b400640 1 -- 172.21.15.121:0/3184693419 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f528c156c20 msgr2=0x7f528c06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:22.609 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.607+0000 7f528b400640 1 --2- 172.21.15.121:0/3184693419 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f528c156c20 0x7f528c06b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:22.609 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.607+0000 7f528b400640 1 -- 172.21.15.121:0/3184693419 >> v1:172.21.15.123:6789/0 conn(0x7f528c15ca40 legacy=0x7f528c0ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:22.609 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.607+0000 7f528b400640 1 -- 172.21.15.121:0/3184693419 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f528c15d270 con 0x7f528c0d0360 2024-04-20T10:22:22.609 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.607+0000 7f528b400640 1 -- 172.21.15.121:0/3184693419 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f52840019f0 con 0x7f528c0d0360 2024-04-20T10:22:22.609 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.607+0000 7f528b400640 1 -- 172.21.15.121:0/3184693419 >> v1:172.21.15.121:6789/0 conn(0x7f528c0d0360 legacy=0x7f528c142140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:22.609 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.607+0000 7f528b400640 1 --2- 172.21.15.121:0/3184693419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52740039e0 0x7f5274003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:22.609 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.607+0000 7f528b400640 1 --2- 172.21.15.121:0/3184693419 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f528c156c20 0x7f5274004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:22.609 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.607+0000 7f528b400640 1 --2- 172.21.15.121:0/3184693419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5274004d40 0x7f52740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:22.609 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.607+0000 7f528b400640 1 -- 172.21.15.121:0/3184693419 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f528c15cf90 con 0x7f5274004d40 2024-04-20T10:22:22.609 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.607+0000 7f528b400640 1 -- 172.21.15.121:0/3184693419 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f527400ba70 con 0x7f52740039e0 2024-04-20T10:22:22.609 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.607+0000 7f528b400640 1 -- 172.21.15.121:0/3184693419 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f527400bc70 con 0x7f528c156c20 2024-04-20T10:22:22.609 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.607+0000 7f5291c00640 1 --2- 172.21.15.121:0/3184693419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5274004d40 0x7f52740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:22.609 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.607+0000 7f528be00640 1 --2- 172.21.15.121:0/3184693419 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f528c156c20 0x7f5274004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:22.609 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.607+0000 7f5291200640 1 --2- 172.21.15.121:0/3184693419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52740039e0 0x7f5274003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:22.610 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.608+0000 7f5291c00640 1 -- 172.21.15.121:0/3184693419 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f528c156c20 msgr2=0x7f5274004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:22.610 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.608+0000 7f5291c00640 1 --2- 172.21.15.121:0/3184693419 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f528c156c20 0x7f5274004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:22.610 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.608+0000 7f5291c00640 1 -- 172.21.15.121:0/3184693419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52740039e0 msgr2=0x7f5274003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:22.610 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.608+0000 7f5291c00640 1 --2- 172.21.15.121:0/3184693419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52740039e0 0x7f5274003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:22.610 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.608+0000 7f5291c00640 1 -- 172.21.15.121:0/3184693419 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5274003520 con 0x7f5274004d40 2024-04-20T10:22:22.610 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.608+0000 7f5291200640 1 --2- 172.21.15.121:0/3184693419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52740039e0 0x7f5274003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:22.611 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.609+0000 7f5291c00640 1 --2- 172.21.15.121:0/3184693419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5274004d40 0x7f52740051c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5284002230 tx=0x7f5274005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96eb5e3a1143511d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:22.611 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.609+0000 7f528b400640 1 -- 172.21.15.121:0/3184693419 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5284038020 con 0x7f5274004d40 2024-04-20T10:22:22.611 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.609+0000 7f528b400640 1 -- 172.21.15.121:0/3184693419 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5284034690 con 0x7f5274004d40 2024-04-20T10:22:22.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.609+0000 7f528b400640 1 -- 172.21.15.121:0/3184693419 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528400a4a0 con 0x7f5274004d40 2024-04-20T10:22:22.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.609+0000 7f5293c00640 1 -- 172.21.15.121:0/3184693419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5274004d40 msgr2=0x7f52740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:22.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.609+0000 7f5293c00640 1 --2- 172.21.15.121:0/3184693419 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5274004d40 0x7f52740051c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5284002230 tx=0x7f5274005830 comp rx=0 tx=0).stop 2024-04-20T10:22:22.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.609+0000 7f5291c00640 1 -- 172.21.15.121:0/3184693419 reap_dead start 2024-04-20T10:22:22.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.609+0000 7f5293c00640 1 -- 172.21.15.121:0/3184693419 shutdown_connections 2024-04-20T10:22:22.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.609+0000 7f5293c00640 1 -- 172.21.15.121:0/3184693419 >> 172.21.15.121:0/3184693419 conn(0x7f528c0b3350 msgr2=0x7f528c0b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:22.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.610+0000 7f5293c00640 1 -- 172.21.15.121:0/3184693419 shutdown_connections 2024-04-20T10:22:22.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.610+0000 7f5293c00640 1 -- 172.21.15.121:0/3184693419 wait complete. 2024-04-20T10:22:22.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.611+0000 7f5293c00640 1 Processor -- start 2024-04-20T10:22:22.614 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.611+0000 7f5293c00640 1 -- start start 2024-04-20T10:22:22.614 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.612+0000 7f5293c00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5274004d40 0x7f528c1f3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:22.615 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.612+0000 7f5293c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f528c073ae0 0x7f528c0ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:22.615 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.612+0000 7f5293c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f528c15de50 0x7f528c0cd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:22.615 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.612+0000 7f5293c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f528c142850 con 0x7f528c15de50 2024-04-20T10:22:22.615 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.612+0000 7f5293c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f528c15d830 con 0x7f528c073ae0 2024-04-20T10:22:22.615 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.612+0000 7f5293c00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f528c1429d0 con 0x7f5274004d40 2024-04-20T10:22:22.615 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.613+0000 7f5291c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f528c15de50 0x7f528c0cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:22.615 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.613+0000 7f5291c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f528c15de50 0x7f528c0cd070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55104/0 (socket says 172.21.15.121:55104) 2024-04-20T10:22:22.615 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.613+0000 7f5291c00640 1 -- 172.21.15.121:0/3579124449 learned_addr learned my addr 172.21.15.121:0/3579124449 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:22.615 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.613+0000 7f5291200640 1 --2- 172.21.15.121:0/3579124449 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5274004d40 0x7f528c1f3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:22.615 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.613+0000 7f528be00640 1 --2- 172.21.15.121:0/3579124449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f528c073ae0 0x7f528c0ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:22.615 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.613+0000 7f5291c00640 1 -- 172.21.15.121:0/3579124449 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5274004d40 msgr2=0x7f528c1f3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:22.615 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.613+0000 7f5291c00640 1 --2- 172.21.15.121:0/3579124449 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5274004d40 0x7f528c1f3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:22.615 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.613+0000 7f5291c00640 1 -- 172.21.15.121:0/3579124449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f528c073ae0 msgr2=0x7f528c0ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:22.615 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.613+0000 7f5291c00640 1 --2- 172.21.15.121:0/3579124449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f528c073ae0 0x7f528c0ccb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:22.616 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.613+0000 7f5291c00640 1 -- 172.21.15.121:0/3579124449 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f528c1f3b70 con 0x7f528c15de50 2024-04-20T10:22:22.616 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.613+0000 7f528be00640 1 --2- 172.21.15.121:0/3579124449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f528c073ae0 0x7f528c0ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:22.616 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.613+0000 7f5291c00640 1 --2- 172.21.15.121:0/3579124449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f528c15de50 0x7f528c0cd070 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f52840019e0 tx=0x7f5284001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:22.616 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.614+0000 7f5288c00640 1 -- 172.21.15.121:0/3579124449 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5284038030 con 0x7f528c15de50 2024-04-20T10:22:22.616 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.614+0000 7f5288c00640 1 -- 172.21.15.121:0/3579124449 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5284001250 con 0x7f528c15de50 2024-04-20T10:22:22.616 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.614+0000 7f5288c00640 1 -- 172.21.15.121:0/3579124449 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f528400a480 con 0x7f528c15de50 2024-04-20T10:22:22.616 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.614+0000 7f5293c00640 1 -- 172.21.15.121:0/3579124449 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f528c0728d0 con 0x7f528c15de50 2024-04-20T10:22:22.616 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.614+0000 7f5293c00640 1 -- 172.21.15.121:0/3579124449 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f528c072e90 con 0x7f528c15de50 2024-04-20T10:22:22.617 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.615+0000 7f5293c00640 1 -- 172.21.15.121:0/3579124449 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f528c068980 con 0x7f528c15de50 2024-04-20T10:22:22.617 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.616+0000 7f5288c00640 1 -- 172.21.15.121:0/3579124449 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f528403b020 con 0x7f528c15de50 2024-04-20T10:22:22.617 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.617+0000 7f5288c00640 1 --2- 172.21.15.121:0/3579124449 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f526c06c920 0x7f526c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:22.617 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.617+0000 7f5288c00640 1 -- 172.21.15.121:0/3579124449 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f52840b18b0 con 0x7f528c15de50 2024-04-20T10:22:22.617 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.617+0000 7f5291200640 1 --2- 172.21.15.121:0/3579124449 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f526c06c920 0x7f526c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:22.618 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.617+0000 7f5291200640 1 --2- 172.21.15.121:0/3579124449 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f526c06c920 0x7f526c06ede0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5280009aa0 tx=0x7f5280008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:22.621 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.621+0000 7f5288c00640 1 -- 172.21.15.121:0/3579124449 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f528407f1f0 con 0x7f528c15de50 2024-04-20T10:22:22.807 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.807+0000 7f5293c00640 1 -- 172.21.15.121:0/3579124449 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f528c0cfd90 con 0x7f528c15de50 2024-04-20T10:22:22.808 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.807+0000 7f5288c00640 1 -- 172.21.15.121:0/3579124449 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f52840834f0 con 0x7f528c15de50 2024-04-20T10:22:22.808 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2024-04-20T10:22:22.810 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.810+0000 7f5293c00640 1 -- 172.21.15.121:0/3579124449 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f526c06c920 msgr2=0x7f526c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:22.810 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.810+0000 7f5293c00640 1 --2- 172.21.15.121:0/3579124449 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f526c06c920 0x7f526c06ede0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5280009aa0 tx=0x7f5280008040 comp rx=0 tx=0).stop 2024-04-20T10:22:22.810 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.810+0000 7f5293c00640 1 -- 172.21.15.121:0/3579124449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f528c15de50 msgr2=0x7f528c0cd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:22.811 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.810+0000 7f5293c00640 1 --2- 172.21.15.121:0/3579124449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f528c15de50 0x7f528c0cd070 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f52840019e0 tx=0x7f5284001dc0 comp rx=0 tx=0).stop 2024-04-20T10:22:22.811 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.810+0000 7f5293c00640 1 -- 172.21.15.121:0/3579124449 shutdown_connections 2024-04-20T10:22:22.811 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.810+0000 7f5293c00640 1 --2- 172.21.15.121:0/3579124449 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f528c15de50 0x7f528c0cd070 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:22.811 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.810+0000 7f5293c00640 1 --2- 172.21.15.121:0/3579124449 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f526c06c920 0x7f526c06ede0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:22.811 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.810+0000 7f5293c00640 1 --2- 172.21.15.121:0/3579124449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f528c073ae0 0x7f528c0ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:22.811 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.810+0000 7f5293c00640 1 --2- 172.21.15.121:0/3579124449 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5274004d40 0x7f528c1f3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:22.811 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.810+0000 7f5293c00640 1 -- 172.21.15.121:0/3579124449 >> 172.21.15.121:0/3579124449 conn(0x7f528c0b3350 msgr2=0x7f528c06ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:22.811 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.810+0000 7f5293c00640 1 -- 172.21.15.121:0/3579124449 shutdown_connections 2024-04-20T10:22:22.811 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:22.810+0000 7f5293c00640 1 -- 172.21.15.121:0/3579124449 wait complete. 2024-04-20T10:22:22.829 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:22:23.027 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.027+0000 7f39f7000640 1 Processor -- start 2024-04-20T10:22:23.028 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.027+0000 7f39f7000640 1 -- start start 2024-04-20T10:22:23.029 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.028+0000 7f39f7000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f39f006e2a0 0x7f39f015eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:23.029 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.028+0000 7f39f7000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39f015f960 con 0x7f39f0158ca0 2024-04-20T10:22:23.029 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.029+0000 7f39f7000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39f006b9a0 con 0x7f39f015f0e0 2024-04-20T10:22:23.030 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.029+0000 7f39f7000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f39f006bc80 con 0x7f39f006e2a0 2024-04-20T10:22:23.030 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.029+0000 7f39f5200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f39f006e2a0 0x7f39f015eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:23.030 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.029+0000 7f39f5c00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f39f0158ca0 0x7f39f006db90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:53480/0 (socket says 172.21.15.121:53480) 2024-04-20T10:22:23.030 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.029+0000 7f39f5200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f39f006e2a0 0x7f39f015eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55108/0 (socket says 172.21.15.121:55108) 2024-04-20T10:22:23.030 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.029+0000 7f39f5c00640 1 -- 172.21.15.121:0/3506310045 learned_addr learned my addr 172.21.15.121:0/3506310045 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:23.031 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.029+0000 7f39f5200640 1 -- 172.21.15.121:0/3506310045 >> v1:172.21.15.123:6789/0 conn(0x7f39f015f0e0 legacy=0x7f39f01460f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:23.031 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.029+0000 7f39f5200640 1 -- 172.21.15.121:0/3506310045 >> v1:172.21.15.121:6789/0 conn(0x7f39f0158ca0 legacy=0x7f39f006db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:23.031 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.029+0000 7f39f5200640 1 -- 172.21.15.121:0/3506310045 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39f006bf60 con 0x7f39f006e2a0 2024-04-20T10:22:23.031 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.030+0000 7f39f5200640 1 --2- 172.21.15.121:0/3506310045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39f006e2a0 0x7f39f015eba0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f39dc0097a0 tx=0x7f39dc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447b5d7eec81cd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:23.031 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.031+0000 7f39efe00640 1 -- 172.21.15.121:0/3506310045 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39dc035020 con 0x7f39f006e2a0 2024-04-20T10:22:23.032 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.031+0000 7f39efe00640 1 -- 172.21.15.121:0/3506310045 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39dc031830 con 0x7f39f006e2a0 2024-04-20T10:22:23.032 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.031+0000 7f39efe00640 1 -- 172.21.15.121:0/3506310045 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39dc031c00 con 0x7f39f006e2a0 2024-04-20T10:22:23.032 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.031+0000 7f39f7000640 1 -- 172.21.15.121:0/3506310045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39f006e2a0 msgr2=0x7f39f015eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:23.032 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.031+0000 7f39f7000640 1 --2- 172.21.15.121:0/3506310045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39f006e2a0 0x7f39f015eba0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f39dc0097a0 tx=0x7f39dc02b8d0 comp rx=0 tx=0).stop 2024-04-20T10:22:23.032 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.032+0000 7f39f7000640 1 -- 172.21.15.121:0/3506310045 shutdown_connections 2024-04-20T10:22:23.033 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.032+0000 7f39f7000640 1 --2- 172.21.15.121:0/3506310045 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39f006e2a0 0x7f39f015eba0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:23.033 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.032+0000 7f39f7000640 1 -- 172.21.15.121:0/3506310045 >> 172.21.15.121:0/3506310045 conn(0x7f39f00b5270 msgr2=0x7f39f00b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:23.033 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.032+0000 7f39f7000640 1 -- 172.21.15.121:0/3506310045 shutdown_connections 2024-04-20T10:22:23.033 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.032+0000 7f39f7000640 1 -- 172.21.15.121:0/3506310045 wait complete. 2024-04-20T10:22:23.034 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.034+0000 7f39f7000640 1 Processor -- start 2024-04-20T10:22:23.034 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.034+0000 7f39f7000640 1 -- start start 2024-04-20T10:22:23.036 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.035+0000 7f39f7000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39f006e2a0 0x7f39f0099340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:23.036 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.035+0000 7f39f7000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39f0158ca0 0x7f39f0152470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:23.036 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.035+0000 7f39f7000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f39f015f0e0 0x7f39f0152e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:23.036 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.035+0000 7f39f7000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f39f006c760 con 0x7f39f0158ca0 2024-04-20T10:22:23.036 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.035+0000 7f39f7000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f39f015f510 con 0x7f39f006e2a0 2024-04-20T10:22:23.036 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.035+0000 7f39f7000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f39f00afe20 con 0x7f39f015f0e0 2024-04-20T10:22:23.036 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.035+0000 7f39f5200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39f0158ca0 0x7f39f0152470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:23.036 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.035+0000 7f39f5200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39f0158ca0 0x7f39f0152470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55124/0 (socket says 172.21.15.121:55124) 2024-04-20T10:22:23.036 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.035+0000 7f39f5200640 1 -- 172.21.15.121:0/3628027473 learned_addr learned my addr 172.21.15.121:0/3628027473 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:23.039 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.036+0000 7f39f5200640 1 -- 172.21.15.121:0/3628027473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f39f015f0e0 msgr2=0x7f39f0152e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:23.039 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.036+0000 7f39f5200640 1 --2- 172.21.15.121:0/3628027473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f39f015f0e0 0x7f39f0152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:23.039 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.036+0000 7f39f5c00640 1 --2- 172.21.15.121:0/3628027473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39f006e2a0 0x7f39f0099340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:23.039 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.036+0000 7f39f5200640 1 -- 172.21.15.121:0/3628027473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39f006e2a0 msgr2=0x7f39f0099340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:23.039 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.036+0000 7f39f5200640 1 --2- 172.21.15.121:0/3628027473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39f006e2a0 0x7f39f0099340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:23.039 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.036+0000 7f39f5200640 1 -- 172.21.15.121:0/3628027473 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39f00b0100 con 0x7f39f0158ca0 2024-04-20T10:22:23.039 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.036+0000 7f39f5200640 1 --2- 172.21.15.121:0/3628027473 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39f0158ca0 0x7f39f0152470 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f39dc02bee0 tx=0x7f39dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:23.039 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.036+0000 7f39ed600640 1 -- 172.21.15.121:0/3628027473 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39dc041070 con 0x7f39f0158ca0 2024-04-20T10:22:23.039 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.036+0000 7f39ed600640 1 -- 172.21.15.121:0/3628027473 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39dc03aa30 con 0x7f39f0158ca0 2024-04-20T10:22:23.039 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.036+0000 7f39ed600640 1 -- 172.21.15.121:0/3628027473 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39dc035040 con 0x7f39f0158ca0 2024-04-20T10:22:23.039 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.037+0000 7f39f7000640 1 -- 172.21.15.121:0/3628027473 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39f00b03e0 con 0x7f39f0158ca0 2024-04-20T10:22:23.039 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.037+0000 7f39f7000640 1 -- 172.21.15.121:0/3628027473 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39f00b09a0 con 0x7f39f0158ca0 2024-04-20T10:22:23.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.038+0000 7f39f7000640 1 -- 172.21.15.121:0/3628027473 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39f0075b30 con 0x7f39f0158ca0 2024-04-20T10:22:23.040 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.039+0000 7f39ed600640 1 -- 172.21.15.121:0/3628027473 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f39dc038020 con 0x7f39f0158ca0 2024-04-20T10:22:23.041 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.040+0000 7f39ed600640 1 --2- 172.21.15.121:0/3628027473 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f39d006c8f0 0x7f39d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:23.041 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.041+0000 7f39ed600640 1 -- 172.21.15.121:0/3628027473 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f39dc0adb70 con 0x7f39f0158ca0 2024-04-20T10:22:23.042 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.041+0000 7f39f5c00640 1 --2- 172.21.15.121:0/3628027473 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f39d006c8f0 0x7f39d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:23.042 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.041+0000 7f39f5c00640 1 --2- 172.21.15.121:0/3628027473 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f39d006c8f0 0x7f39d006edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f39e4000f60 tx=0x7f39e4009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:23.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.047+0000 7f39ed600640 1 -- 172.21.15.121:0/3628027473 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f39dc07bb00 con 0x7f39f0158ca0 2024-04-20T10:22:23.250 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.249+0000 7f39f7000640 1 -- 172.21.15.121:0/3628027473 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f39f00b2530 con 0x7f39f0158ca0 2024-04-20T10:22:23.251 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.250+0000 7f39ed600640 1 -- 172.21.15.121:0/3628027473 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f39dc07b400 con 0x7f39f0158ca0 2024-04-20T10:22:23.251 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 32 2024-04-20T10:22:23.254 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.253+0000 7f39f7000640 1 -- 172.21.15.121:0/3628027473 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f39d006c8f0 msgr2=0x7f39d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:23.254 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.253+0000 7f39f7000640 1 --2- 172.21.15.121:0/3628027473 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f39d006c8f0 0x7f39d006edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f39e4000f60 tx=0x7f39e4009040 comp rx=0 tx=0).stop 2024-04-20T10:22:23.254 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.253+0000 7f39f7000640 1 -- 172.21.15.121:0/3628027473 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39f0158ca0 msgr2=0x7f39f0152470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:23.254 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.253+0000 7f39f7000640 1 --2- 172.21.15.121:0/3628027473 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39f0158ca0 0x7f39f0152470 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f39dc02bee0 tx=0x7f39dc0046c0 comp rx=0 tx=0).stop 2024-04-20T10:22:23.254 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.253+0000 7f39f7000640 1 -- 172.21.15.121:0/3628027473 shutdown_connections 2024-04-20T10:22:23.254 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.253+0000 7f39f7000640 1 --2- 172.21.15.121:0/3628027473 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f39f015f0e0 0x7f39f0152e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:23.254 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.253+0000 7f39f7000640 1 --2- 172.21.15.121:0/3628027473 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f39f0158ca0 0x7f39f0152470 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:23.254 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.253+0000 7f39f7000640 1 --2- 172.21.15.121:0/3628027473 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f39d006c8f0 0x7f39d006edb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:23.254 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.253+0000 7f39f7000640 1 --2- 172.21.15.121:0/3628027473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39f006e2a0 0x7f39f0099340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:23.254 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.253+0000 7f39f7000640 1 -- 172.21.15.121:0/3628027473 >> 172.21.15.121:0/3628027473 conn(0x7f39f00b5270 msgr2=0x7f39f00ad3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:23.255 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.253+0000 7f39f7000640 1 -- 172.21.15.121:0/3628027473 shutdown_connections 2024-04-20T10:22:23.255 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.253+0000 7f39f7000640 1 -- 172.21.15.121:0/3628027473 wait complete. 2024-04-20T10:22:23.273 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:22:23.480 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.479+0000 7ffba9800640 1 Processor -- start 2024-04-20T10:22:23.480 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.479+0000 7ffba9800640 1 -- start start 2024-04-20T10:22:23.482 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.480+0000 7ffba9800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffba415b3b0 0x7ffba406eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:23.482 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.480+0000 7ffba9800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ffba40d0360 0x7ffba40d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:23.482 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.480+0000 7ffba9800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffba40ade40 con 0x7ffba406dcb0 2024-04-20T10:22:23.482 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.480+0000 7ffba9800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7ffba40ae0e0 con 0x7ffba40d0360 2024-04-20T10:22:23.482 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.480+0000 7ffba9800640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ffba40d0cf0 con 0x7ffba415b3b0 2024-04-20T10:22:23.482 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.481+0000 7ffba3400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ffba40d0360 0x7ffba40d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:23.482 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.481+0000 7ffba3400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ffba40d0360 0x7ffba40d07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55136/0 (socket says 172.21.15.121:55136) 2024-04-20T10:22:23.482 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.481+0000 7ffba3400640 1 -- 172.21.15.121:0/3597136269 learned_addr learned my addr 172.21.15.121:0/3597136269 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:23.482 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.481+0000 7ffba2a00640 1 --2- 172.21.15.121:0/3597136269 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffba415b3b0 0x7ffba406eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:23.483 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.481+0000 7ffba3400640 1 -- 172.21.15.121:0/3597136269 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffba415b3b0 msgr2=0x7ffba406eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:23.483 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.481+0000 7ffba3400640 1 --2- 172.21.15.121:0/3597136269 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffba415b3b0 0x7ffba406eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:23.483 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.481+0000 7ffba3400640 1 -- 172.21.15.121:0/3597136269 >> v1:172.21.15.123:6789/0 conn(0x7ffba406dcb0 legacy=0x7ffba4071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:23.483 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.481+0000 7ffba3400640 1 -- 172.21.15.121:0/3597136269 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba415b790 con 0x7ffba40d0360 2024-04-20T10:22:23.483 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.483+0000 7ffba3400640 1 --2- 172.21.15.121:0/3597136269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffba40d0360 0x7ffba40d07b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c009e50 tx=0x7ffb9c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2afe1264710df2c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:23.483 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.483+0000 7ffba1600640 1 -- 172.21.15.121:0/3597136269 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c02ca70 con 0x7ffba40d0360 2024-04-20T10:22:23.484 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.483+0000 7ffba1600640 1 -- 172.21.15.121:0/3597136269 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb9c02cc10 con 0x7ffba40d0360 2024-04-20T10:22:23.484 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.483+0000 7ffba1600640 1 -- 172.21.15.121:0/3597136269 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9c0343f0 con 0x7ffba40d0360 2024-04-20T10:22:23.484 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.484+0000 7ffba9800640 1 -- 172.21.15.121:0/3597136269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffba40d0360 msgr2=0x7ffba40d07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:23.484 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.484+0000 7ffba9800640 1 --2- 172.21.15.121:0/3597136269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffba40d0360 0x7ffba40d07b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c009e50 tx=0x7ffb9c02bed0 comp rx=0 tx=0).stop 2024-04-20T10:22:23.484 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.484+0000 7ffba9800640 1 -- 172.21.15.121:0/3597136269 shutdown_connections 2024-04-20T10:22:23.485 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.484+0000 7ffba9800640 1 --2- 172.21.15.121:0/3597136269 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffba40d0360 0x7ffba40d07b0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:23.485 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.484+0000 7ffba9800640 1 --2- 172.21.15.121:0/3597136269 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffba415b3b0 0x7ffba406eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:23.485 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.484+0000 7ffba9800640 1 -- 172.21.15.121:0/3597136269 >> 172.21.15.121:0/3597136269 conn(0x7ffba40b3350 msgr2=0x7ffba40b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:23.485 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.484+0000 7ffba9800640 1 -- 172.21.15.121:0/3597136269 shutdown_connections 2024-04-20T10:22:23.485 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.484+0000 7ffba9800640 1 -- 172.21.15.121:0/3597136269 wait complete. 2024-04-20T10:22:23.486 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.485+0000 7ffba9800640 1 Processor -- start 2024-04-20T10:22:23.486 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.486+0000 7ffba9800640 1 -- start start 2024-04-20T10:22:23.489 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.486+0000 7ffba9800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffba406dcb0 0x7ffba409dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:23.489 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.487+0000 7ffba9800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffba40d0360 0x7ffba415cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:23.489 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.487+0000 7ffba9800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffba415b3b0 0x7ffba415ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:23.489 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.487+0000 7ffba9800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ffba40ae660 con 0x7ffba40d0360 2024-04-20T10:22:23.489 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.487+0000 7ffba9800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffba40ae380 con 0x7ffba415b3b0 2024-04-20T10:22:23.489 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.487+0000 7ffba9800640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ffba40ae940 con 0x7ffba406dcb0 2024-04-20T10:22:23.489 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.487+0000 7ffba2000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffba40d0360 0x7ffba415cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:23.489 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.487+0000 7ffba2000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffba40d0360 0x7ffba415cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55146/0 (socket says 172.21.15.121:55146) 2024-04-20T10:22:23.489 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.487+0000 7ffba2000640 1 -- 172.21.15.121:0/393214738 learned_addr learned my addr 172.21.15.121:0/393214738 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:23.489 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.487+0000 7ffba2a00640 1 --2- 172.21.15.121:0/393214738 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffba406dcb0 0x7ffba409dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:23.489 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.487+0000 7ffba3400640 1 --2- 172.21.15.121:0/393214738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffba415b3b0 0x7ffba415ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:23.490 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.487+0000 7ffba2000640 1 -- 172.21.15.121:0/393214738 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffba406dcb0 msgr2=0x7ffba409dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:23.490 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.487+0000 7ffba2000640 1 --2- 172.21.15.121:0/393214738 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffba406dcb0 0x7ffba409dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:23.490 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.487+0000 7ffba2000640 1 -- 172.21.15.121:0/393214738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffba415b3b0 msgr2=0x7ffba415ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:23.490 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.487+0000 7ffba2000640 1 --2- 172.21.15.121:0/393214738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffba415b3b0 0x7ffba415ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:23.490 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.488+0000 7ffba2000640 1 -- 172.21.15.121:0/393214738 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba415d660 con 0x7ffba40d0360 2024-04-20T10:22:23.490 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.488+0000 7ffba2000640 1 --2- 172.21.15.121:0/393214738 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffba40d0360 0x7ffba415cf30 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ffb9000b420 tx=0x7ffb9000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:23.490 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.488+0000 7ffb9aa00640 1 -- 172.21.15.121:0/393214738 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb90011830 con 0x7ffba40d0360 2024-04-20T10:22:23.490 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.488+0000 7ffb9aa00640 1 -- 172.21.15.121:0/393214738 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb90016070 con 0x7ffba40d0360 2024-04-20T10:22:23.490 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.488+0000 7ffb9aa00640 1 -- 172.21.15.121:0/393214738 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb90011c10 con 0x7ffba40d0360 2024-04-20T10:22:23.490 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.488+0000 7ffba9800640 1 -- 172.21.15.121:0/393214738 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffba415d940 con 0x7ffba40d0360 2024-04-20T10:22:23.490 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.488+0000 7ffba9800640 1 -- 172.21.15.121:0/393214738 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffba415df00 con 0x7ffba40d0360 2024-04-20T10:22:23.491 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.489+0000 7ffba9800640 1 -- 172.21.15.121:0/393214738 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffba40d09c0 con 0x7ffba40d0360 2024-04-20T10:22:23.491 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.490+0000 7ffb9aa00640 1 -- 172.21.15.121:0/393214738 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffb9001b460 con 0x7ffba40d0360 2024-04-20T10:22:23.492 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.491+0000 7ffb9aa00640 1 --2- 172.21.15.121:0/393214738 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ffb8406c8f0 0x7ffb8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:23.492 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.491+0000 7ffb9aa00640 1 -- 172.21.15.121:0/393214738 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ffb9008ba90 con 0x7ffba40d0360 2024-04-20T10:22:23.492 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.492+0000 7ffba2a00640 1 --2- 172.21.15.121:0/393214738 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ffb8406c8f0 0x7ffb8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:23.493 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.492+0000 7ffba2a00640 1 --2- 172.21.15.121:0/393214738 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ffb8406c8f0 0x7ffb8406edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ffb9400a970 tx=0x7ffb94008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:23.496 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.496+0000 7ffb9aa00640 1 -- 172.21.15.121:0/393214738 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ffba40d09c0 con 0x7ffba40d0360 2024-04-20T10:22:23.696 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.695+0000 7ffba9800640 1 -- 172.21.15.121:0/393214738 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ffba4068980 con 0x7ffba40d0360 2024-04-20T10:22:23.697 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.696+0000 7ffb9aa00640 1 -- 172.21.15.121:0/393214738 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7ffb90020020 con 0x7ffba40d0360 2024-04-20T10:22:23.697 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2024-04-20T10:22:23.700 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.699+0000 7ffba9800640 1 -- 172.21.15.121:0/393214738 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ffb8406c8f0 msgr2=0x7ffb8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:23.700 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.699+0000 7ffba9800640 1 --2- 172.21.15.121:0/393214738 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ffb8406c8f0 0x7ffb8406edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ffb9400a970 tx=0x7ffb94008040 comp rx=0 tx=0).stop 2024-04-20T10:22:23.700 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.699+0000 7ffba9800640 1 -- 172.21.15.121:0/393214738 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffba40d0360 msgr2=0x7ffba415cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:23.700 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.699+0000 7ffba9800640 1 --2- 172.21.15.121:0/393214738 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffba40d0360 0x7ffba415cf30 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ffb9000b420 tx=0x7ffb9000b8e0 comp rx=0 tx=0).stop 2024-04-20T10:22:23.700 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.699+0000 7ffba9800640 1 -- 172.21.15.121:0/393214738 shutdown_connections 2024-04-20T10:22:23.700 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.699+0000 7ffba9800640 1 --2- 172.21.15.121:0/393214738 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ffb8406c8f0 0x7ffb8406edb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:23.700 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.699+0000 7ffba9800640 1 --2- 172.21.15.121:0/393214738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffba415b3b0 0x7ffba415ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:23.701 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.699+0000 7ffba9800640 1 --2- 172.21.15.121:0/393214738 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ffba40d0360 0x7ffba415cf30 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:23.701 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.699+0000 7ffba9800640 1 --2- 172.21.15.121:0/393214738 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ffba406dcb0 0x7ffba409dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:23.701 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.699+0000 7ffba9800640 1 -- 172.21.15.121:0/393214738 >> 172.21.15.121:0/393214738 conn(0x7ffba40b3350 msgr2=0x7ffba406ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:23.701 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.700+0000 7ffba9800640 1 -- 172.21.15.121:0/393214738 shutdown_connections 2024-04-20T10:22:23.701 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.700+0000 7ffba9800640 1 -- 172.21.15.121:0/393214738 wait complete. 2024-04-20T10:22:23.719 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T10:22:23.921 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.920+0000 7f7734400640 1 Processor -- start 2024-04-20T10:22:23.921 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.920+0000 7f7734400640 1 -- start start 2024-04-20T10:22:23.922 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.922+0000 7f7734400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f772c15ca40 0x7f772c0ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:23.922 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.922+0000 7f7734400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f772c0ad8e0 0x7f772c0d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:23.922 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.922+0000 7f7734400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f772c141c60 con 0x7f772c156c20 2024-04-20T10:22:23.923 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.922+0000 7f7734400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f772c06c010 con 0x7f772c15ca40 2024-04-20T10:22:23.923 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.922+0000 7f7734400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f772c15cf90 con 0x7f772c0ad8e0 2024-04-20T10:22:23.923 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.922+0000 7f7731000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f772c15ca40 0x7f772c0ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:23.923 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.922+0000 7f7731000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f772c15ca40 0x7f772c0ad3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55154/0 (socket says 172.21.15.121:55154) 2024-04-20T10:22:23.923 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.922+0000 7f7731000640 1 -- 172.21.15.121:0/419922759 learned_addr learned my addr 172.21.15.121:0/419922759 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:23.923 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.922+0000 7f7731a00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f772c156c20 0x7f772c06b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:53496/0 (socket says 172.21.15.121:53496) 2024-04-20T10:22:23.924 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.922+0000 7f7731000640 1 -- 172.21.15.121:0/419922759 >> v2:172.21.15.123:3300/0 conn(0x7f772c0ad8e0 msgr2=0x7f772c0d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:23.924 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.922+0000 7f7731000640 1 --2- 172.21.15.121:0/419922759 >> v2:172.21.15.123:3300/0 conn(0x7f772c0ad8e0 0x7f772c0d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:23.924 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.922+0000 7f7731000640 1 -- 172.21.15.121:0/419922759 >> v1:172.21.15.121:6789/0 conn(0x7f772c156c20 legacy=0x7f772c06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:23.924 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.922+0000 7f7731000640 1 -- 172.21.15.121:0/419922759 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f772c15d270 con 0x7f772c15ca40 2024-04-20T10:22:23.924 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.922+0000 7f772be00640 1 -- 172.21.15.121:0/419922759 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 167341547 0 0) 0x7f772c141c60 con 0x7f772c156c20 2024-04-20T10:22:23.924 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.924+0000 7f7731000640 1 --2- 172.21.15.121:0/419922759 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f772c15ca40 0x7f772c0ad3a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f772c06ff40 tx=0x7f771c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6413885eefe327 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:23.925 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.924+0000 7f772be00640 1 -- 172.21.15.121:0/419922759 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771c034070 con 0x7f772c15ca40 2024-04-20T10:22:23.926 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.924+0000 7f772be00640 1 -- 172.21.15.121:0/419922759 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f771c030860 con 0x7f772c15ca40 2024-04-20T10:22:23.926 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.924+0000 7f772be00640 1 -- 172.21.15.121:0/419922759 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771c030c30 con 0x7f772c15ca40 2024-04-20T10:22:23.926 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.925+0000 7f7734400640 1 -- 172.21.15.121:0/419922759 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f772c15ca40 msgr2=0x7f772c0ad3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:23.926 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.925+0000 7f7734400640 1 --2- 172.21.15.121:0/419922759 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f772c15ca40 0x7f772c0ad3a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f772c06ff40 tx=0x7f771c02b9a0 comp rx=0 tx=0).stop 2024-04-20T10:22:23.926 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.925+0000 7f7734400640 1 -- 172.21.15.121:0/419922759 shutdown_connections 2024-04-20T10:22:23.926 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.925+0000 7f7734400640 1 --2- 172.21.15.121:0/419922759 >> v2:172.21.15.123:3300/0 conn(0x7f772c0ad8e0 0x7f772c0d0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:23.926 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.925+0000 7f7734400640 1 --2- 172.21.15.121:0/419922759 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f772c15ca40 0x7f772c0ad3a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:23.926 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.925+0000 7f7734400640 1 -- 172.21.15.121:0/419922759 >> 172.21.15.121:0/419922759 conn(0x7f772c0b3350 msgr2=0x7f772c0b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:23.926 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.925+0000 7f7734400640 1 -- 172.21.15.121:0/419922759 shutdown_connections 2024-04-20T10:22:23.926 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.925+0000 7f7734400640 1 -- 172.21.15.121:0/419922759 wait complete. 2024-04-20T10:22:23.927 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.927+0000 7f7734400640 1 Processor -- start 2024-04-20T10:22:23.927 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.927+0000 7f7734400640 1 -- start start 2024-04-20T10:22:23.929 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.928+0000 7f7734400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f772c0ad8e0 0x7f772c1f59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:23.929 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.928+0000 7f7734400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f772c156c20 0x7f772c1e6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:23.929 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.928+0000 7f7734400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f772c15ca40 0x7f772c1e68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:23.929 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.928+0000 7f7734400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f772c0d08a0 con 0x7f772c15ca40 2024-04-20T10:22:23.929 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.928+0000 7f7734400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f772c15d830 con 0x7f772c156c20 2024-04-20T10:22:23.929 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.928+0000 7f7734400640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f772c0d0a20 con 0x7f772c0ad8e0 2024-04-20T10:22:23.929 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.929+0000 7f7732400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f772c15ca40 0x7f772c1e68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:23.930 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.929+0000 7f7732400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f772c15ca40 0x7f772c1e68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55170/0 (socket says 172.21.15.121:55170) 2024-04-20T10:22:23.930 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.929+0000 7f7732400640 1 -- 172.21.15.121:0/2265958330 learned_addr learned my addr 172.21.15.121:0/2265958330 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:23.930 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.929+0000 7f7731a00640 1 --2- 172.21.15.121:0/2265958330 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f772c0ad8e0 0x7f772c1f59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:23.930 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.929+0000 7f7731000640 1 --2- 172.21.15.121:0/2265958330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f772c156c20 0x7f772c1e6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:23.931 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.929+0000 7f7732400640 1 -- 172.21.15.121:0/2265958330 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f772c0ad8e0 msgr2=0x7f772c1f59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:23.931 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.929+0000 7f7732400640 1 --2- 172.21.15.121:0/2265958330 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f772c0ad8e0 0x7f772c1f59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:23.931 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.929+0000 7f7732400640 1 -- 172.21.15.121:0/2265958330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f772c156c20 msgr2=0x7f772c1e6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:23.931 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.929+0000 7f7732400640 1 --2- 172.21.15.121:0/2265958330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f772c156c20 0x7f772c1e6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:23.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.929+0000 7f7732400640 1 -- 172.21.15.121:0/2265958330 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f772c0ce990 con 0x7f772c15ca40 2024-04-20T10:22:23.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.929+0000 7f7732400640 1 --2- 172.21.15.121:0/2265958330 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f772c15ca40 0x7f772c1e68c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f772400ba10 tx=0x7f772400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:23.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.930+0000 7f7729600640 1 -- 172.21.15.121:0/2265958330 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7724019070 con 0x7f772c15ca40 2024-04-20T10:22:23.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.930+0000 7f7729600640 1 -- 172.21.15.121:0/2265958330 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f772400cac0 con 0x7f772c15ca40 2024-04-20T10:22:23.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.930+0000 7f7729600640 1 -- 172.21.15.121:0/2265958330 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7724014440 con 0x7f772c15ca40 2024-04-20T10:22:23.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.930+0000 7f7734400640 1 -- 172.21.15.121:0/2265958330 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f772c0cec70 con 0x7f772c15ca40 2024-04-20T10:22:23.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.930+0000 7f7734400640 1 -- 172.21.15.121:0/2265958330 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f772c0cf230 con 0x7f772c15ca40 2024-04-20T10:22:23.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.931+0000 7f7734400640 1 -- 172.21.15.121:0/2265958330 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f772c068980 con 0x7f772c15ca40 2024-04-20T10:22:23.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.932+0000 7f7729600640 1 -- 172.21.15.121:0/2265958330 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f772401c070 con 0x7f772c15ca40 2024-04-20T10:22:23.936 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.934+0000 7f7729600640 1 --2- 172.21.15.121:0/2265958330 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f771006c8f0 0x7f771006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:23.936 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.934+0000 7f7729600640 1 -- 172.21.15.121:0/2265958330 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f772408bb90 con 0x7f772c15ca40 2024-04-20T10:22:23.936 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.934+0000 7f7731a00640 1 --2- 172.21.15.121:0/2265958330 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f771006c8f0 0x7f771006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:23.936 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.935+0000 7f7731a00640 1 --2- 172.21.15.121:0/2265958330 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f771006c8f0 0x7f771006edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7720000f60 tx=0x7f7720007b50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:23.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:23.938+0000 7f7729600640 1 -- 172.21.15.121:0/2265958330 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7724059ba0 con 0x7f772c15ca40 2024-04-20T10:22:24.139 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.139+0000 7f7734400640 1 -- 172.21.15.121:0/2265958330 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f772c0cfd90 con 0x7f772c15ca40 2024-04-20T10:22:24.140 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.139+0000 7f7729600640 1 -- 172.21.15.121:0/2265958330 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f77240594a0 con 0x7f772c15ca40 2024-04-20T10:22:24.140 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 32 2024-04-20T10:22:24.143 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.142+0000 7f7734400640 1 -- 172.21.15.121:0/2265958330 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f771006c8f0 msgr2=0x7f771006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:24.143 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.142+0000 7f7734400640 1 --2- 172.21.15.121:0/2265958330 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f771006c8f0 0x7f771006edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7720000f60 tx=0x7f7720007b50 comp rx=0 tx=0).stop 2024-04-20T10:22:24.143 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.142+0000 7f7734400640 1 -- 172.21.15.121:0/2265958330 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f772c15ca40 msgr2=0x7f772c1e68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:24.143 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.142+0000 7f7734400640 1 --2- 172.21.15.121:0/2265958330 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f772c15ca40 0x7f772c1e68c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f772400ba10 tx=0x7f772400bed0 comp rx=0 tx=0).stop 2024-04-20T10:22:24.143 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.142+0000 7f7734400640 1 -- 172.21.15.121:0/2265958330 shutdown_connections 2024-04-20T10:22:24.143 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.142+0000 7f7734400640 1 --2- 172.21.15.121:0/2265958330 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f772c15ca40 0x7f772c1e68c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:24.143 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.142+0000 7f7734400640 1 --2- 172.21.15.121:0/2265958330 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f771006c8f0 0x7f771006edb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:24.143 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.142+0000 7f7734400640 1 --2- 172.21.15.121:0/2265958330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f772c156c20 0x7f772c1e6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:24.144 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.142+0000 7f7734400640 1 --2- 172.21.15.121:0/2265958330 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f772c0ad8e0 0x7f772c1f59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:24.144 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.142+0000 7f7734400640 1 -- 172.21.15.121:0/2265958330 >> 172.21.15.121:0/2265958330 conn(0x7f772c0b3350 msgr2=0x7f772c154c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:24.144 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.143+0000 7f7734400640 1 -- 172.21.15.121:0/2265958330 shutdown_connections 2024-04-20T10:22:24.144 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.143+0000 7f7734400640 1 -- 172.21.15.121:0/2265958330 wait complete. 2024-04-20T10:22:24.162 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T10:22:24.348 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.347+0000 7f9c2ba00640 1 Processor -- start 2024-04-20T10:22:24.348 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.347+0000 7f9c2ba00640 1 -- start start 2024-04-20T10:22:24.350 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.349+0000 7f9c2ba00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9c240d1a00 0x7f9c240b1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:24.350 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.349+0000 7f9c2ba00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c240d1e90 con 0x7f9c240b17d0 2024-04-20T10:22:24.350 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.349+0000 7f9c2ba00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c240d3820 con 0x7f9c240d4270 2024-04-20T10:22:24.350 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.349+0000 7f9c2ba00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f9c240d40c0 con 0x7f9c240d1a00 2024-04-20T10:22:24.350 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.349+0000 7f9c23e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f9c240d1a00 0x7f9c240b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:24.350 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.349+0000 7f9c29a00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f9c240b17d0 0x7f9c240d0380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:53498/0 (socket says 172.21.15.121:53498) 2024-04-20T10:22:24.350 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.349+0000 7f9c29a00640 1 -- 172.21.15.121:0/854088353 learned_addr learned my addr 172.21.15.121:0/854088353 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:24.351 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.350+0000 7f9c23e00640 1 -- 172.21.15.121:0/854088353 >> v1:172.21.15.123:6789/0 conn(0x7f9c240d4270 legacy=0x7f9c240d4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:24.351 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.350+0000 7f9c23e00640 1 -- 172.21.15.121:0/854088353 >> v1:172.21.15.121:6789/0 conn(0x7f9c240b17d0 legacy=0x7f9c240d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:24.351 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.350+0000 7f9c23e00640 1 -- 172.21.15.121:0/854088353 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c240d3b00 con 0x7f9c240d1a00 2024-04-20T10:22:24.351 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.350+0000 7f9c23400640 1 -- 172.21.15.121:0/854088353 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3724078665 0 0) 0x7f9c1c001110 con 0x7f9c240b17d0 2024-04-20T10:22:24.352 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.351+0000 7f9c23e00640 1 --2- 172.21.15.121:0/854088353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c240d1a00 0x7f9c240b1290 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c009d90 tx=0x7f9c0c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce4f4a97b61a7a67 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:24.353 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.351+0000 7f9c23400640 1 -- 172.21.15.121:0/854088353 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c039070 con 0x7f9c240d1a00 2024-04-20T10:22:24.353 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.351+0000 7f9c23400640 1 -- 172.21.15.121:0/854088353 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c0c02cc60 con 0x7f9c240d1a00 2024-04-20T10:22:24.353 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.351+0000 7f9c23400640 1 -- 172.21.15.121:0/854088353 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c034660 con 0x7f9c240d1a00 2024-04-20T10:22:24.353 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.352+0000 7f9c2ba00640 1 -- 172.21.15.121:0/854088353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c240d1a00 msgr2=0x7f9c240b1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:24.353 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.352+0000 7f9c2ba00640 1 --2- 172.21.15.121:0/854088353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c240d1a00 0x7f9c240b1290 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c009d90 tx=0x7f9c0c02beb0 comp rx=0 tx=0).stop 2024-04-20T10:22:24.353 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.352+0000 7f9c2ba00640 1 -- 172.21.15.121:0/854088353 shutdown_connections 2024-04-20T10:22:24.353 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.352+0000 7f9c2ba00640 1 --2- 172.21.15.121:0/854088353 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c240d1a00 0x7f9c240b1290 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:24.353 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.352+0000 7f9c2ba00640 1 -- 172.21.15.121:0/854088353 >> 172.21.15.121:0/854088353 conn(0x7f9c240b99c0 msgr2=0x7f9c240b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:24.353 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.352+0000 7f9c2ba00640 1 -- 172.21.15.121:0/854088353 shutdown_connections 2024-04-20T10:22:24.353 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.352+0000 7f9c2ba00640 1 -- 172.21.15.121:0/854088353 wait complete. 2024-04-20T10:22:24.354 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.354+0000 7f9c2ba00640 1 Processor -- start 2024-04-20T10:22:24.354 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.354+0000 7f9c2ba00640 1 -- start start 2024-04-20T10:22:24.357 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.355+0000 7f9c2ba00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9c240b17d0 0x7f9c240cc590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:24.357 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.355+0000 7f9c2ba00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c240d1a00 0x7f9c241f4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:24.357 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.355+0000 7f9c2ba00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c240d4270 0x7f9c241f50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:24.357 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.355+0000 7f9c2ba00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9c240d53b0 con 0x7f9c240d4270 2024-04-20T10:22:24.357 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.355+0000 7f9c2ba00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9c240cfe60 con 0x7f9c240d1a00 2024-04-20T10:22:24.357 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.355+0000 7f9c2ba00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f9c24161f70 con 0x7f9c240b17d0 2024-04-20T10:22:24.357 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.355+0000 7f9c29a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c240d4270 0x7f9c241f50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:24.357 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.355+0000 7f9c29a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c240d4270 0x7f9c241f50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55184/0 (socket says 172.21.15.121:55184) 2024-04-20T10:22:24.357 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.355+0000 7f9c29a00640 1 -- 172.21.15.121:0/2997854862 learned_addr learned my addr 172.21.15.121:0/2997854862 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:24.357 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.355+0000 7f9c29000640 1 --2- 172.21.15.121:0/2997854862 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9c240b17d0 0x7f9c240cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:24.358 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.355+0000 7f9c29a00640 1 -- 172.21.15.121:0/2997854862 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9c240b17d0 msgr2=0x7f9c240cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:24.358 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.355+0000 7f9c29a00640 1 --2- 172.21.15.121:0/2997854862 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9c240b17d0 0x7f9c240cc590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:24.358 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.355+0000 7f9c29a00640 1 -- 172.21.15.121:0/2997854862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c240d1a00 msgr2=0x7f9c241f4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:24.358 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.355+0000 7f9c23e00640 1 --2- 172.21.15.121:0/2997854862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c240d1a00 0x7f9c241f4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:24.358 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.356+0000 7f9c29a00640 1 --2- 172.21.15.121:0/2997854862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c240d1a00 0x7f9c241f4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:24.358 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.356+0000 7f9c29a00640 1 -- 172.21.15.121:0/2997854862 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c240cd500 con 0x7f9c240d4270 2024-04-20T10:22:24.358 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.356+0000 7f9c29a00640 1 --2- 172.21.15.121:0/2997854862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c240d4270 0x7f9c241f50b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00c750 tx=0x7f9c1c00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:24.358 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.356+0000 7f9c20c00640 1 -- 172.21.15.121:0/2997854862 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c018070 con 0x7f9c240d4270 2024-04-20T10:22:24.358 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.356+0000 7f9c20c00640 1 -- 172.21.15.121:0/2997854862 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c1c0070d0 con 0x7f9c240d4270 2024-04-20T10:22:24.358 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.356+0000 7f9c20c00640 1 -- 172.21.15.121:0/2997854862 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c0074c0 con 0x7f9c240d4270 2024-04-20T10:22:24.358 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.356+0000 7f9c2ba00640 1 -- 172.21.15.121:0/2997854862 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c240cd7e0 con 0x7f9c240d4270 2024-04-20T10:22:24.358 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.356+0000 7f9c2ba00640 1 -- 172.21.15.121:0/2997854862 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c240cdda0 con 0x7f9c240d4270 2024-04-20T10:22:24.359 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.357+0000 7f9c2ba00640 1 -- 172.21.15.121:0/2997854862 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c240ca940 con 0x7f9c240d4270 2024-04-20T10:22:24.359 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.358+0000 7f9c20c00640 1 -- 172.21.15.121:0/2997854862 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9c1c0047e0 con 0x7f9c240d4270 2024-04-20T10:22:24.360 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.360+0000 7f9c20c00640 1 --2- 172.21.15.121:0/2997854862 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9c0006c8f0 0x7f9c0006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:24.360 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.360+0000 7f9c20c00640 1 -- 172.21.15.121:0/2997854862 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f9c1c08b2b0 con 0x7f9c240d4270 2024-04-20T10:22:24.360 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.360+0000 7f9c29000640 1 --2- 172.21.15.121:0/2997854862 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9c0006c8f0 0x7f9c0006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:24.361 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.361+0000 7f9c29000640 1 --2- 172.21.15.121:0/2997854862 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9c0006c8f0 0x7f9c0006edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9c14006fd0 tx=0x7f9c14008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:24.364 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.364+0000 7f9c20c00640 1 -- 172.21.15.121:0/2997854862 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9c240ca940 con 0x7f9c240d4270 2024-04-20T10:22:24.571 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.570+0000 7f9c2ba00640 1 -- 172.21.15.121:0/2997854862 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9c24068980 con 0x7f9c240d4270 2024-04-20T10:22:24.572 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.571+0000 7f9c20c00640 1 -- 172.21.15.121:0/2997854862 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f9c240d1e90 con 0x7f9c240d4270 2024-04-20T10:22:24.572 INFO:teuthology.orchestra.run.smithi121.stderr:listed 0 entries 2024-04-20T10:22:24.574 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.574+0000 7f9c2ba00640 1 -- 172.21.15.121:0/2997854862 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9c0006c8f0 msgr2=0x7f9c0006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:24.574 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.574+0000 7f9c2ba00640 1 --2- 172.21.15.121:0/2997854862 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9c0006c8f0 0x7f9c0006edb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9c14006fd0 tx=0x7f9c14008040 comp rx=0 tx=0).stop 2024-04-20T10:22:24.574 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.574+0000 7f9c2ba00640 1 -- 172.21.15.121:0/2997854862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c240d4270 msgr2=0x7f9c241f50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:24.575 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.574+0000 7f9c2ba00640 1 --2- 172.21.15.121:0/2997854862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c240d4270 0x7f9c241f50b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00c750 tx=0x7f9c1c00cc10 comp rx=0 tx=0).stop 2024-04-20T10:22:24.575 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.574+0000 7f9c2ba00640 1 -- 172.21.15.121:0/2997854862 shutdown_connections 2024-04-20T10:22:24.575 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.574+0000 7f9c2ba00640 1 --2- 172.21.15.121:0/2997854862 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9c240d4270 0x7f9c241f50b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:24.575 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.574+0000 7f9c2ba00640 1 --2- 172.21.15.121:0/2997854862 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f9c0006c8f0 0x7f9c0006edb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:24.575 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.574+0000 7f9c2ba00640 1 --2- 172.21.15.121:0/2997854862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c240d1a00 0x7f9c241f4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:24.575 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.574+0000 7f9c2ba00640 1 --2- 172.21.15.121:0/2997854862 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9c240b17d0 0x7f9c240cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:24.575 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.574+0000 7f9c2ba00640 1 -- 172.21.15.121:0/2997854862 >> 172.21.15.121:0/2997854862 conn(0x7f9c240b99c0 msgr2=0x7f9c240d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:24.575 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.574+0000 7f9c2ba00640 1 -- 172.21.15.121:0/2997854862 shutdown_connections 2024-04-20T10:22:24.575 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.574+0000 7f9c2ba00640 1 -- 172.21.15.121:0/2997854862 wait complete. 2024-04-20T10:22:24.591 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T10:22:24.591 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:22:24.640 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:22:24.875 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.874+0000 7f45b6400640 1 Processor -- start 2024-04-20T10:22:24.875 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.874+0000 7f45b6400640 1 -- start start 2024-04-20T10:22:24.876 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.876+0000 7f45b6400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45b0156c20 0x7f45b006b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:24.876 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.876+0000 7f45b6400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f45b00d0360 0x7f45b00d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:24.876 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.876+0000 7f45b6400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45b0141c60 con 0x7f45b015ca40 2024-04-20T10:22:24.876 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.876+0000 7f45b6400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f45b00adab0 con 0x7f45b00d0360 2024-04-20T10:22:24.876 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.876+0000 7f45b6400640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f45b015cf90 con 0x7f45b0156c20 2024-04-20T10:22:24.877 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.876+0000 7f45afe00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f45b00d0360 0x7f45b00d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:24.877 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.876+0000 7f45a7e00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f45b015ca40 0x7f45b00ad3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:53500/0 (socket says 172.21.15.121:53500) 2024-04-20T10:22:24.878 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.876+0000 7f45a7e00640 1 -- 172.21.15.121:0/3790500085 learned_addr learned my addr 172.21.15.121:0/3790500085 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:24.878 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.876+0000 7f45afe00640 1 --2- 172.21.15.121:0/3790500085 >> v2:172.21.15.121:3300/0 conn(0x7f45b00d0360 0x7f45b00d07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55188/0 (socket says 172.21.15.121:55188) 2024-04-20T10:22:24.878 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.877+0000 7f45af400640 1 --2- 172.21.15.121:0/3790500085 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45b0156c20 0x7f45b006b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:24.878 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.877+0000 7f45afe00640 1 -- 172.21.15.121:0/3790500085 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45b0156c20 msgr2=0x7f45b006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:24.878 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.877+0000 7f45afe00640 1 --2- 172.21.15.121:0/3790500085 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45b0156c20 0x7f45b006b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:24.878 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.877+0000 7f45afe00640 1 -- 172.21.15.121:0/3790500085 >> v1:172.21.15.121:6789/0 conn(0x7f45b015ca40 legacy=0x7f45b00ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:24.878 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.877+0000 7f45afe00640 1 -- 172.21.15.121:0/3790500085 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b015d270 con 0x7f45b00d0360 2024-04-20T10:22:24.879 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.878+0000 7f45afe00640 1 --2- 172.21.15.121:0/3790500085 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45b00d0360 0x7f45b00d07f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f45a8009d90 tx=0x7f45a802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4989c3dfc10f4f22 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:24.879 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.878+0000 7f45aea00640 1 -- 172.21.15.121:0/3790500085 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a8039070 con 0x7f45b00d0360 2024-04-20T10:22:24.879 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.878+0000 7f45aea00640 1 -- 172.21.15.121:0/3790500085 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45a802cc60 con 0x7f45b00d0360 2024-04-20T10:22:24.879 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.878+0000 7f45aea00640 1 -- 172.21.15.121:0/3790500085 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45a8034660 con 0x7f45b00d0360 2024-04-20T10:22:24.880 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.879+0000 7f45b6400640 1 -- 172.21.15.121:0/3790500085 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45b00d0360 msgr2=0x7f45b00d07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:24.880 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.879+0000 7f45b6400640 1 --2- 172.21.15.121:0/3790500085 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45b00d0360 0x7f45b00d07f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f45a8009d90 tx=0x7f45a802beb0 comp rx=0 tx=0).stop 2024-04-20T10:22:24.880 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.879+0000 7f45b6400640 1 -- 172.21.15.121:0/3790500085 shutdown_connections 2024-04-20T10:22:24.880 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.879+0000 7f45b6400640 1 --2- 172.21.15.121:0/3790500085 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45b00d0360 0x7f45b00d07f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:24.880 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.879+0000 7f45b6400640 1 --2- 172.21.15.121:0/3790500085 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45b0156c20 0x7f45b006b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:24.880 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.879+0000 7f45b6400640 1 -- 172.21.15.121:0/3790500085 >> 172.21.15.121:0/3790500085 conn(0x7f45b00b3350 msgr2=0x7f45b00b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:24.880 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.879+0000 7f45b6400640 1 -- 172.21.15.121:0/3790500085 shutdown_connections 2024-04-20T10:22:24.880 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.879+0000 7f45b6400640 1 -- 172.21.15.121:0/3790500085 wait complete. 2024-04-20T10:22:24.881 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.881+0000 7f45b6400640 1 Processor -- start 2024-04-20T10:22:24.881 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.881+0000 7f45b6400640 1 -- start start 2024-04-20T10:22:24.883 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.882+0000 7f45b6400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45b00d0360 0x7f45b006dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:24.885 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.882+0000 7f45b6400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45b0156c20 0x7f45b0151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:24.885 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.882+0000 7f45b6400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45b015ca40 0x7f45b01f4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:24.885 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.882+0000 7f45b6400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f45b006be40 con 0x7f45b0156c20 2024-04-20T10:22:24.885 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.882+0000 7f45b6400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f45b015d830 con 0x7f45b00d0360 2024-04-20T10:22:24.885 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.882+0000 7f45b6400640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f45b006bfc0 con 0x7f45b015ca40 2024-04-20T10:22:24.885 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.882+0000 7f45a7e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45b0156c20 0x7f45b0151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:24.885 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.882+0000 7f45a7e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45b0156c20 0x7f45b0151fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55204/0 (socket says 172.21.15.121:55204) 2024-04-20T10:22:24.885 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.882+0000 7f45a7e00640 1 -- 172.21.15.121:0/3736427995 learned_addr learned my addr 172.21.15.121:0/3736427995 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:24.886 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.883+0000 7f45a7e00640 1 -- 172.21.15.121:0/3736427995 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45b015ca40 msgr2=0x7f45b01f4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:24.886 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.883+0000 7f45a7e00640 1 --2- 172.21.15.121:0/3736427995 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45b015ca40 0x7f45b01f4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:24.886 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.883+0000 7f45a7e00640 1 -- 172.21.15.121:0/3736427995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45b00d0360 msgr2=0x7f45b006dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:24.886 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.883+0000 7f45a7e00640 1 --2- 172.21.15.121:0/3736427995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45b00d0360 0x7f45b006dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:24.886 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.883+0000 7f45a7e00640 1 -- 172.21.15.121:0/3736427995 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b00cdc00 con 0x7f45b0156c20 2024-04-20T10:22:24.886 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.883+0000 7f45a7e00640 1 --2- 172.21.15.121:0/3736427995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45b0156c20 0x7f45b0151fd0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f459800c400 tx=0x7f459800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:24.886 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.883+0000 7f45a7400640 1 -- 172.21.15.121:0/3736427995 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4598017070 con 0x7f45b0156c20 2024-04-20T10:22:24.886 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.883+0000 7f45a7400640 1 -- 172.21.15.121:0/3736427995 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4598005ce0 con 0x7f45b0156c20 2024-04-20T10:22:24.886 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.883+0000 7f45b6400640 1 -- 172.21.15.121:0/3736427995 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45b0152510 con 0x7f45b0156c20 2024-04-20T10:22:24.886 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.883+0000 7f45a7400640 1 -- 172.21.15.121:0/3736427995 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45980072f0 con 0x7f45b0156c20 2024-04-20T10:22:24.886 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.883+0000 7f45b6400640 1 -- 172.21.15.121:0/3736427995 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45b006e750 con 0x7f45b0156c20 2024-04-20T10:22:24.887 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.884+0000 7f45b6400640 1 -- 172.21.15.121:0/3736427995 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45b0070460 con 0x7f45b0156c20 2024-04-20T10:22:24.887 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.885+0000 7f45a7400640 1 -- 172.21.15.121:0/3736427995 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f459801a070 con 0x7f45b0156c20 2024-04-20T10:22:24.887 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.886+0000 7f45a7400640 1 --2- 172.21.15.121:0/3736427995 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f458c06c8f0 0x7f458c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:24.887 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.886+0000 7f45a7400640 1 -- 172.21.15.121:0/3736427995 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f459808b3d0 con 0x7f45b0156c20 2024-04-20T10:22:24.887 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.886+0000 7f45af400640 1 --2- 172.21.15.121:0/3736427995 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f458c06c8f0 0x7f458c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:24.888 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.887+0000 7f45af400640 1 --2- 172.21.15.121:0/3736427995 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f458c06c8f0 0x7f458c06edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f45a00098f0 tx=0x7f45a0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:24.891 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:24.891+0000 7f45a7400640 1 -- 172.21.15.121:0/3736427995 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f45b0070460 con 0x7f45b0156c20 2024-04-20T10:22:25.093 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.093+0000 7f45b6400640 1 -- 172.21.15.121:0/3736427995 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f45b0068980 con 0x7f45b0156c20 2024-04-20T10:22:25.164 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.096+0000 7f45a7400640 1 -- 172.21.15.121:0/3736427995 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18940 (secure 0 0 0) 0x7f4598059360 con 0x7f45b0156c20 2024-04-20T10:22:25.165 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:22:25.165 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":27,"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","created":"2024-04-20T10:20:38.076076+0000","modified":"2024-04-20T10:21:44.715898+0000","last_up_change":"2024-04-20T10:20:50.162700+0000","last_in_change":"2024-04-20T10:20:41.510382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:20:51.141365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:21:00.436937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:21:01.026957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:21:04.862693+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"629a7b8c-a40a-4a2a-a3cc-d9f357774310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6809","nonce":1044588835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6811","nonce":1044588835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6815","nonce":1044588835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6813","nonce":1044588835}]},"public_addr":"172.21.15.121:6809/1044588835","cluster_addr":"172.21.15.121:6811/1044588835","heartbeat_back_addr":"172.21.15.121:6815/1044588835","heartbeat_front_addr":"172.21.15.121:6813/1044588835","state":["exists","up"]},{"osd":1,"uuid":"f2570399-aa50-4ff3-bf5d-3d09207290db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6801","nonce":3505199624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6803","nonce":3505199624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6807","nonce":3505199624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6805","nonce":3505199624}]},"public_addr":"172.21.15.121:6801/3505199624","cluster_addr":"172.21.15.121:6803/3505199624","heartbeat_back_addr":"172.21.15.121:6807/3505199624","heartbeat_front_addr":"172.21.15.121:6805/3505199624","state":["exists","up"]},{"osd":2,"uuid":"50893807-dc30-49f4-b41f-739366626658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6817","nonce":3656724207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6819","nonce":3656724207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6827","nonce":3656724207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6822","nonce":3656724207}]},"public_addr":"172.21.15.121:6817/3656724207","cluster_addr":"172.21.15.121:6819/3656724207","heartbeat_back_addr":"172.21.15.121:6827/3656724207","heartbeat_front_addr":"172.21.15.121:6822/3656724207","state":["exists","up"]},{"osd":3,"uuid":"dadb01bc-5491-4ffc-8824-d48a057240b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6823","nonce":2232342832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6826","nonce":2232342832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6831","nonce":2232342832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6829","nonce":2232342832}]},"public_addr":"172.21.15.121:6823/2232342832","cluster_addr":"172.21.15.121:6826/2232342832","heartbeat_back_addr":"172.21.15.121:6831/2232342832","heartbeat_front_addr":"172.21.15.121:6829/2232342832","state":["exists","up"]},{"osd":4,"uuid":"b3f3b4a1-b375-4526-b69e-344ab24c2da1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6801","nonce":99353111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6803","nonce":99353111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6807","nonce":99353111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6805","nonce":99353111}]},"public_addr":"172.21.15.123:6801/99353111","cluster_addr":"172.21.15.123:6803/99353111","heartbeat_back_addr":"172.21.15.123:6807/99353111","heartbeat_front_addr":"172.21.15.123:6805/99353111","state":["exists","up"]},{"osd":5,"uuid":"2ee24a08-1624-4961-9642-d7b510282484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6818","nonce":2504374688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6822","nonce":2504374688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6830","nonce":2504374688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6826","nonce":2504374688}]},"public_addr":"172.21.15.123:6818/2504374688","cluster_addr":"172.21.15.123:6822/2504374688","heartbeat_back_addr":"172.21.15.123:6830/2504374688","heartbeat_front_addr":"172.21.15.123:6826/2504374688","state":["exists","up"]},{"osd":6,"uuid":"186118cc-6128-49de-acca-55260945c1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6819","nonce":4139299599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6823","nonce":4139299599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6829","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6831","nonce":4139299599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6827","nonce":4139299599}]},"public_addr":"172.21.15.123:6819/4139299599","cluster_addr":"172.21.15.123:6823/4139299599","heartbeat_back_addr":"172.21.15.123:6831/4139299599","heartbeat_front_addr":"172.21.15.123:6827/4139299599","state":["exists","up"]},{"osd":7,"uuid":"a00aac6e-4bd9-415a-a93d-ae85210865ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6809","nonce":4043981069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6811","nonce":4043981069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6815","nonce":4043981069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6813","nonce":4043981069}]},"public_addr":"172.21.15.123:6809/4043981069","cluster_addr":"172.21.15.123:6811/4043981069","heartbeat_back_addr":"172.21.15.123:6815/4043981069","heartbeat_front_addr":"172.21.15.123:6813/4043981069","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.904540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.958585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.616367+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.885244+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.103127+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.167248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.884348+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.862356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.23","mappings":[{"from":3,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:22:25.166 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.098+0000 7f45b6400640 1 -- 172.21.15.121:0/3736427995 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f458c06c8f0 msgr2=0x7f458c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:25.166 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.098+0000 7f45b6400640 1 --2- 172.21.15.121:0/3736427995 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f458c06c8f0 0x7f458c06edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f45a00098f0 tx=0x7f45a0008040 comp rx=0 tx=0).stop 2024-04-20T10:22:25.166 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.098+0000 7f45b6400640 1 -- 172.21.15.121:0/3736427995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45b0156c20 msgr2=0x7f45b0151fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:25.166 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.098+0000 7f45b6400640 1 --2- 172.21.15.121:0/3736427995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45b0156c20 0x7f45b0151fd0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f459800c400 tx=0x7f459800c8c0 comp rx=0 tx=0).stop 2024-04-20T10:22:25.166 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.099+0000 7f45b6400640 1 -- 172.21.15.121:0/3736427995 shutdown_connections 2024-04-20T10:22:25.166 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.099+0000 7f45b6400640 1 --2- 172.21.15.121:0/3736427995 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f45b015ca40 0x7f45b01f4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:25.166 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.099+0000 7f45b6400640 1 --2- 172.21.15.121:0/3736427995 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f45b0156c20 0x7f45b0151fd0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:25.166 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.099+0000 7f45b6400640 1 --2- 172.21.15.121:0/3736427995 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f458c06c8f0 0x7f458c06edb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:25.166 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.099+0000 7f45b6400640 1 --2- 172.21.15.121:0/3736427995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45b00d0360 0x7f45b006dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:25.166 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.099+0000 7f45b6400640 1 -- 172.21.15.121:0/3736427995 >> 172.21.15.121:0/3736427995 conn(0x7f45b00b3350 msgr2=0x7f45b015dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:25.167 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.099+0000 7f45b6400640 1 -- 172.21.15.121:0/3736427995 shutdown_connections 2024-04-20T10:22:25.167 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.099+0000 7f45b6400640 1 -- 172.21.15.121:0/3736427995 wait complete. 2024-04-20T10:22:25.167 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:20:51.141365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:21:00.436937+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:21:01.026957+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T10:21:04.862693+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T10:22:25.168 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:22:25.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.381+0000 7f8246000640 1 Processor -- start 2024-04-20T10:22:25.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.381+0000 7f8246000640 1 -- start start 2024-04-20T10:22:25.384 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.383+0000 7f8246000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f82401412e0 0x7f824006d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:25.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.383+0000 7f8246000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82401416c0 0x7f82400c7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:25.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.383+0000 7f8246000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8240141c10 con 0x7f82400c7570 2024-04-20T10:22:25.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.383+0000 7f8246000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f8240142820 con 0x7f82401412e0 2024-04-20T10:22:25.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.383+0000 7f8246000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f8240141ef0 con 0x7f82401416c0 2024-04-20T10:22:25.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.383+0000 7f8245600640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f82400c7570 0x7f8240162490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:53504/0 (socket says 172.21.15.121:53504) 2024-04-20T10:22:25.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.383+0000 7f8245600640 1 -- 172.21.15.121:0/2184150872 learned_addr learned my addr 172.21.15.121:0/2184150872 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:25.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.383+0000 7f8244c00640 1 --2- 172.21.15.121:0/2184150872 >> v2:172.21.15.121:3300/0 conn(0x7f82401412e0 0x7f824006d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:25.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.384+0000 7f823fe00640 1 --2- 172.21.15.121:0/2184150872 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82401416c0 0x7f82400c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:25.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.384+0000 7f823f400640 1 -- 172.21.15.121:0/2184150872 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3908829152 0 0) 0x7f8240141c10 con 0x7f82400c7570 2024-04-20T10:22:25.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.384+0000 7f823f400640 1 -- 172.21.15.121:0/2184150872 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8224003520 con 0x7f82400c7570 2024-04-20T10:22:25.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.384+0000 7f8244c00640 1 -- 172.21.15.121:0/2184150872 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82401416c0 msgr2=0x7f82400c7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:25.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.384+0000 7f8244c00640 1 --2- 172.21.15.121:0/2184150872 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82401416c0 0x7f82400c7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:25.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.384+0000 7f8244c00640 1 -- 172.21.15.121:0/2184150872 >> v1:172.21.15.121:6789/0 conn(0x7f82400c7570 legacy=0x7f8240162490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:25.386 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.384+0000 7f8244c00640 1 -- 172.21.15.121:0/2184150872 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82401421d0 con 0x7f82401412e0 2024-04-20T10:22:25.387 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.385+0000 7f8244c00640 1 --2- 172.21.15.121:0/2184150872 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82401412e0 0x7f824006d2a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f82280097a0 tx=0x7f822802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bec8e07ff138ea server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:25.387 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.385+0000 7f823f400640 1 -- 172.21.15.121:0/2184150872 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8228035020 con 0x7f82401412e0 2024-04-20T10:22:25.387 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.386+0000 7f823f400640 1 -- 172.21.15.121:0/2184150872 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8228031830 con 0x7f82401412e0 2024-04-20T10:22:25.387 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.386+0000 7f823f400640 1 -- 172.21.15.121:0/2184150872 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8228031c00 con 0x7f82401412e0 2024-04-20T10:22:25.387 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.386+0000 7f8246000640 1 -- 172.21.15.121:0/2184150872 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82401412e0 msgr2=0x7f824006d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:25.387 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.386+0000 7f8246000640 1 --2- 172.21.15.121:0/2184150872 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82401412e0 0x7f824006d2a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f82280097a0 tx=0x7f822802b8d0 comp rx=0 tx=0).stop 2024-04-20T10:22:25.387 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.386+0000 7f8246000640 1 -- 172.21.15.121:0/2184150872 shutdown_connections 2024-04-20T10:22:25.387 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.386+0000 7f8246000640 1 --2- 172.21.15.121:0/2184150872 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82401416c0 0x7f82400c7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:25.388 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.386+0000 7f8246000640 1 --2- 172.21.15.121:0/2184150872 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82401412e0 0x7f824006d2a0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:25.388 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.386+0000 7f8246000640 1 -- 172.21.15.121:0/2184150872 >> 172.21.15.121:0/2184150872 conn(0x7f82400b3dc0 msgr2=0x7f82400b41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:25.388 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.386+0000 7f8246000640 1 -- 172.21.15.121:0/2184150872 shutdown_connections 2024-04-20T10:22:25.388 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.386+0000 7f8246000640 1 -- 172.21.15.121:0/2184150872 wait complete. 2024-04-20T10:22:25.388 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.388+0000 7f8246000640 1 Processor -- start 2024-04-20T10:22:25.388 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.388+0000 7f8246000640 1 -- start start 2024-04-20T10:22:25.390 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.389+0000 7f8246000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82400c7570 0x7f8240140380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:25.390 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.389+0000 7f8246000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82401412e0 0x7f8240156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:25.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.389+0000 7f8246000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82401416c0 0x7f82401eb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:25.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.389+0000 7f8246000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8240162d20 con 0x7f82401412e0 2024-04-20T10:22:25.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.389+0000 7f8246000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8240162ba0 con 0x7f82401416c0 2024-04-20T10:22:25.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.389+0000 7f8246000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f8240162ea0 con 0x7f82400c7570 2024-04-20T10:22:25.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.389+0000 7f823fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82401412e0 0x7f8240156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:25.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.390+0000 7f823fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82401412e0 0x7f8240156110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55208/0 (socket says 172.21.15.121:55208) 2024-04-20T10:22:25.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.390+0000 7f823fe00640 1 -- 172.21.15.121:0/3970508574 learned_addr learned my addr 172.21.15.121:0/3970508574 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:25.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.390+0000 7f8244c00640 1 --2- 172.21.15.121:0/3970508574 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82400c7570 0x7f8240140380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:25.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.390+0000 7f8245600640 1 --2- 172.21.15.121:0/3970508574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82401416c0 0x7f82401eb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:25.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.390+0000 7f823fe00640 1 -- 172.21.15.121:0/3970508574 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82400c7570 msgr2=0x7f8240140380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:25.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.390+0000 7f823fe00640 1 --2- 172.21.15.121:0/3970508574 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82400c7570 0x7f8240140380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:25.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.390+0000 7f823fe00640 1 -- 172.21.15.121:0/3970508574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82401416c0 msgr2=0x7f82401eb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:25.392 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.390+0000 7f823fe00640 1 --2- 172.21.15.121:0/3970508574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82401416c0 0x7f82401eb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:25.392 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.390+0000 7f823fe00640 1 -- 172.21.15.121:0/3970508574 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8240140e40 con 0x7f82401412e0 2024-04-20T10:22:25.392 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.390+0000 7f8245600640 1 --2- 172.21.15.121:0/3970508574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82401416c0 0x7f82401eb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:25.392 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.390+0000 7f823fe00640 1 --2- 172.21.15.121:0/3970508574 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82401412e0 0x7f8240156110 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f823000d800 tx=0x7f823000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:25.393 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.391+0000 7f823cc00640 1 -- 172.21.15.121:0/3970508574 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823000b6c0 con 0x7f82401412e0 2024-04-20T10:22:25.393 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.391+0000 7f823cc00640 1 -- 172.21.15.121:0/3970508574 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8230019070 con 0x7f82401412e0 2024-04-20T10:22:25.393 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.391+0000 7f823cc00640 1 -- 172.21.15.121:0/3970508574 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823000ba80 con 0x7f82401412e0 2024-04-20T10:22:25.393 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.391+0000 7f8246000640 1 -- 172.21.15.121:0/3970508574 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82401e98e0 con 0x7f82401412e0 2024-04-20T10:22:25.393 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.391+0000 7f8246000640 1 -- 172.21.15.121:0/3970508574 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82401ebca0 con 0x7f82401412e0 2024-04-20T10:22:25.393 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.392+0000 7f8246000640 1 -- 172.21.15.121:0/3970508574 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8240075b30 con 0x7f82401412e0 2024-04-20T10:22:25.393 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.393+0000 7f823cc00640 1 -- 172.21.15.121:0/3970508574 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f823000bc20 con 0x7f82401412e0 2024-04-20T10:22:25.394 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.394+0000 7f823cc00640 1 --2- 172.21.15.121:0/3970508574 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f821c06c920 0x7f821c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:25.395 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.394+0000 7f823cc00640 1 -- 172.21.15.121:0/3970508574 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f823008c500 con 0x7f82401412e0 2024-04-20T10:22:25.395 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.394+0000 7f8244c00640 1 --2- 172.21.15.121:0/3970508574 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f821c06c920 0x7f821c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:25.395 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.395+0000 7f8244c00640 1 --2- 172.21.15.121:0/3970508574 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f821c06c920 0x7f821c06ede0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8228036000 tx=0x7f8228008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:25.399 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.399+0000 7f823cc00640 1 -- 172.21.15.121:0/3970508574 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f823005a400 con 0x7f82401412e0 2024-04-20T10:22:25.569 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:22:25.577 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:22:25.599 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.599+0000 7f8246000640 1 -- 172.21.15.121:0/3970508574 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f82400c7310 con 0x7f82401412e0 2024-04-20T10:22:25.600 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.600+0000 7f823cc00640 1 -- 172.21.15.121:0/3970508574 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f8230059d00 con 0x7f82401412e0 2024-04-20T10:22:25.600 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2024-04-20T10:22:25.602 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.602+0000 7f8246000640 1 -- 172.21.15.121:0/3970508574 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f821c06c920 msgr2=0x7f821c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:25.602 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.602+0000 7f8246000640 1 --2- 172.21.15.121:0/3970508574 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f821c06c920 0x7f821c06ede0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8228036000 tx=0x7f8228008030 comp rx=0 tx=0).stop 2024-04-20T10:22:25.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.602+0000 7f8246000640 1 -- 172.21.15.121:0/3970508574 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82401412e0 msgr2=0x7f8240156110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:25.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.602+0000 7f8246000640 1 --2- 172.21.15.121:0/3970508574 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82401412e0 0x7f8240156110 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f823000d800 tx=0x7f823000dcc0 comp rx=0 tx=0).stop 2024-04-20T10:22:25.605 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.602+0000 7f8246000640 1 -- 172.21.15.121:0/3970508574 shutdown_connections 2024-04-20T10:22:25.605 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.602+0000 7f8246000640 1 --2- 172.21.15.121:0/3970508574 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f821c06c920 0x7f821c06ede0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:25.605 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.602+0000 7f8246000640 1 --2- 172.21.15.121:0/3970508574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82401416c0 0x7f82401eb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:25.605 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.602+0000 7f8246000640 1 --2- 172.21.15.121:0/3970508574 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f82401412e0 0x7f8240156110 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:25.605 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.602+0000 7f8246000640 1 --2- 172.21.15.121:0/3970508574 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f82400c7570 0x7f8240140380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:25.605 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.602+0000 7f8246000640 1 -- 172.21.15.121:0/3970508574 >> 172.21.15.121:0/3970508574 conn(0x7f82400b3dc0 msgr2=0x7f8240142cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:25.605 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.602+0000 7f8246000640 1 -- 172.21.15.121:0/3970508574 shutdown_connections 2024-04-20T10:22:25.605 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.602+0000 7f8246000640 1 -- 172.21.15.121:0/3970508574 wait complete. 2024-04-20T10:22:25.619 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:22:25.860 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.859+0000 7f05e3200640 1 Processor -- start 2024-04-20T10:22:25.860 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.859+0000 7f05e3200640 1 -- start start 2024-04-20T10:22:25.861 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.861+0000 7f05e3200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05dc15b3b0 0x7f05dc06eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:25.861 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.861+0000 7f05e3200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f05dc06dcb0 0x7f05dc071b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:25.861 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.861+0000 7f05e3200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05dc0ade40 con 0x7f05dc072090 2024-04-20T10:22:25.861 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.861+0000 7f05e3200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f05dc0ae0e0 con 0x7f05dc06dcb0 2024-04-20T10:22:25.861 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.861+0000 7f05e3200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f05dc15b790 con 0x7f05dc15b3b0 2024-04-20T10:22:25.862 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.861+0000 7f05d3400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f05dc06dcb0 0x7f05dc071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:25.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.861+0000 7f05d3400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f05dc06dcb0 0x7f05dc071b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55220/0 (socket says 172.21.15.121:55220) 2024-04-20T10:22:25.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.861+0000 7f05d3400640 1 -- 172.21.15.121:0/2071178262 learned_addr learned my addr 172.21.15.121:0/2071178262 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:25.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.862+0000 7f05dbe00640 1 --2- 172.21.15.121:0/2071178262 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05dc15b3b0 0x7f05dc06eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:25.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.862+0000 7f05d3400640 1 -- 172.21.15.121:0/2071178262 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05dc15b3b0 msgr2=0x7f05dc06eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:25.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.862+0000 7f05d3400640 1 --2- 172.21.15.121:0/2071178262 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05dc15b3b0 0x7f05dc06eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:25.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.862+0000 7f05d3400640 1 -- 172.21.15.121:0/2071178262 >> v1:172.21.15.123:6789/0 conn(0x7f05dc072090 legacy=0x7f05dc0d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:25.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.862+0000 7f05d3400640 1 -- 172.21.15.121:0/2071178262 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05dc0ae380 con 0x7f05dc06dcb0 2024-04-20T10:22:25.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.863+0000 7f05d3400640 1 --2- 172.21.15.121:0/2071178262 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05dc06dcb0 0x7f05dc071b50 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f05c4009870 tx=0x7f05c402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=979f0df5b821522e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:25.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.864+0000 7f05db400640 1 -- 172.21.15.121:0/2071178262 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c4034070 con 0x7f05dc06dcb0 2024-04-20T10:22:25.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.864+0000 7f05db400640 1 -- 172.21.15.121:0/2071178262 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05c4030840 con 0x7f05dc06dcb0 2024-04-20T10:22:25.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.864+0000 7f05db400640 1 -- 172.21.15.121:0/2071178262 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c4030c10 con 0x7f05dc06dcb0 2024-04-20T10:22:25.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.864+0000 7f05e3200640 1 -- 172.21.15.121:0/2071178262 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05dc06dcb0 msgr2=0x7f05dc071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:25.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.864+0000 7f05e3200640 1 --2- 172.21.15.121:0/2071178262 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05dc06dcb0 0x7f05dc071b50 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f05c4009870 tx=0x7f05c402b980 comp rx=0 tx=0).stop 2024-04-20T10:22:25.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.864+0000 7f05e3200640 1 -- 172.21.15.121:0/2071178262 shutdown_connections 2024-04-20T10:22:25.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.864+0000 7f05e3200640 1 --2- 172.21.15.121:0/2071178262 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05dc06dcb0 0x7f05dc071b50 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:25.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.864+0000 7f05e3200640 1 --2- 172.21.15.121:0/2071178262 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05dc15b3b0 0x7f05dc06eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:25.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.864+0000 7f05e3200640 1 -- 172.21.15.121:0/2071178262 >> 172.21.15.121:0/2071178262 conn(0x7f05dc0b3350 msgr2=0x7f05dc0b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:25.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.864+0000 7f05e3200640 1 -- 172.21.15.121:0/2071178262 shutdown_connections 2024-04-20T10:22:25.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.864+0000 7f05e3200640 1 -- 172.21.15.121:0/2071178262 wait complete. 2024-04-20T10:22:25.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.866+0000 7f05e3200640 1 Processor -- start 2024-04-20T10:22:25.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.866+0000 7f05e3200640 1 -- start start 2024-04-20T10:22:25.869 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.867+0000 7f05e3200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05dc06dcb0 0x7f05dc14f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:25.869 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.867+0000 7f05e3200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05dc15b3b0 0x7f05dc154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:25.869 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.867+0000 7f05e3200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05dc154a50 0x7f05dc0b0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:25.869 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.867+0000 7f05e3200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc0aec20 con 0x7f05dc154a50 2024-04-20T10:22:25.869 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.867+0000 7f05e3200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f05dc0ae940 con 0x7f05dc06dcb0 2024-04-20T10:22:25.869 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.867+0000 7f05e3200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f05dc06efe0 con 0x7f05dc15b3b0 2024-04-20T10:22:25.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.868+0000 7f05e1200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05dc154a50 0x7f05dc0b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:25.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.868+0000 7f05e1200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05dc154a50 0x7f05dc0b0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:55224/0 (socket says 172.21.15.121:55224) 2024-04-20T10:22:25.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.868+0000 7f05e1200640 1 -- 172.21.15.121:0/866261488 learned_addr learned my addr 172.21.15.121:0/866261488 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:25.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.868+0000 7f05dbe00640 1 --2- 172.21.15.121:0/866261488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05dc06dcb0 0x7f05dc14f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:25.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.868+0000 7f05d3400640 1 --2- 172.21.15.121:0/866261488 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05dc15b3b0 0x7f05dc154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:25.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.868+0000 7f05e1200640 1 -- 172.21.15.121:0/866261488 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05dc15b3b0 msgr2=0x7f05dc154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:25.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.868+0000 7f05e1200640 1 --2- 172.21.15.121:0/866261488 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05dc15b3b0 0x7f05dc154510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:25.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.868+0000 7f05e1200640 1 -- 172.21.15.121:0/866261488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05dc06dcb0 msgr2=0x7f05dc14f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:25.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.868+0000 7f05e1200640 1 --2- 172.21.15.121:0/866261488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05dc06dcb0 0x7f05dc14f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:25.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.868+0000 7f05e1200640 1 -- 172.21.15.121:0/866261488 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05dc14f9c0 con 0x7f05dc154a50 2024-04-20T10:22:25.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.868+0000 7f05d3400640 1 --2- 172.21.15.121:0/866261488 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05dc15b3b0 0x7f05dc154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:25.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.868+0000 7f05e1200640 1 --2- 172.21.15.121:0/866261488 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05dc154a50 0x7f05dc0b0970 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f05d400c730 tx=0x7f05d400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:25.870 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.869+0000 7f05d8c00640 1 -- 172.21.15.121:0/866261488 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d4003b90 con 0x7f05dc154a50 2024-04-20T10:22:25.871 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.869+0000 7f05d8c00640 1 -- 172.21.15.121:0/866261488 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05d4003d30 con 0x7f05dc154a50 2024-04-20T10:22:25.871 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.869+0000 7f05e3200640 1 -- 172.21.15.121:0/866261488 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05dc154250 con 0x7f05dc154a50 2024-04-20T10:22:25.871 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.869+0000 7f05d8c00640 1 -- 172.21.15.121:0/866261488 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d4005310 con 0x7f05dc154a50 2024-04-20T10:22:25.871 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.869+0000 7f05e3200640 1 -- 172.21.15.121:0/866261488 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05dc1523d0 con 0x7f05dc154a50 2024-04-20T10:22:25.871 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.870+0000 7f05e3200640 1 -- 172.21.15.121:0/866261488 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05dc151e80 con 0x7f05dc154a50 2024-04-20T10:22:25.871 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.871+0000 7f05d8c00640 1 -- 172.21.15.121:0/866261488 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f05d40056c0 con 0x7f05dc154a50 2024-04-20T10:22:25.872 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.872+0000 7f05d8c00640 1 --2- 172.21.15.121:0/866261488 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f05b806c920 0x7f05b806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:25.872 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.872+0000 7f05d8c00640 1 -- 172.21.15.121:0/866261488 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f05d408bf40 con 0x7f05dc154a50 2024-04-20T10:22:25.872 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.872+0000 7f05dbe00640 1 --2- 172.21.15.121:0/866261488 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f05b806c920 0x7f05b806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:25.873 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.873+0000 7f05dbe00640 1 --2- 172.21.15.121:0/866261488 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f05b806c920 0x7f05b806ede0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f05cc006fd0 tx=0x7f05cc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:25.876 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:25.876+0000 7f05d8c00640 1 -- 172.21.15.121:0/866261488 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f05d4059f60 con 0x7f05dc154a50 2024-04-20T10:22:26.077 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.076+0000 7f05e3200640 1 -- 172.21.15.121:0/866261488 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f05dc068980 con 0x7f05dc154a50 2024-04-20T10:22:26.077 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.077+0000 7f05d8c00640 1 -- 172.21.15.121:0/866261488 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f05d4059860 con 0x7f05dc154a50 2024-04-20T10:22:26.078 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 32 2024-04-20T10:22:26.081 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.080+0000 7f05e3200640 1 -- 172.21.15.121:0/866261488 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f05b806c920 msgr2=0x7f05b806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:26.081 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.080+0000 7f05e3200640 1 --2- 172.21.15.121:0/866261488 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f05b806c920 0x7f05b806ede0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f05cc006fd0 tx=0x7f05cc008040 comp rx=0 tx=0).stop 2024-04-20T10:22:26.081 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.080+0000 7f05e3200640 1 -- 172.21.15.121:0/866261488 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05dc154a50 msgr2=0x7f05dc0b0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:26.081 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.080+0000 7f05e3200640 1 --2- 172.21.15.121:0/866261488 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05dc154a50 0x7f05dc0b0970 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f05d400c730 tx=0x7f05d400cbf0 comp rx=0 tx=0).stop 2024-04-20T10:22:26.081 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.080+0000 7f05e3200640 1 -- 172.21.15.121:0/866261488 shutdown_connections 2024-04-20T10:22:26.081 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.080+0000 7f05e3200640 1 --2- 172.21.15.121:0/866261488 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f05dc154a50 0x7f05dc0b0970 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:26.081 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.080+0000 7f05e3200640 1 --2- 172.21.15.121:0/866261488 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f05dc15b3b0 0x7f05dc154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:26.081 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.080+0000 7f05e3200640 1 --2- 172.21.15.121:0/866261488 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f05b806c920 0x7f05b806ede0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:26.081 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.080+0000 7f05e3200640 1 --2- 172.21.15.121:0/866261488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05dc06dcb0 0x7f05dc14f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:26.082 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.080+0000 7f05e3200640 1 -- 172.21.15.121:0/866261488 >> 172.21.15.121:0/866261488 conn(0x7f05dc0b3350 msgr2=0x7f05dc0735c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:26.082 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.080+0000 7f05e3200640 1 -- 172.21.15.121:0/866261488 shutdown_connections 2024-04-20T10:22:26.082 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.080+0000 7f05e3200640 1 -- 172.21.15.121:0/866261488 wait complete. 2024-04-20T10:22:26.099 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:22:26.295 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.294+0000 7fee98a00640 1 Processor -- start 2024-04-20T10:22:26.295 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.294+0000 7fee98a00640 1 -- start start 2024-04-20T10:22:26.297 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.296+0000 7fee98a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fee94161b40 0x7fee940c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:26.297 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.296+0000 7fee98a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fee940ae3e0 0x7fee940c9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:26.298 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.296+0000 7fee98a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee940aded0 con 0x7fee940cdea0 2024-04-20T10:22:26.298 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.296+0000 7fee98a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fee940ad2c0 con 0x7fee94161b40 2024-04-20T10:22:26.298 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.296+0000 7fee98a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fee940ad5a0 con 0x7fee940ae3e0 2024-04-20T10:22:26.298 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.296+0000 7fee93400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fee94161b40 0x7fee940c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:26.298 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.296+0000 7fee93e00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fee940cdea0 0x7fee941620d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:56166/0 (socket says 172.21.15.121:56166) 2024-04-20T10:22:26.298 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.296+0000 7fee93e00640 1 -- 172.21.15.121:0/3824469945 learned_addr learned my addr 172.21.15.121:0/3824469945 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:26.298 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.297+0000 7fee8be00640 1 --2- 172.21.15.121:0/3824469945 >> v2:172.21.15.123:3300/0 conn(0x7fee940ae3e0 0x7fee940c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:26.298 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.297+0000 7fee93400640 1 -- 172.21.15.121:0/3824469945 >> v2:172.21.15.123:3300/0 conn(0x7fee940ae3e0 msgr2=0x7fee940c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:26.298 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.297+0000 7fee93400640 1 --2- 172.21.15.121:0/3824469945 >> v2:172.21.15.123:3300/0 conn(0x7fee940ae3e0 0x7fee940c9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:26.298 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.297+0000 7fee93400640 1 -- 172.21.15.121:0/3824469945 >> v1:172.21.15.121:6789/0 conn(0x7fee940cdea0 legacy=0x7fee941620d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:26.298 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.297+0000 7fee93400640 1 -- 172.21.15.121:0/3824469945 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee940ad880 con 0x7fee94161b40 2024-04-20T10:22:26.298 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.297+0000 7fee8be00640 1 --2- 172.21.15.121:0/3824469945 >> v2:172.21.15.123:3300/0 conn(0x7fee940ae3e0 0x7fee940c9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:26.298 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.297+0000 7fee92a00640 1 -- 172.21.15.121:0/3824469945 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 84375010 0 0) 0x7fee940aded0 con 0x7fee940cdea0 2024-04-20T10:22:26.300 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.298+0000 7fee93400640 1 --2- 172.21.15.121:0/3824469945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee94161b40 0x7fee940c92f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fee9406d680 tx=0x7fee8402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a0206ee8e22ec2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:26.300 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.298+0000 7fee92a00640 1 -- 172.21.15.121:0/3824469945 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee84035020 con 0x7fee94161b40 2024-04-20T10:22:26.300 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.298+0000 7fee92a00640 1 -- 172.21.15.121:0/3824469945 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee84031830 con 0x7fee94161b40 2024-04-20T10:22:26.300 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.298+0000 7fee92a00640 1 -- 172.21.15.121:0/3824469945 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee84031c00 con 0x7fee94161b40 2024-04-20T10:22:26.300 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.299+0000 7fee98a00640 1 -- 172.21.15.121:0/3824469945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee94161b40 msgr2=0x7fee940c92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:26.300 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.299+0000 7fee98a00640 1 --2- 172.21.15.121:0/3824469945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee94161b40 0x7fee940c92f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fee9406d680 tx=0x7fee8402b8d0 comp rx=0 tx=0).stop 2024-04-20T10:22:26.300 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.299+0000 7fee98a00640 1 -- 172.21.15.121:0/3824469945 shutdown_connections 2024-04-20T10:22:26.300 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.299+0000 7fee98a00640 1 --2- 172.21.15.121:0/3824469945 >> v2:172.21.15.123:3300/0 conn(0x7fee940ae3e0 0x7fee940c9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:26.300 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.299+0000 7fee98a00640 1 --2- 172.21.15.121:0/3824469945 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee94161b40 0x7fee940c92f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:26.300 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.299+0000 7fee98a00640 1 -- 172.21.15.121:0/3824469945 >> 172.21.15.121:0/3824469945 conn(0x7fee940cffa0 msgr2=0x7fee940d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:26.300 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.299+0000 7fee98a00640 1 -- 172.21.15.121:0/3824469945 shutdown_connections 2024-04-20T10:22:26.301 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.299+0000 7fee98a00640 1 -- 172.21.15.121:0/3824469945 wait complete. 2024-04-20T10:22:26.301 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.301+0000 7fee98a00640 1 Processor -- start 2024-04-20T10:22:26.301 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.301+0000 7fee98a00640 1 -- start start 2024-04-20T10:22:26.303 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.302+0000 7fee98a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee940ae3e0 0x7fee9409d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:26.303 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.302+0000 7fee98a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee940cdea0 0x7fee941f7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:26.303 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.302+0000 7fee98a00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee94161b40 0x7fee941f7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:26.303 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.302+0000 7fee98a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fee9406d0c0 con 0x7fee940ae3e0 2024-04-20T10:22:26.304 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.302+0000 7fee98a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fee940acf50 con 0x7fee940cdea0 2024-04-20T10:22:26.304 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.302+0000 7fee98a00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fee9406d240 con 0x7fee94161b40 2024-04-20T10:22:26.304 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.302+0000 7fee93400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee940ae3e0 0x7fee9409d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:26.304 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.302+0000 7fee93400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee940ae3e0 0x7fee9409d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58982/0 (socket says 172.21.15.121:58982) 2024-04-20T10:22:26.304 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.302+0000 7fee93400640 1 -- 172.21.15.121:0/4186910515 learned_addr learned my addr 172.21.15.121:0/4186910515 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:26.304 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.303+0000 7fee8be00640 1 --2- 172.21.15.121:0/4186910515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee940cdea0 0x7fee941f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:26.304 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.303+0000 7fee93400640 1 -- 172.21.15.121:0/4186910515 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee94161b40 msgr2=0x7fee941f7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:26.304 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.303+0000 7fee93400640 1 --2- 172.21.15.121:0/4186910515 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee94161b40 0x7fee941f7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:26.304 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.303+0000 7fee93400640 1 -- 172.21.15.121:0/4186910515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee940cdea0 msgr2=0x7fee941f7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:26.304 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.303+0000 7fee93400640 1 --2- 172.21.15.121:0/4186910515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee940cdea0 0x7fee941f7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:26.304 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.303+0000 7fee93400640 1 -- 172.21.15.121:0/4186910515 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee94161380 con 0x7fee940ae3e0 2024-04-20T10:22:26.304 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.303+0000 7fee93400640 1 --2- 172.21.15.121:0/4186910515 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee940ae3e0 0x7fee9409d2a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fee84000c00 tx=0x7fee840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:26.305 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.303+0000 7fee8b400640 1 -- 172.21.15.121:0/4186910515 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee84040070 con 0x7fee940ae3e0 2024-04-20T10:22:26.305 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.303+0000 7fee8b400640 1 -- 172.21.15.121:0/4186910515 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee84039940 con 0x7fee940ae3e0 2024-04-20T10:22:26.305 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.303+0000 7fee8b400640 1 -- 172.21.15.121:0/4186910515 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee84035040 con 0x7fee940ae3e0 2024-04-20T10:22:26.305 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.303+0000 7fee98a00640 1 -- 172.21.15.121:0/4186910515 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee940b3ce0 con 0x7fee940ae3e0 2024-04-20T10:22:26.305 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.303+0000 7fee98a00640 1 -- 172.21.15.121:0/4186910515 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee940b42a0 con 0x7fee940ae3e0 2024-04-20T10:22:26.306 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.305+0000 7fee98a00640 1 -- 172.21.15.121:0/4186910515 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee94075b30 con 0x7fee940ae3e0 2024-04-20T10:22:26.306 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.306+0000 7fee8b400640 1 -- 172.21.15.121:0/4186910515 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee8404b060 con 0x7fee940ae3e0 2024-04-20T10:22:26.307 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.306+0000 7fee8b400640 1 --2- 172.21.15.121:0/4186910515 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fee7006c920 0x7fee7006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:26.307 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.307+0000 7fee8b400640 1 -- 172.21.15.121:0/4186910515 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fee840ad9e0 con 0x7fee940ae3e0 2024-04-20T10:22:26.307 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.307+0000 7fee8be00640 1 --2- 172.21.15.121:0/4186910515 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fee7006c920 0x7fee7006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:26.308 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.307+0000 7fee8be00640 1 --2- 172.21.15.121:0/4186910515 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fee7006c920 0x7fee7006ede0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fee94152ad0 tx=0x7fee7c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:26.311 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.311+0000 7fee8b400640 1 -- 172.21.15.121:0/4186910515 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fee8407b970 con 0x7fee940ae3e0 2024-04-20T10:22:26.513 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.513+0000 7fee98a00640 1 -- 172.21.15.121:0/4186910515 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fee94162130 con 0x7fee940ae3e0 2024-04-20T10:22:26.514 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.514+0000 7fee8b400640 1 -- 172.21.15.121:0/4186910515 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fee8407b270 con 0x7fee940ae3e0 2024-04-20T10:22:26.514 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2024-04-20T10:22:26.518 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.517+0000 7fee98a00640 1 -- 172.21.15.121:0/4186910515 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fee7006c920 msgr2=0x7fee7006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:26.518 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.517+0000 7fee98a00640 1 --2- 172.21.15.121:0/4186910515 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fee7006c920 0x7fee7006ede0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fee94152ad0 tx=0x7fee7c008040 comp rx=0 tx=0).stop 2024-04-20T10:22:26.518 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.517+0000 7fee98a00640 1 -- 172.21.15.121:0/4186910515 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee940ae3e0 msgr2=0x7fee9409d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:26.518 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.517+0000 7fee98a00640 1 --2- 172.21.15.121:0/4186910515 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee940ae3e0 0x7fee9409d2a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fee84000c00 tx=0x7fee840046c0 comp rx=0 tx=0).stop 2024-04-20T10:22:26.518 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.517+0000 7fee98a00640 1 -- 172.21.15.121:0/4186910515 shutdown_connections 2024-04-20T10:22:26.518 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.517+0000 7fee98a00640 1 --2- 172.21.15.121:0/4186910515 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fee94161b40 0x7fee941f7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:26.518 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.517+0000 7fee98a00640 1 --2- 172.21.15.121:0/4186910515 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fee7006c920 0x7fee7006ede0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:26.518 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.517+0000 7fee98a00640 1 --2- 172.21.15.121:0/4186910515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee940cdea0 0x7fee941f7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:26.518 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.517+0000 7fee98a00640 1 --2- 172.21.15.121:0/4186910515 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fee940ae3e0 0x7fee9409d2a0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:26.519 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.517+0000 7fee98a00640 1 -- 172.21.15.121:0/4186910515 >> 172.21.15.121:0/4186910515 conn(0x7fee940cffa0 msgr2=0x7fee94162930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:26.519 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.517+0000 7fee98a00640 1 -- 172.21.15.121:0/4186910515 shutdown_connections 2024-04-20T10:22:26.519 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.517+0000 7fee98a00640 1 -- 172.21.15.121:0/4186910515 wait complete. 2024-04-20T10:22:26.536 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T10:22:26.735 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.734+0000 7f919ae00640 1 Processor -- start 2024-04-20T10:22:26.735 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.735+0000 7f919ae00640 1 -- start start 2024-04-20T10:22:26.736 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.736+0000 7f919ae00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9194161a50 0x7f91940b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:26.737 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.736+0000 7f919ae00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f91940d3b20 0x7f919406b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:26.737 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.736+0000 7f919ae00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91940d0980 con 0x7f91940d0480 2024-04-20T10:22:26.737 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.736+0000 7f919ae00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f91940d0c60 con 0x7f91940d3b20 2024-04-20T10:22:26.737 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.736+0000 7f919ae00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f9194162ae0 con 0x7f9194161a50 2024-04-20T10:22:26.737 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.737+0000 7f9199a00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9194161a50 0x7f91940b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:26.738 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.737+0000 7f9199000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f91940d3b20 0x7f919406b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:26.738 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.737+0000 7f9199a00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9194161a50 0x7f91940b3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.121:41180/0 (socket says 172.21.15.121:41180) 2024-04-20T10:22:26.738 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.737+0000 7f9199a00640 1 -- 172.21.15.121:0/491956024 learned_addr learned my addr 172.21.15.121:0/491956024 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:26.738 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.738+0000 7f9199a00640 1 -- 172.21.15.121:0/491956024 >> v2:172.21.15.123:3300/0 conn(0x7f91940d3b20 msgr2=0x7f919406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:26.739 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.738+0000 7f9199a00640 1 --2- 172.21.15.121:0/491956024 >> v2:172.21.15.123:3300/0 conn(0x7f91940d3b20 0x7f919406b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:26.739 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.738+0000 7f9199a00640 1 -- 172.21.15.121:0/491956024 >> v1:172.21.15.123:6789/0 conn(0x7f91940d0480 legacy=0x7f919406c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:26.739 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.738+0000 7f9199a00640 1 -- 172.21.15.121:0/491956024 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9194162490 con 0x7f9194161a50 2024-04-20T10:22:26.739 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.739+0000 7f9199a00640 1 --2- 172.21.15.121:0/491956024 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9194161a50 0x7f91940b3180 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9188009e50 tx=0x7f918802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=613d6b9589d045b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:26.740 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.740+0000 7f9193e00640 1 -- 172.21.15.121:0/491956024 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918802cb00 con 0x7f9194161a50 2024-04-20T10:22:26.740 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.740+0000 7f9193e00640 1 -- 172.21.15.121:0/491956024 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f918802cca0 con 0x7f9194161a50 2024-04-20T10:22:26.740 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.740+0000 7f9193e00640 1 -- 172.21.15.121:0/491956024 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9188034610 con 0x7f9194161a50 2024-04-20T10:22:26.741 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.740+0000 7f919ae00640 1 -- 172.21.15.121:0/491956024 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9194161a50 msgr2=0x7f91940b3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:26.741 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.740+0000 7f919ae00640 1 --2- 172.21.15.121:0/491956024 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9194161a50 0x7f91940b3180 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9188009e50 tx=0x7f918802bf60 comp rx=0 tx=0).stop 2024-04-20T10:22:26.741 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.741+0000 7f919ae00640 1 -- 172.21.15.121:0/491956024 shutdown_connections 2024-04-20T10:22:26.742 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.741+0000 7f919ae00640 1 --2- 172.21.15.121:0/491956024 >> v2:172.21.15.123:3300/0 conn(0x7f91940d3b20 0x7f919406b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:26.742 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.741+0000 7f919ae00640 1 --2- 172.21.15.121:0/491956024 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9194161a50 0x7f91940b3180 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:26.742 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.741+0000 7f919ae00640 1 -- 172.21.15.121:0/491956024 >> 172.21.15.121:0/491956024 conn(0x7f91940d0ed0 msgr2=0x7f91940d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:26.742 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.741+0000 7f919ae00640 1 -- 172.21.15.121:0/491956024 shutdown_connections 2024-04-20T10:22:26.742 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.741+0000 7f919ae00640 1 -- 172.21.15.121:0/491956024 wait complete. 2024-04-20T10:22:26.743 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.742+0000 7f919ae00640 1 Processor -- start 2024-04-20T10:22:26.743 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.743+0000 7f919ae00640 1 -- start start 2024-04-20T10:22:26.744 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.743+0000 7f919ae00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91940d0480 0x7f9194073020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:26.744 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.743+0000 7f919ae00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9194161a50 0x7f91941fcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:26.744 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.743+0000 7f919ae00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f91941fd330 0x7f91940b8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:26.744 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.744+0000 7f919ae00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f9194161fb0 con 0x7f9194161a50 2024-04-20T10:22:26.745 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.744+0000 7f919ae00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9194161e30 con 0x7f91940d0480 2024-04-20T10:22:26.745 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.744+0000 7f919ae00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f9194162130 con 0x7f91941fd330 2024-04-20T10:22:26.745 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.744+0000 7f9199000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9194161a50 0x7f91941fcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:26.745 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.744+0000 7f9199000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9194161a50 0x7f91941fcdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58990/0 (socket says 172.21.15.121:58990) 2024-04-20T10:22:26.745 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.744+0000 7f9199000640 1 -- 172.21.15.121:0/3096312767 learned_addr learned my addr 172.21.15.121:0/3096312767 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:26.746 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.744+0000 7f9199000640 1 -- 172.21.15.121:0/3096312767 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f91941fd330 msgr2=0x7f91940b8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:26.746 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.744+0000 7f9199000640 1 --2- 172.21.15.121:0/3096312767 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f91941fd330 0x7f91940b8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:26.746 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.744+0000 7f9199000640 1 -- 172.21.15.121:0/3096312767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91940d0480 msgr2=0x7f9194073020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:26.746 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.744+0000 7f9199a00640 1 --2- 172.21.15.121:0/3096312767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91940d0480 0x7f9194073020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:26.746 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.744+0000 7f9199000640 1 --2- 172.21.15.121:0/3096312767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91940d0480 0x7f9194073020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:26.746 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.744+0000 7f9199000640 1 -- 172.21.15.121:0/3096312767 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9194153430 con 0x7f9194161a50 2024-04-20T10:22:26.746 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.745+0000 7f9199000640 1 --2- 172.21.15.121:0/3096312767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9194161a50 0x7f91941fcdf0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f918000e9b0 tx=0x7f918000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:26.747 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.745+0000 7f9191600640 1 -- 172.21.15.121:0/3096312767 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9180017070 con 0x7f9194161a50 2024-04-20T10:22:26.747 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.745+0000 7f9191600640 1 -- 172.21.15.121:0/3096312767 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f918000c9a0 con 0x7f9194161a50 2024-04-20T10:22:26.747 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.745+0000 7f9191600640 1 -- 172.21.15.121:0/3096312767 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918000cd90 con 0x7f9194161a50 2024-04-20T10:22:26.747 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.745+0000 7f919ae00640 1 -- 172.21.15.121:0/3096312767 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9194153710 con 0x7f9194161a50 2024-04-20T10:22:26.747 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.745+0000 7f919ae00640 1 -- 172.21.15.121:0/3096312767 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9194153cd0 con 0x7f9194161a50 2024-04-20T10:22:26.747 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.746+0000 7f919ae00640 1 -- 172.21.15.121:0/3096312767 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91940b77c0 con 0x7f9194161a50 2024-04-20T10:22:26.748 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.747+0000 7f9191600640 1 -- 172.21.15.121:0/3096312767 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f918001a070 con 0x7f9194161a50 2024-04-20T10:22:26.748 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.748+0000 7f9191600640 1 --2- 172.21.15.121:0/3096312767 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f917406c8f0 0x7f917406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:26.749 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.748+0000 7f9191600640 1 -- 172.21.15.121:0/3096312767 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f918008bf60 con 0x7f9194161a50 2024-04-20T10:22:26.749 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.748+0000 7f9199a00640 1 --2- 172.21.15.121:0/3096312767 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f917406c8f0 0x7f917406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:26.749 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.749+0000 7f9199a00640 1 --2- 172.21.15.121:0/3096312767 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f917406c8f0 0x7f917406edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9188035000 tx=0x7f9188030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:26.753 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.753+0000 7f9191600640 1 -- 172.21.15.121:0/3096312767 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9180059f90 con 0x7f9194161a50 2024-04-20T10:22:26.960 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.959+0000 7f919ae00640 1 -- 172.21.15.121:0/3096312767 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f91941602f0 con 0x7f9194161a50 2024-04-20T10:22:26.960 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.960+0000 7f9191600640 1 -- 172.21.15.121:0/3096312767 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f9180059890 con 0x7f9194161a50 2024-04-20T10:22:26.961 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 32 2024-04-20T10:22:26.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.962+0000 7f919ae00640 1 -- 172.21.15.121:0/3096312767 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f917406c8f0 msgr2=0x7f917406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:26.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.962+0000 7f919ae00640 1 --2- 172.21.15.121:0/3096312767 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f917406c8f0 0x7f917406edb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9188035000 tx=0x7f9188030000 comp rx=0 tx=0).stop 2024-04-20T10:22:26.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.963+0000 7f919ae00640 1 -- 172.21.15.121:0/3096312767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9194161a50 msgr2=0x7f91941fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:26.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.963+0000 7f919ae00640 1 --2- 172.21.15.121:0/3096312767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9194161a50 0x7f91941fcdf0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f918000e9b0 tx=0x7f918000ee70 comp rx=0 tx=0).stop 2024-04-20T10:22:26.963 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.963+0000 7f919ae00640 1 -- 172.21.15.121:0/3096312767 shutdown_connections 2024-04-20T10:22:26.964 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.963+0000 7f919ae00640 1 --2- 172.21.15.121:0/3096312767 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f91941fd330 0x7f91940b8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:26.964 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.963+0000 7f919ae00640 1 --2- 172.21.15.121:0/3096312767 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9194161a50 0x7f91941fcdf0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:26.964 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.963+0000 7f919ae00640 1 --2- 172.21.15.121:0/3096312767 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f917406c8f0 0x7f917406edb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:26.964 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.963+0000 7f919ae00640 1 --2- 172.21.15.121:0/3096312767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91940d0480 0x7f9194073020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:26.964 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.963+0000 7f919ae00640 1 -- 172.21.15.121:0/3096312767 >> 172.21.15.121:0/3096312767 conn(0x7f91940d0ed0 msgr2=0x7f91940b4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:26.964 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.963+0000 7f919ae00640 1 -- 172.21.15.121:0/3096312767 shutdown_connections 2024-04-20T10:22:26.964 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:26.963+0000 7f919ae00640 1 -- 172.21.15.121:0/3096312767 wait complete. 2024-04-20T10:22:26.984 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T10:22:27.181 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.180+0000 7f8b6a400640 1 Processor -- start 2024-04-20T10:22:27.181 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.181+0000 7f8b6a400640 1 -- start start 2024-04-20T10:22:27.183 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.182+0000 7f8b6a400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8b64159e10 0x7f8b64071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:27.183 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.182+0000 7f8b6a400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8b6415a1f0 0x7f8b64071fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:27.183 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.182+0000 7f8b6a400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8b6406da90 0x7f8b64160030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:27.183 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.182+0000 7f8b6a400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f8b64072900 con 0x7f8b6415a1f0 2024-04-20T10:22:27.183 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.182+0000 7f8b6a400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f8b6415ca40 con 0x7f8b6406da90 2024-04-20T10:22:27.183 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.182+0000 7f8b6a400640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f8b6415cc90 con 0x7f8b64159e10 2024-04-20T10:22:27.183 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.182+0000 7f8b63e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8b6415a1f0 0x7f8b64071fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:27.183 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.183+0000 7f8b63e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8b6415a1f0 0x7f8b64071fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:58996/0 (socket says 172.21.15.121:58996) 2024-04-20T10:22:27.184 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.183+0000 7f8b63e00640 1 -- 172.21.15.121:0/3395317411 learned_addr learned my addr 172.21.15.121:0/3395317411 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:27.184 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.183+0000 7f8b69000640 1 --2- 172.21.15.121:0/3395317411 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8b64159e10 0x7f8b64071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:27.184 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.183+0000 7f8b69a00640 1 --2- 172.21.15.121:0/3395317411 >> v2:172.21.15.123:3300/0 conn(0x7f8b6406da90 0x7f8b64160030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:27.185 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.183+0000 7f8b63e00640 1 -- 172.21.15.121:0/3395317411 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8b64159e10 msgr2=0x7f8b64071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:27.185 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.183+0000 7f8b63e00640 1 --2- 172.21.15.121:0/3395317411 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8b64159e10 0x7f8b64071a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:27.185 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.183+0000 7f8b63e00640 1 -- 172.21.15.121:0/3395317411 >> v2:172.21.15.123:3300/0 conn(0x7f8b6406da90 msgr2=0x7f8b64160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:27.185 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.183+0000 7f8b63e00640 1 --2- 172.21.15.121:0/3395317411 >> v2:172.21.15.123:3300/0 conn(0x7f8b6406da90 0x7f8b64160030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:27.185 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.183+0000 7f8b63e00640 1 -- 172.21.15.121:0/3395317411 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b6415cf70 con 0x7f8b6415a1f0 2024-04-20T10:22:27.185 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.184+0000 7f8b63e00640 1 --2- 172.21.15.121:0/3395317411 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b6415a1f0 0x7f8b64071fc0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8b640cffe0 tx=0x7f8b5002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c51a8b518f11aac server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:27.186 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.184+0000 7f8b63400640 1 -- 172.21.15.121:0/3395317411 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b50035020 con 0x7f8b6415a1f0 2024-04-20T10:22:27.186 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.184+0000 7f8b63400640 1 -- 172.21.15.121:0/3395317411 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b5002d040 con 0x7f8b6415a1f0 2024-04-20T10:22:27.186 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.184+0000 7f8b63400640 1 -- 172.21.15.121:0/3395317411 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b50031a80 con 0x7f8b6415a1f0 2024-04-20T10:22:27.186 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.185+0000 7f8b6a400640 1 -- 172.21.15.121:0/3395317411 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b6415a1f0 msgr2=0x7f8b64071fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:27.186 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.185+0000 7f8b6a400640 1 --2- 172.21.15.121:0/3395317411 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b6415a1f0 0x7f8b64071fc0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8b640cffe0 tx=0x7f8b5002b900 comp rx=0 tx=0).stop 2024-04-20T10:22:27.186 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.185+0000 7f8b6a400640 1 -- 172.21.15.121:0/3395317411 shutdown_connections 2024-04-20T10:22:27.187 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.185+0000 7f8b6a400640 1 --2- 172.21.15.121:0/3395317411 >> v2:172.21.15.123:3300/0 conn(0x7f8b6406da90 0x7f8b64160030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:27.187 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.185+0000 7f8b6a400640 1 --2- 172.21.15.121:0/3395317411 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b6415a1f0 0x7f8b64071fc0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:27.187 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.185+0000 7f8b6a400640 1 --2- 172.21.15.121:0/3395317411 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8b64159e10 0x7f8b64071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:27.187 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.185+0000 7f8b6a400640 1 -- 172.21.15.121:0/3395317411 >> 172.21.15.121:0/3395317411 conn(0x7f8b640d0040 msgr2=0x7f8b640d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:27.187 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.185+0000 7f8b6a400640 1 -- 172.21.15.121:0/3395317411 shutdown_connections 2024-04-20T10:22:27.187 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.185+0000 7f8b6a400640 1 -- 172.21.15.121:0/3395317411 wait complete. 2024-04-20T10:22:27.187 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.187+0000 7f8b6a400640 1 Processor -- start 2024-04-20T10:22:27.188 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.187+0000 7f8b6a400640 1 -- start start 2024-04-20T10:22:27.189 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.188+0000 7f8b6a400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8b6406da90 0x7f8b640cca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:27.189 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.188+0000 7f8b6a400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b64159e10 0x7f8b6414dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:27.190 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.188+0000 7f8b6a400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b6415a1f0 0x7f8b6415ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:27.190 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.188+0000 7f8b6a400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6415d810 con 0x7f8b64159e10 2024-04-20T10:22:27.190 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.188+0000 7f8b6a400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6415d530 con 0x7f8b6415a1f0 2024-04-20T10:22:27.190 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.188+0000 7f8b6a400640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f8b6406b900 con 0x7f8b6406da90 2024-04-20T10:22:27.190 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.188+0000 7f8b63e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b64159e10 0x7f8b6414dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:27.190 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.188+0000 7f8b63e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b64159e10 0x7f8b6414dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59004/0 (socket says 172.21.15.121:59004) 2024-04-20T10:22:27.190 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.188+0000 7f8b63e00640 1 -- 172.21.15.121:0/2586429690 learned_addr learned my addr 172.21.15.121:0/2586429690 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:27.190 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.189+0000 7f8b69000640 1 --2- 172.21.15.121:0/2586429690 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8b6406da90 0x7f8b640cca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:27.190 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.189+0000 7f8b69a00640 1 --2- 172.21.15.121:0/2586429690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b6415a1f0 0x7f8b6415ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:27.190 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.189+0000 7f8b63e00640 1 -- 172.21.15.121:0/2586429690 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8b6406da90 msgr2=0x7f8b640cca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:27.190 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.189+0000 7f8b63e00640 1 --2- 172.21.15.121:0/2586429690 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8b6406da90 0x7f8b640cca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:27.191 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.189+0000 7f8b63e00640 1 -- 172.21.15.121:0/2586429690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b6415a1f0 msgr2=0x7f8b6415ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:27.191 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.189+0000 7f8b63e00640 1 --2- 172.21.15.121:0/2586429690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b6415a1f0 0x7f8b6415ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:27.191 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.189+0000 7f8b63e00640 1 -- 172.21.15.121:0/2586429690 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b6415aa60 con 0x7f8b64159e10 2024-04-20T10:22:27.191 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.189+0000 7f8b69000640 1 --2- 172.21.15.121:0/2586429690 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8b6406da90 0x7f8b640cca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:27.191 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.189+0000 7f8b63e00640 1 --2- 172.21.15.121:0/2586429690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b64159e10 0x7f8b6414dbc0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f8b5002b940 tx=0x7f8b500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:27.192 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.189+0000 7f8b60c00640 1 -- 172.21.15.121:0/2586429690 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b50040070 con 0x7f8b64159e10 2024-04-20T10:22:27.192 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.189+0000 7f8b60c00640 1 -- 172.21.15.121:0/2586429690 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b50039c10 con 0x7f8b64159e10 2024-04-20T10:22:27.192 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.190+0000 7f8b60c00640 1 -- 172.21.15.121:0/2586429690 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b50035040 con 0x7f8b64159e10 2024-04-20T10:22:27.192 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.190+0000 7f8b6a400640 1 -- 172.21.15.121:0/2586429690 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b6415ad40 con 0x7f8b64159e10 2024-04-20T10:22:27.192 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.190+0000 7f8b6a400640 1 -- 172.21.15.121:0/2586429690 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b6415b300 con 0x7f8b64159e10 2024-04-20T10:22:27.192 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.191+0000 7f8b6a400640 1 -- 172.21.15.121:0/2586429690 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b64075b60 con 0x7f8b64159e10 2024-04-20T10:22:27.192 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.192+0000 7f8b60c00640 1 -- 172.21.15.121:0/2586429690 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8b5004b060 con 0x7f8b64159e10 2024-04-20T10:22:27.193 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.193+0000 7f8b60c00640 1 --2- 172.21.15.121:0/2586429690 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f8b4406c920 0x7f8b4406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:27.193 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.193+0000 7f8b60c00640 1 -- 172.21.15.121:0/2586429690 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f8b500adb60 con 0x7f8b64159e10 2024-04-20T10:22:27.194 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.193+0000 7f8b69000640 1 --2- 172.21.15.121:0/2586429690 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f8b4406c920 0x7f8b4406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:27.194 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.194+0000 7f8b69000640 1 --2- 172.21.15.121:0/2586429690 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f8b4406c920 0x7f8b4406ede0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8b58006fd0 tx=0x7f8b58008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:27.197 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.197+0000 7f8b60c00640 1 -- 172.21.15.121:0/2586429690 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8b5007baf0 con 0x7f8b64159e10 2024-04-20T10:22:27.409 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.409+0000 7f8b6a400640 1 -- 172.21.15.121:0/2586429690 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8b64157f70 con 0x7f8b64159e10 2024-04-20T10:22:27.410 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.409+0000 7f8b60c00640 1 -- 172.21.15.121:0/2586429690 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f8b5007b3f0 con 0x7f8b64159e10 2024-04-20T10:22:27.410 INFO:teuthology.orchestra.run.smithi121.stderr:listed 0 entries 2024-04-20T10:22:27.412 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.412+0000 7f8b6a400640 1 -- 172.21.15.121:0/2586429690 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f8b4406c920 msgr2=0x7f8b4406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:27.412 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.412+0000 7f8b6a400640 1 --2- 172.21.15.121:0/2586429690 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f8b4406c920 0x7f8b4406ede0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8b58006fd0 tx=0x7f8b58008040 comp rx=0 tx=0).stop 2024-04-20T10:22:27.413 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.412+0000 7f8b6a400640 1 -- 172.21.15.121:0/2586429690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b64159e10 msgr2=0x7f8b6414dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:27.413 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.412+0000 7f8b6a400640 1 --2- 172.21.15.121:0/2586429690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b64159e10 0x7f8b6414dbc0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f8b5002b940 tx=0x7f8b500046c0 comp rx=0 tx=0).stop 2024-04-20T10:22:27.413 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.412+0000 7f8b6a400640 1 -- 172.21.15.121:0/2586429690 shutdown_connections 2024-04-20T10:22:27.413 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.412+0000 7f8b6a400640 1 --2- 172.21.15.121:0/2586429690 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f8b4406c920 0x7f8b4406ede0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:27.413 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.412+0000 7f8b6a400640 1 --2- 172.21.15.121:0/2586429690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b6415a1f0 0x7f8b6415ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:27.413 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.412+0000 7f8b6a400640 1 --2- 172.21.15.121:0/2586429690 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b64159e10 0x7f8b6414dbc0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:27.414 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.412+0000 7f8b6a400640 1 --2- 172.21.15.121:0/2586429690 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8b6406da90 0x7f8b640cca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:27.414 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.412+0000 7f8b6a400640 1 -- 172.21.15.121:0/2586429690 >> 172.21.15.121:0/2586429690 conn(0x7f8b640d0040 msgr2=0x7f8b6406bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:27.414 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.412+0000 7f8b6a400640 1 -- 172.21.15.121:0/2586429690 shutdown_connections 2024-04-20T10:22:27.414 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.413+0000 7f8b6a400640 1 -- 172.21.15.121:0/2586429690 wait complete. 2024-04-20T10:22:27.431 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi121.front.sepia.ceph.com... 2024-04-20T10:22:27.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:22:27.432 DEBUG:teuthology.orchestra.run.smithi121:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-04-20T10:22:27.512 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:22:27.513 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi121.front.sepia.ceph.com 2024-04-20T10:22:27.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:22:27.513 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-20T10:22:27.563 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T10:22:27.563 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:22:27.620 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T10:22:27.620 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:22:27.675 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:22:27.888 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.887+0000 7f1335000640 1 Processor -- start 2024-04-20T10:22:27.888 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.888+0000 7f1335000640 1 -- start start 2024-04-20T10:22:27.889 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.889+0000 7f1335000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f133015ca40 0x7f13300ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:27.890 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.889+0000 7f1335000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f13300ad8e0 0x7f13300d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:27.890 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.889+0000 7f1335000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1330141c60 con 0x7f1330156c20 2024-04-20T10:22:27.890 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.889+0000 7f1335000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f133006c010 con 0x7f13300ad8e0 2024-04-20T10:22:27.890 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.889+0000 7f1335000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f133015cf90 con 0x7f133015ca40 2024-04-20T10:22:27.890 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.889+0000 7f132de00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f1330156c20 0x7f133006b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:56176/0 (socket says 172.21.15.121:56176) 2024-04-20T10:22:27.890 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.889+0000 7f132de00640 1 -- 172.21.15.121:0/769743044 learned_addr learned my addr 172.21.15.121:0/769743044 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:27.890 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.889+0000 7f132e800640 1 --2- 172.21.15.121:0/769743044 >> v2:172.21.15.121:3300/0 conn(0x7f13300ad8e0 0x7f13300d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:27.891 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.890+0000 7f1327e00640 1 --2- 172.21.15.121:0/769743044 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f133015ca40 0x7f13300ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:27.891 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.890+0000 7f132e800640 1 -- 172.21.15.121:0/769743044 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f133015ca40 msgr2=0x7f13300ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:27.891 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.890+0000 7f132e800640 1 --2- 172.21.15.121:0/769743044 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f133015ca40 0x7f13300ad3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:27.891 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.890+0000 7f132e800640 1 -- 172.21.15.121:0/769743044 >> v1:172.21.15.121:6789/0 conn(0x7f1330156c20 legacy=0x7f133006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:27.891 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.890+0000 7f132e800640 1 -- 172.21.15.121:0/769743044 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f133015d270 con 0x7f13300ad8e0 2024-04-20T10:22:27.891 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.890+0000 7f1327e00640 1 --2- 172.21.15.121:0/769743044 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f133015ca40 0x7f13300ad3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:27.892 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.891+0000 7f132e800640 1 --2- 172.21.15.121:0/769743044 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13300ad8e0 0x7f13300d0360 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1328009d90 tx=0x7f132802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=676d191bb96406c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:27.892 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.891+0000 7f132d400640 1 -- 172.21.15.121:0/769743044 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1328039070 con 0x7f13300ad8e0 2024-04-20T10:22:27.892 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.891+0000 7f132d400640 1 -- 172.21.15.121:0/769743044 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f132802cc60 con 0x7f13300ad8e0 2024-04-20T10:22:27.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.891+0000 7f132d400640 1 -- 172.21.15.121:0/769743044 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1328034660 con 0x7f13300ad8e0 2024-04-20T10:22:27.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.892+0000 7f1335000640 1 -- 172.21.15.121:0/769743044 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13300ad8e0 msgr2=0x7f13300d0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:27.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.892+0000 7f1335000640 1 --2- 172.21.15.121:0/769743044 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13300ad8e0 0x7f13300d0360 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1328009d90 tx=0x7f132802beb0 comp rx=0 tx=0).stop 2024-04-20T10:22:27.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.892+0000 7f1335000640 1 -- 172.21.15.121:0/769743044 shutdown_connections 2024-04-20T10:22:27.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.892+0000 7f1335000640 1 --2- 172.21.15.121:0/769743044 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f13300ad8e0 0x7f13300d0360 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:27.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.892+0000 7f1335000640 1 --2- 172.21.15.121:0/769743044 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f133015ca40 0x7f13300ad3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:27.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.892+0000 7f1335000640 1 -- 172.21.15.121:0/769743044 >> 172.21.15.121:0/769743044 conn(0x7f13300b3350 msgr2=0x7f13300b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:27.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.892+0000 7f1335000640 1 -- 172.21.15.121:0/769743044 shutdown_connections 2024-04-20T10:22:27.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.892+0000 7f1335000640 1 -- 172.21.15.121:0/769743044 wait complete. 2024-04-20T10:22:27.894 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.894+0000 7f1335000640 1 Processor -- start 2024-04-20T10:22:27.894 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.894+0000 7f1335000640 1 -- start start 2024-04-20T10:22:27.896 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.895+0000 7f1335000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f13300ad8e0 0x7f13301f4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:27.896 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.895+0000 7f1335000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1330156c20 0x7f133014f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:27.896 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.895+0000 7f1335000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f133015ca40 0x7f133014fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:27.896 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.895+0000 7f1335000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f13300d08a0 con 0x7f133015ca40 2024-04-20T10:22:27.896 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.895+0000 7f1335000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f133015d830 con 0x7f1330156c20 2024-04-20T10:22:27.896 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.895+0000 7f1335000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f13300d0a20 con 0x7f13300ad8e0 2024-04-20T10:22:27.896 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.895+0000 7f132e800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f133015ca40 0x7f133014fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:27.896 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.895+0000 7f132e800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f133015ca40 0x7f133014fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59020/0 (socket says 172.21.15.121:59020) 2024-04-20T10:22:27.896 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.895+0000 7f132e800640 1 -- 172.21.15.121:0/870327088 learned_addr learned my addr 172.21.15.121:0/870327088 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:27.897 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.895+0000 7f132de00640 1 --2- 172.21.15.121:0/870327088 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f13300ad8e0 0x7f13301f4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:27.897 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.895+0000 7f1327e00640 1 --2- 172.21.15.121:0/870327088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1330156c20 0x7f133014f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:27.897 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.895+0000 7f132e800640 1 -- 172.21.15.121:0/870327088 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f13300ad8e0 msgr2=0x7f13301f4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:27.897 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.895+0000 7f132e800640 1 --2- 172.21.15.121:0/870327088 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f13300ad8e0 0x7f13301f4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:27.897 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.895+0000 7f132e800640 1 -- 172.21.15.121:0/870327088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1330156c20 msgr2=0x7f133014f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:27.897 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.895+0000 7f132e800640 1 --2- 172.21.15.121:0/870327088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1330156c20 0x7f133014f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:27.897 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.895+0000 7f132e800640 1 -- 172.21.15.121:0/870327088 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13301f4860 con 0x7f133015ca40 2024-04-20T10:22:27.898 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.896+0000 7f132e800640 1 --2- 172.21.15.121:0/870327088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f133015ca40 0x7f133014fa60 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1328009d90 tx=0x7f1328004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:27.898 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.896+0000 7f1326000640 1 -- 172.21.15.121:0/870327088 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1328039050 con 0x7f133015ca40 2024-04-20T10:22:27.898 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.896+0000 7f1326000640 1 -- 172.21.15.121:0/870327088 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1328045070 con 0x7f133015ca40 2024-04-20T10:22:27.898 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.896+0000 7f1326000640 1 -- 172.21.15.121:0/870327088 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132803e850 con 0x7f133015ca40 2024-04-20T10:22:27.898 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.896+0000 7f1335000640 1 -- 172.21.15.121:0/870327088 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13300b18b0 con 0x7f133015ca40 2024-04-20T10:22:27.898 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.896+0000 7f1335000640 1 -- 172.21.15.121:0/870327088 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1330150c10 con 0x7f133015ca40 2024-04-20T10:22:27.898 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.897+0000 7f1335000640 1 -- 172.21.15.121:0/870327088 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1330068980 con 0x7f133015ca40 2024-04-20T10:22:27.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.898+0000 7f1326000640 1 -- 172.21.15.121:0/870327088 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f132803e9f0 con 0x7f133015ca40 2024-04-20T10:22:27.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.899+0000 7f1326000640 1 --2- 172.21.15.121:0/870327088 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f130c06c8f0 0x7f130c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:27.900 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.899+0000 7f1326000640 1 -- 172.21.15.121:0/870327088 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f13280aece0 con 0x7f133015ca40 2024-04-20T10:22:27.900 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.899+0000 7f132de00640 1 --2- 172.21.15.121:0/870327088 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f130c06c8f0 0x7f130c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:27.900 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.900+0000 7f132de00640 1 --2- 172.21.15.121:0/870327088 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f130c06c8f0 0x7f130c06edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1320000f60 tx=0x7f1320009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:27.904 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:27.904+0000 7f1326000640 1 -- 172.21.15.121:0/870327088 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f13301f4860 con 0x7f133015ca40 2024-04-20T10:22:28.104 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.104+0000 7f1335000640 1 -- 172.21.15.121:0/870327088 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f13300cfd90 con 0x7f133015ca40 2024-04-20T10:22:28.107 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.107+0000 7f1326000640 1 -- 172.21.15.121:0/870327088 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18940 (secure 0 0 0) 0x7f132807cd30 con 0x7f133015ca40 2024-04-20T10:22:28.108 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:22:28.108 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":27,"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","created":"2024-04-20T10:20:38.076076+0000","modified":"2024-04-20T10:21:44.715898+0000","last_up_change":"2024-04-20T10:20:50.162700+0000","last_in_change":"2024-04-20T10:20:41.510382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:20:51.141365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:21:00.436937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:21:01.026957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:21:04.862693+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"629a7b8c-a40a-4a2a-a3cc-d9f357774310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6809","nonce":1044588835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6811","nonce":1044588835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6815","nonce":1044588835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6813","nonce":1044588835}]},"public_addr":"172.21.15.121:6809/1044588835","cluster_addr":"172.21.15.121:6811/1044588835","heartbeat_back_addr":"172.21.15.121:6815/1044588835","heartbeat_front_addr":"172.21.15.121:6813/1044588835","state":["exists","up"]},{"osd":1,"uuid":"f2570399-aa50-4ff3-bf5d-3d09207290db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6801","nonce":3505199624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6803","nonce":3505199624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6807","nonce":3505199624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6805","nonce":3505199624}]},"public_addr":"172.21.15.121:6801/3505199624","cluster_addr":"172.21.15.121:6803/3505199624","heartbeat_back_addr":"172.21.15.121:6807/3505199624","heartbeat_front_addr":"172.21.15.121:6805/3505199624","state":["exists","up"]},{"osd":2,"uuid":"50893807-dc30-49f4-b41f-739366626658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6817","nonce":3656724207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6819","nonce":3656724207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6827","nonce":3656724207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6822","nonce":3656724207}]},"public_addr":"172.21.15.121:6817/3656724207","cluster_addr":"172.21.15.121:6819/3656724207","heartbeat_back_addr":"172.21.15.121:6827/3656724207","heartbeat_front_addr":"172.21.15.121:6822/3656724207","state":["exists","up"]},{"osd":3,"uuid":"dadb01bc-5491-4ffc-8824-d48a057240b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6823","nonce":2232342832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6826","nonce":2232342832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6831","nonce":2232342832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6829","nonce":2232342832}]},"public_addr":"172.21.15.121:6823/2232342832","cluster_addr":"172.21.15.121:6826/2232342832","heartbeat_back_addr":"172.21.15.121:6831/2232342832","heartbeat_front_addr":"172.21.15.121:6829/2232342832","state":["exists","up"]},{"osd":4,"uuid":"b3f3b4a1-b375-4526-b69e-344ab24c2da1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6801","nonce":99353111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6803","nonce":99353111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6807","nonce":99353111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6805","nonce":99353111}]},"public_addr":"172.21.15.123:6801/99353111","cluster_addr":"172.21.15.123:6803/99353111","heartbeat_back_addr":"172.21.15.123:6807/99353111","heartbeat_front_addr":"172.21.15.123:6805/99353111","state":["exists","up"]},{"osd":5,"uuid":"2ee24a08-1624-4961-9642-d7b510282484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6818","nonce":2504374688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6822","nonce":2504374688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6830","nonce":2504374688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6826","nonce":2504374688}]},"public_addr":"172.21.15.123:6818/2504374688","cluster_addr":"172.21.15.123:6822/2504374688","heartbeat_back_addr":"172.21.15.123:6830/2504374688","heartbeat_front_addr":"172.21.15.123:6826/2504374688","state":["exists","up"]},{"osd":6,"uuid":"186118cc-6128-49de-acca-55260945c1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6819","nonce":4139299599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6823","nonce":4139299599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6829","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6831","nonce":4139299599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6827","nonce":4139299599}]},"public_addr":"172.21.15.123:6819/4139299599","cluster_addr":"172.21.15.123:6823/4139299599","heartbeat_back_addr":"172.21.15.123:6831/4139299599","heartbeat_front_addr":"172.21.15.123:6827/4139299599","state":["exists","up"]},{"osd":7,"uuid":"a00aac6e-4bd9-415a-a93d-ae85210865ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6809","nonce":4043981069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6811","nonce":4043981069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6815","nonce":4043981069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6813","nonce":4043981069}]},"public_addr":"172.21.15.123:6809/4043981069","cluster_addr":"172.21.15.123:6811/4043981069","heartbeat_back_addr":"172.21.15.123:6815/4043981069","heartbeat_front_addr":"172.21.15.123:6813/4043981069","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.904540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.958585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.616367+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.885244+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.103127+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.167248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.884348+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.862356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.23","mappings":[{"from":3,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:22:28.111 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.110+0000 7f1335000640 1 -- 172.21.15.121:0/870327088 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f130c06c8f0 msgr2=0x7f130c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:28.111 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.110+0000 7f1335000640 1 --2- 172.21.15.121:0/870327088 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f130c06c8f0 0x7f130c06edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1320000f60 tx=0x7f1320009040 comp rx=0 tx=0).stop 2024-04-20T10:22:28.111 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.110+0000 7f1335000640 1 -- 172.21.15.121:0/870327088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f133015ca40 msgr2=0x7f133014fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:28.111 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.110+0000 7f1335000640 1 --2- 172.21.15.121:0/870327088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f133015ca40 0x7f133014fa60 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1328009d90 tx=0x7f1328004a40 comp rx=0 tx=0).stop 2024-04-20T10:22:28.111 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.110+0000 7f1335000640 1 -- 172.21.15.121:0/870327088 shutdown_connections 2024-04-20T10:22:28.111 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.110+0000 7f1335000640 1 --2- 172.21.15.121:0/870327088 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f133015ca40 0x7f133014fa60 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:28.111 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.110+0000 7f1335000640 1 --2- 172.21.15.121:0/870327088 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f130c06c8f0 0x7f130c06edb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:28.111 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.110+0000 7f1335000640 1 --2- 172.21.15.121:0/870327088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1330156c20 0x7f133014f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:28.111 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.110+0000 7f1335000640 1 --2- 172.21.15.121:0/870327088 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f13300ad8e0 0x7f13301f4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:28.111 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.110+0000 7f1335000640 1 -- 172.21.15.121:0/870327088 >> 172.21.15.121:0/870327088 conn(0x7f13300b3350 msgr2=0x7f13300996c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:28.112 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.110+0000 7f1335000640 1 -- 172.21.15.121:0/870327088 shutdown_connections 2024-04-20T10:22:28.112 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.110+0000 7f1335000640 1 -- 172.21.15.121:0/870327088 wait complete. 2024-04-20T10:22:28.130 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:20:51.141365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:21:00.436937+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:21:01.026957+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T10:21:04.862693+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T10:22:28.131 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:22:28.337 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.336+0000 7fe0f5800640 1 Processor -- start 2024-04-20T10:22:28.337 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.336+0000 7fe0f5800640 1 -- start start 2024-04-20T10:22:28.338 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.338+0000 7fe0f5800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe0f0073b80 0x7fe0f006f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:28.339 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.338+0000 7fe0f5800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe0f0071a60 0x7fe0f006f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:28.339 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.338+0000 7fe0f5800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0f015cc20 con 0x7fe0f015a130 2024-04-20T10:22:28.339 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.338+0000 7fe0f5800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fe0f015d7f0 con 0x7fe0f0073b80 2024-04-20T10:22:28.339 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.338+0000 7fe0f5800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fe0f006fbe0 con 0x7fe0f0071a60 2024-04-20T10:22:28.339 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.338+0000 7fe0ef400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe0f0073b80 0x7fe0f006f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:28.339 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.338+0000 7fe0ef400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe0f0073b80 0x7fe0f006f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59026/0 (socket says 172.21.15.121:59026) 2024-04-20T10:22:28.339 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.338+0000 7fe0ef400640 1 -- 172.21.15.121:0/3596319892 learned_addr learned my addr 172.21.15.121:0/3596319892 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:28.340 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.339+0000 7fe0ef400640 1 -- 172.21.15.121:0/3596319892 >> v2:172.21.15.123:3300/0 conn(0x7fe0f0071a60 msgr2=0x7fe0f006f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:28.340 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.339+0000 7fe0ef400640 1 --2- 172.21.15.121:0/3596319892 >> v2:172.21.15.123:3300/0 conn(0x7fe0f0071a60 0x7fe0f006f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:28.340 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.339+0000 7fe0ef400640 1 -- 172.21.15.121:0/3596319892 >> v1:172.21.15.123:6789/0 conn(0x7fe0f015a130 legacy=0x7fe0f0073470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:28.340 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.339+0000 7fe0ef400640 1 -- 172.21.15.121:0/3596319892 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0f015cec0 con 0x7fe0f0073b80 2024-04-20T10:22:28.340 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.340+0000 7fe0ef400640 1 --2- 172.21.15.121:0/3596319892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0f0073b80 0x7fe0f006f160 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe0f00d0f90 tx=0x7fe0dc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95fbe841c2ad4e78 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:28.341 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.340+0000 7fe0eea00640 1 -- 172.21.15.121:0/3596319892 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc02e070 con 0x7fe0f0073b80 2024-04-20T10:22:28.341 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.340+0000 7fe0eea00640 1 -- 172.21.15.121:0/3596319892 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0dc037070 con 0x7fe0f0073b80 2024-04-20T10:22:28.341 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.340+0000 7fe0eea00640 1 -- 172.21.15.121:0/3596319892 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc031a00 con 0x7fe0f0073b80 2024-04-20T10:22:28.342 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.341+0000 7fe0f5800640 1 -- 172.21.15.121:0/3596319892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0f0073b80 msgr2=0x7fe0f006f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:28.342 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.341+0000 7fe0f5800640 1 --2- 172.21.15.121:0/3596319892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0f0073b80 0x7fe0f006f160 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe0f00d0f90 tx=0x7fe0dc02b880 comp rx=0 tx=0).stop 2024-04-20T10:22:28.342 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.341+0000 7fe0f5800640 1 -- 172.21.15.121:0/3596319892 shutdown_connections 2024-04-20T10:22:28.342 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.341+0000 7fe0f5800640 1 --2- 172.21.15.121:0/3596319892 >> v2:172.21.15.123:3300/0 conn(0x7fe0f0071a60 0x7fe0f006f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:28.342 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.341+0000 7fe0f5800640 1 --2- 172.21.15.121:0/3596319892 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0f0073b80 0x7fe0f006f160 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:28.342 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.341+0000 7fe0f5800640 1 -- 172.21.15.121:0/3596319892 >> 172.21.15.121:0/3596319892 conn(0x7fe0f00cffa0 msgr2=0x7fe0f00d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:28.342 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.341+0000 7fe0f5800640 1 -- 172.21.15.121:0/3596319892 shutdown_connections 2024-04-20T10:22:28.342 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.341+0000 7fe0f5800640 1 -- 172.21.15.121:0/3596319892 wait complete. 2024-04-20T10:22:28.343 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.343+0000 7fe0f5800640 1 Processor -- start 2024-04-20T10:22:28.343 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.343+0000 7fe0f5800640 1 -- start start 2024-04-20T10:22:28.344 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.344+0000 7fe0f5800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0f0071a60 0x7fe0f009a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:28.345 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.344+0000 7fe0f5800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0f0073b80 0x7fe0f01f4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:28.345 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.344+0000 7fe0f5800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0f015a130 0x7fe0f0158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:28.345 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.344+0000 7fe0f5800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f006b900 con 0x7fe0f015a130 2024-04-20T10:22:28.345 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.344+0000 7fe0f5800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe0f015d480 con 0x7fe0f0071a60 2024-04-20T10:22:28.345 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.344+0000 7fe0f5800640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe0f006ba80 con 0x7fe0f0073b80 2024-04-20T10:22:28.345 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.344+0000 7fe0f4e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0f015a130 0x7fe0f0158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:28.345 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.344+0000 7fe0f4e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0f015a130 0x7fe0f0158d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59038/0 (socket says 172.21.15.121:59038) 2024-04-20T10:22:28.345 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.344+0000 7fe0f4e00640 1 -- 172.21.15.121:0/1382556915 learned_addr learned my addr 172.21.15.121:0/1382556915 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:28.346 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.345+0000 7fe0efe00640 1 --2- 172.21.15.121:0/1382556915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0f0071a60 0x7fe0f009a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:28.347 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.345+0000 7fe0ef400640 1 --2- 172.21.15.121:0/1382556915 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0f0073b80 0x7fe0f01f4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:28.347 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.345+0000 7fe0f4e00640 1 -- 172.21.15.121:0/1382556915 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0f0073b80 msgr2=0x7fe0f01f4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:28.347 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.345+0000 7fe0f4e00640 1 --2- 172.21.15.121:0/1382556915 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0f0073b80 0x7fe0f01f4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:28.347 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.345+0000 7fe0f4e00640 1 -- 172.21.15.121:0/1382556915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0f0071a60 msgr2=0x7fe0f009a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:28.347 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.345+0000 7fe0f4e00640 1 --2- 172.21.15.121:0/1382556915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0f0071a60 0x7fe0f009a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:28.347 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.345+0000 7fe0f4e00640 1 -- 172.21.15.121:0/1382556915 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0f01f7940 con 0x7fe0f015a130 2024-04-20T10:22:28.347 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.345+0000 7fe0f4e00640 1 --2- 172.21.15.121:0/1382556915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0f015a130 0x7fe0f0158d10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe0e800b970 tx=0x7fe0e800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:28.347 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.345+0000 7fe0e7e00640 1 -- 172.21.15.121:0/1382556915 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8019070 con 0x7fe0f015a130 2024-04-20T10:22:28.347 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.345+0000 7fe0e7e00640 1 -- 172.21.15.121:0/1382556915 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0e800cb60 con 0x7fe0f015a130 2024-04-20T10:22:28.347 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.345+0000 7fe0f5800640 1 -- 172.21.15.121:0/1382556915 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0f009afd0 con 0x7fe0f015a130 2024-04-20T10:22:28.347 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.345+0000 7fe0e7e00640 1 -- 172.21.15.121:0/1382556915 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8014440 con 0x7fe0f015a130 2024-04-20T10:22:28.347 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.346+0000 7fe0f5800640 1 -- 172.21.15.121:0/1382556915 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0f01f7ee0 con 0x7fe0f015a130 2024-04-20T10:22:28.348 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.347+0000 7fe0f5800640 1 -- 172.21.15.121:0/1382556915 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0f0075b30 con 0x7fe0f015a130 2024-04-20T10:22:28.348 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.348+0000 7fe0e7e00640 1 -- 172.21.15.121:0/1382556915 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe0e801c070 con 0x7fe0f015a130 2024-04-20T10:22:28.349 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.348+0000 7fe0e7e00640 1 --2- 172.21.15.121:0/1382556915 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe0d006c8f0 0x7fe0d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:28.349 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.349+0000 7fe0e7e00640 1 -- 172.21.15.121:0/1382556915 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fe0e808bd70 con 0x7fe0f015a130 2024-04-20T10:22:28.349 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.349+0000 7fe0efe00640 1 --2- 172.21.15.121:0/1382556915 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe0d006c8f0 0x7fe0d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:28.350 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.349+0000 7fe0efe00640 1 --2- 172.21.15.121:0/1382556915 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe0d006c8f0 0x7fe0d006edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0006fd0 tx=0x7fe0e0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:28.353 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.353+0000 7fe0e7e00640 1 -- 172.21.15.121:0/1382556915 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe0e8059d00 con 0x7fe0f015a130 2024-04-20T10:22:28.557 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.556+0000 7fe0f5800640 1 -- 172.21.15.121:0/1382556915 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe0f006d010 con 0x7fe0f015a130 2024-04-20T10:22:28.558 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.557+0000 7fe0e7e00640 1 -- 172.21.15.121:0/1382556915 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fe0e8059600 con 0x7fe0f015a130 2024-04-20T10:22:28.558 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2024-04-20T10:22:28.561 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.560+0000 7fe0f5800640 1 -- 172.21.15.121:0/1382556915 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe0d006c8f0 msgr2=0x7fe0d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:28.561 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.560+0000 7fe0f5800640 1 --2- 172.21.15.121:0/1382556915 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe0d006c8f0 0x7fe0d006edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0006fd0 tx=0x7fe0e0008040 comp rx=0 tx=0).stop 2024-04-20T10:22:28.561 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.560+0000 7fe0f5800640 1 -- 172.21.15.121:0/1382556915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0f015a130 msgr2=0x7fe0f0158d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:28.561 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.560+0000 7fe0f5800640 1 --2- 172.21.15.121:0/1382556915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0f015a130 0x7fe0f0158d10 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe0e800b970 tx=0x7fe0e800be30 comp rx=0 tx=0).stop 2024-04-20T10:22:28.561 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.560+0000 7fe0f5800640 1 -- 172.21.15.121:0/1382556915 shutdown_connections 2024-04-20T10:22:28.561 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.560+0000 7fe0f5800640 1 --2- 172.21.15.121:0/1382556915 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe0f015a130 0x7fe0f0158d10 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:28.561 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.560+0000 7fe0f5800640 1 --2- 172.21.15.121:0/1382556915 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe0f0073b80 0x7fe0f01f4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:28.561 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.560+0000 7fe0f5800640 1 --2- 172.21.15.121:0/1382556915 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe0d006c8f0 0x7fe0d006edb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:28.561 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.560+0000 7fe0f5800640 1 --2- 172.21.15.121:0/1382556915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0f0071a60 0x7fe0f009a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:28.561 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.560+0000 7fe0f5800640 1 -- 172.21.15.121:0/1382556915 >> 172.21.15.121:0/1382556915 conn(0x7fe0f00cffa0 msgr2=0x7fe0f00cd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:28.561 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.560+0000 7fe0f5800640 1 -- 172.21.15.121:0/1382556915 shutdown_connections 2024-04-20T10:22:28.562 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.560+0000 7fe0f5800640 1 -- 172.21.15.121:0/1382556915 wait complete. 2024-04-20T10:22:28.579 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:22:28.785 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.783+0000 7f339be00640 1 Processor -- start 2024-04-20T10:22:28.785 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.784+0000 7f339be00640 1 -- start start 2024-04-20T10:22:28.786 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.785+0000 7f339be00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f339c161a50 0x7f339c0b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:28.786 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.785+0000 7f339be00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f339c0d3b20 0x7f339c06b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:28.786 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.785+0000 7f339be00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f339c0d0980 con 0x7f339c0d0480 2024-04-20T10:22:28.786 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.785+0000 7f339be00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f339c0d0c60 con 0x7f339c0d3b20 2024-04-20T10:22:28.787 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.785+0000 7f339be00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f339c162ae0 con 0x7f339c161a50 2024-04-20T10:22:28.787 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.785+0000 7f339a000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f339c0d3b20 0x7f339c06b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:28.787 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.785+0000 7f339a000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f339c0d3b20 0x7f339c06b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59054/0 (socket says 172.21.15.121:59054) 2024-04-20T10:22:28.787 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.785+0000 7f339a000640 1 -- 172.21.15.121:0/4114565499 learned_addr learned my addr 172.21.15.121:0/4114565499 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:28.787 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.786+0000 7f339aa00640 1 --2- 172.21.15.121:0/4114565499 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f339c161a50 0x7f339c0b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:28.787 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.786+0000 7f339a000640 1 -- 172.21.15.121:0/4114565499 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f339c161a50 msgr2=0x7f339c0b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:28.787 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.786+0000 7f339a000640 1 --2- 172.21.15.121:0/4114565499 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f339c161a50 0x7f339c0b3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:28.787 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.786+0000 7f339a000640 1 -- 172.21.15.121:0/4114565499 >> v1:172.21.15.121:6789/0 conn(0x7f339c0d0480 legacy=0x7f339c06c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:28.787 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.786+0000 7f339a000640 1 -- 172.21.15.121:0/4114565499 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f339c162490 con 0x7f339c0d3b20 2024-04-20T10:22:28.788 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.787+0000 7f339a000640 1 --2- 172.21.15.121:0/4114565499 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f339c0d3b20 0x7f339c06b900 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3384009e50 tx=0x7f338402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca7a86013a17309 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:28.788 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.787+0000 7f3399600640 1 -- 172.21.15.121:0/4114565499 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338402cb00 con 0x7f339c0d3b20 2024-04-20T10:22:28.788 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.787+0000 7f3399600640 1 -- 172.21.15.121:0/4114565499 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f338402cca0 con 0x7f339c0d3b20 2024-04-20T10:22:28.788 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.787+0000 7f3399600640 1 -- 172.21.15.121:0/4114565499 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3384034610 con 0x7f339c0d3b20 2024-04-20T10:22:28.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.788+0000 7f339be00640 1 -- 172.21.15.121:0/4114565499 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f339c0d3b20 msgr2=0x7f339c06b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:28.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.788+0000 7f339be00640 1 --2- 172.21.15.121:0/4114565499 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f339c0d3b20 0x7f339c06b900 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f3384009e50 tx=0x7f338402bf60 comp rx=0 tx=0).stop 2024-04-20T10:22:28.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.788+0000 7f339be00640 1 -- 172.21.15.121:0/4114565499 shutdown_connections 2024-04-20T10:22:28.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.788+0000 7f339be00640 1 --2- 172.21.15.121:0/4114565499 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f339c0d3b20 0x7f339c06b900 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:28.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.788+0000 7f339be00640 1 --2- 172.21.15.121:0/4114565499 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f339c161a50 0x7f339c0b3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:28.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.788+0000 7f339be00640 1 -- 172.21.15.121:0/4114565499 >> 172.21.15.121:0/4114565499 conn(0x7f339c0d0ed0 msgr2=0x7f339c0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:28.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.788+0000 7f339be00640 1 -- 172.21.15.121:0/4114565499 shutdown_connections 2024-04-20T10:22:28.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.788+0000 7f339be00640 1 -- 172.21.15.121:0/4114565499 wait complete. 2024-04-20T10:22:28.790 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.790+0000 7f339be00640 1 Processor -- start 2024-04-20T10:22:28.790 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.790+0000 7f339be00640 1 -- start start 2024-04-20T10:22:28.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.791+0000 7f339be00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f339c0d0480 0x7f339c155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:28.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.791+0000 7f339be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f339c161a50 0x7f339c156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:28.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.791+0000 7f339be00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f339c156d40 0x7f339c15b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:28.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.791+0000 7f339be00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f339c161fb0 con 0x7f339c161a50 2024-04-20T10:22:28.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.791+0000 7f339be00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f339c161e30 con 0x7f339c0d0480 2024-04-20T10:22:28.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.791+0000 7f339be00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f339c162130 con 0x7f339c156d40 2024-04-20T10:22:28.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.791+0000 7f339a000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f339c161a50 0x7f339c156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:28.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.791+0000 7f339a000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f339c161a50 0x7f339c156800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59058/0 (socket says 172.21.15.121:59058) 2024-04-20T10:22:28.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.791+0000 7f339a000640 1 -- 172.21.15.121:0/1067726306 learned_addr learned my addr 172.21.15.121:0/1067726306 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:28.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.792+0000 7f339aa00640 1 --2- 172.21.15.121:0/1067726306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f339c0d0480 0x7f339c155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:28.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.792+0000 7f339b400640 1 --2- 172.21.15.121:0/1067726306 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f339c156d40 0x7f339c15b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:28.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.792+0000 7f339a000640 1 -- 172.21.15.121:0/1067726306 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f339c156d40 msgr2=0x7f339c15b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:28.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.792+0000 7f339a000640 1 --2- 172.21.15.121:0/1067726306 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f339c156d40 0x7f339c15b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:28.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.792+0000 7f339a000640 1 -- 172.21.15.121:0/1067726306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f339c0d0480 msgr2=0x7f339c155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:28.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.792+0000 7f339a000640 1 --2- 172.21.15.121:0/1067726306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f339c0d0480 0x7f339c155d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:28.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.792+0000 7f339a000640 1 -- 172.21.15.121:0/1067726306 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f339c156260 con 0x7f339c161a50 2024-04-20T10:22:28.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.792+0000 7f339aa00640 1 --2- 172.21.15.121:0/1067726306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f339c0d0480 0x7f339c155d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:28.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.792+0000 7f339a000640 1 --2- 172.21.15.121:0/1067726306 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f339c161a50 0x7f339c156800 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3384036040 tx=0x7f3384004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:28.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.792+0000 7f3393400640 1 -- 172.21.15.121:0/1067726306 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3384043020 con 0x7f339c161a50 2024-04-20T10:22:28.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.792+0000 7f3393400640 1 -- 172.21.15.121:0/1067726306 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f338402ce50 con 0x7f339c161a50 2024-04-20T10:22:28.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.792+0000 7f3393400640 1 -- 172.21.15.121:0/1067726306 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3384009510 con 0x7f339c161a50 2024-04-20T10:22:28.795 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.792+0000 7f339be00640 1 -- 172.21.15.121:0/1067726306 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f339c157250 con 0x7f339c161a50 2024-04-20T10:22:28.795 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.793+0000 7f339be00640 1 -- 172.21.15.121:0/1067726306 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f339c157810 con 0x7f339c161a50 2024-04-20T10:22:28.795 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.794+0000 7f339be00640 1 -- 172.21.15.121:0/1067726306 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f339c075b30 con 0x7f339c161a50 2024-04-20T10:22:28.795 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.795+0000 7f3393400640 1 -- 172.21.15.121:0/1067726306 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3384046020 con 0x7f339c161a50 2024-04-20T10:22:28.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.796+0000 7f3393400640 1 --2- 172.21.15.121:0/1067726306 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f337806c8f0 0x7f337806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:28.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.796+0000 7f3393400640 1 -- 172.21.15.121:0/1067726306 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f33840ae660 con 0x7f339c161a50 2024-04-20T10:22:28.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.796+0000 7f339aa00640 1 --2- 172.21.15.121:0/1067726306 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f337806c8f0 0x7f337806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:28.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.797+0000 7f339aa00640 1 --2- 172.21.15.121:0/1067726306 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f337806c8f0 0x7f337806edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f338c006fd0 tx=0x7f338c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:28.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:28.800+0000 7f3393400640 1 -- 172.21.15.121:0/1067726306 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f338404d030 con 0x7f339c161a50 2024-04-20T10:22:29.002 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.002+0000 7f339be00640 1 -- 172.21.15.121:0/1067726306 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f339c160200 con 0x7f339c161a50 2024-04-20T10:22:29.003 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.003+0000 7f3393400640 1 -- 172.21.15.121:0/1067726306 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f338407c5f0 con 0x7f339c161a50 2024-04-20T10:22:29.003 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 32 2024-04-20T10:22:29.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.005+0000 7f339be00640 1 -- 172.21.15.121:0/1067726306 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f337806c8f0 msgr2=0x7f337806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:29.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.005+0000 7f339be00640 1 --2- 172.21.15.121:0/1067726306 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f337806c8f0 0x7f337806edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f338c006fd0 tx=0x7f338c008040 comp rx=0 tx=0).stop 2024-04-20T10:22:29.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.005+0000 7f339be00640 1 -- 172.21.15.121:0/1067726306 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f339c161a50 msgr2=0x7f339c156800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:29.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.005+0000 7f339be00640 1 --2- 172.21.15.121:0/1067726306 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f339c161a50 0x7f339c156800 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3384036040 tx=0x7f3384004a40 comp rx=0 tx=0).stop 2024-04-20T10:22:29.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.005+0000 7f339be00640 1 -- 172.21.15.121:0/1067726306 shutdown_connections 2024-04-20T10:22:29.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.005+0000 7f339be00640 1 --2- 172.21.15.121:0/1067726306 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f339c156d40 0x7f339c15b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:29.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.005+0000 7f339be00640 1 --2- 172.21.15.121:0/1067726306 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f339c161a50 0x7f339c156800 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:29.007 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.005+0000 7f339be00640 1 --2- 172.21.15.121:0/1067726306 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f337806c8f0 0x7f337806edb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:29.007 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.005+0000 7f339be00640 1 --2- 172.21.15.121:0/1067726306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f339c0d0480 0x7f339c155d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:29.007 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.005+0000 7f339be00640 1 -- 172.21.15.121:0/1067726306 >> 172.21.15.121:0/1067726306 conn(0x7f339c0d0ed0 msgr2=0x7f339c15f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:29.007 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.006+0000 7f339be00640 1 -- 172.21.15.121:0/1067726306 shutdown_connections 2024-04-20T10:22:29.007 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.006+0000 7f339be00640 1 -- 172.21.15.121:0/1067726306 wait complete. 2024-04-20T10:22:29.023 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:22:29.224 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.224+0000 7f4a16400640 1 Processor -- start 2024-04-20T10:22:29.224 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.224+0000 7f4a16400640 1 -- start start 2024-04-20T10:22:29.226 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.225+0000 7f4a16400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4a1015ca40 0x7f4a100ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:29.226 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.225+0000 7f4a16400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a100ad8e0 0x7f4a100d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:29.226 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.225+0000 7f4a16400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a10141c60 con 0x7f4a10156c20 2024-04-20T10:22:29.226 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.225+0000 7f4a16400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f4a1006c010 con 0x7f4a1015ca40 2024-04-20T10:22:29.226 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.225+0000 7f4a16400640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f4a1015cf90 con 0x7f4a100ad8e0 2024-04-20T10:22:29.226 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.226+0000 7f4a0f400640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f4a10156c20 0x7f4a1006b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:56182/0 (socket says 172.21.15.121:56182) 2024-04-20T10:22:29.226 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.226+0000 7f4a0f400640 1 -- 172.21.15.121:0/2990518577 learned_addr learned my addr 172.21.15.121:0/2990518577 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:29.227 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.226+0000 7f4a0ea00640 1 --2- 172.21.15.121:0/2990518577 >> v2:172.21.15.121:3300/0 conn(0x7f4a1015ca40 0x7f4a100ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:29.227 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.226+0000 7f4a0fe00640 1 --2- 172.21.15.121:0/2990518577 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a100ad8e0 0x7f4a100d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:29.227 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.226+0000 7f4a0e000640 1 -- 172.21.15.121:0/2990518577 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2985164829 0 0) 0x7f4a10141c60 con 0x7f4a10156c20 2024-04-20T10:22:29.228 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.226+0000 7f4a0e000640 1 -- 172.21.15.121:0/2990518577 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f49f4003520 con 0x7f4a10156c20 2024-04-20T10:22:29.228 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.226+0000 7f4a0ea00640 1 -- 172.21.15.121:0/2990518577 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a100ad8e0 msgr2=0x7f4a100d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:29.228 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.226+0000 7f4a0ea00640 1 --2- 172.21.15.121:0/2990518577 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a100ad8e0 0x7f4a100d0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:29.228 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.226+0000 7f4a0ea00640 1 -- 172.21.15.121:0/2990518577 >> v1:172.21.15.121:6789/0 conn(0x7f4a10156c20 legacy=0x7f4a1006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:29.228 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.226+0000 7f4a0ea00640 1 -- 172.21.15.121:0/2990518577 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a1015d270 con 0x7f4a1015ca40 2024-04-20T10:22:29.228 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.228+0000 7f4a0ea00640 1 --2- 172.21.15.121:0/2990518577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a1015ca40 0x7f4a100ad3a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4a00009d90 tx=0x7f4a0002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a4ed89313dd561 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:29.229 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.228+0000 7f4a0e000640 1 -- 172.21.15.121:0/2990518577 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00039070 con 0x7f4a1015ca40 2024-04-20T10:22:29.229 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.228+0000 7f4a0e000640 1 -- 172.21.15.121:0/2990518577 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a0002cc50 con 0x7f4a1015ca40 2024-04-20T10:22:29.229 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.228+0000 7f4a0e000640 1 -- 172.21.15.121:0/2990518577 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00034440 con 0x7f4a1015ca40 2024-04-20T10:22:29.230 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.229+0000 7f4a16400640 1 -- 172.21.15.121:0/2990518577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a1015ca40 msgr2=0x7f4a100ad3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:29.230 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.229+0000 7f4a16400640 1 --2- 172.21.15.121:0/2990518577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a1015ca40 0x7f4a100ad3a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4a00009d90 tx=0x7f4a0002bea0 comp rx=0 tx=0).stop 2024-04-20T10:22:29.230 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.229+0000 7f4a16400640 1 -- 172.21.15.121:0/2990518577 shutdown_connections 2024-04-20T10:22:29.230 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.229+0000 7f4a16400640 1 --2- 172.21.15.121:0/2990518577 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a100ad8e0 0x7f4a100d0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:29.230 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.229+0000 7f4a16400640 1 --2- 172.21.15.121:0/2990518577 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a1015ca40 0x7f4a100ad3a0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:29.230 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.229+0000 7f4a16400640 1 -- 172.21.15.121:0/2990518577 >> 172.21.15.121:0/2990518577 conn(0x7f4a100b3350 msgr2=0x7f4a100b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:29.230 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.229+0000 7f4a16400640 1 -- 172.21.15.121:0/2990518577 shutdown_connections 2024-04-20T10:22:29.230 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.229+0000 7f4a16400640 1 -- 172.21.15.121:0/2990518577 wait complete. 2024-04-20T10:22:29.231 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.231+0000 7f4a16400640 1 Processor -- start 2024-04-20T10:22:29.231 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.231+0000 7f4a16400640 1 -- start start 2024-04-20T10:22:29.234 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.232+0000 7f4a16400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a100ad8e0 0x7f4a101f56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:29.234 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.232+0000 7f4a16400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a10156c20 0x7f4a101f44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:29.234 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.232+0000 7f4a16400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a1015ca40 0x7f4a100b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:29.234 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.232+0000 7f4a16400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4a100d08a0 con 0x7f4a1015ca40 2024-04-20T10:22:29.235 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.232+0000 7f4a16400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1015d830 con 0x7f4a10156c20 2024-04-20T10:22:29.235 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.232+0000 7f4a16400640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f4a100d0a20 con 0x7f4a100ad8e0 2024-04-20T10:22:29.235 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.232+0000 7f4a0fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a1015ca40 0x7f4a100b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:29.235 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.232+0000 7f4a0fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a1015ca40 0x7f4a100b0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59080/0 (socket says 172.21.15.121:59080) 2024-04-20T10:22:29.235 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.232+0000 7f4a0fe00640 1 -- 172.21.15.121:0/1845707889 learned_addr learned my addr 172.21.15.121:0/1845707889 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:29.235 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.233+0000 7f4a0f400640 1 --2- 172.21.15.121:0/1845707889 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a100ad8e0 0x7f4a101f56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:29.235 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.233+0000 7f4a0ea00640 1 --2- 172.21.15.121:0/1845707889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a10156c20 0x7f4a101f44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:29.235 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.233+0000 7f4a0fe00640 1 -- 172.21.15.121:0/1845707889 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a100ad8e0 msgr2=0x7f4a101f56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:29.235 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.233+0000 7f4a0fe00640 1 --2- 172.21.15.121:0/1845707889 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a100ad8e0 0x7f4a101f56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:29.235 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.233+0000 7f4a0fe00640 1 -- 172.21.15.121:0/1845707889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a10156c20 msgr2=0x7f4a101f44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:29.235 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.233+0000 7f4a0fe00640 1 --2- 172.21.15.121:0/1845707889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a10156c20 0x7f4a101f44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:29.235 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.233+0000 7f4a0fe00640 1 -- 172.21.15.121:0/1845707889 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a101f5c30 con 0x7f4a1015ca40 2024-04-20T10:22:29.235 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.233+0000 7f4a0ea00640 1 --2- 172.21.15.121:0/1845707889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a10156c20 0x7f4a101f44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:29.235 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.233+0000 7f4a0fe00640 1 --2- 172.21.15.121:0/1845707889 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a1015ca40 0x7f4a100b0fd0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4a0800d4f0 tx=0x7f4a0800d9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:29.236 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.233+0000 7f4a07400640 1 -- 172.21.15.121:0/1845707889 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a080125f0 con 0x7f4a1015ca40 2024-04-20T10:22:29.236 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.233+0000 7f4a07400640 1 -- 172.21.15.121:0/1845707889 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a08019070 con 0x7f4a1015ca40 2024-04-20T10:22:29.236 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.233+0000 7f4a07400640 1 -- 172.21.15.121:0/1845707889 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a080129f0 con 0x7f4a1015ca40 2024-04-20T10:22:29.236 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.234+0000 7f4a16400640 1 -- 172.21.15.121:0/1845707889 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a10153bf0 con 0x7f4a1015ca40 2024-04-20T10:22:29.236 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.234+0000 7f4a16400640 1 -- 172.21.15.121:0/1845707889 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a10154490 con 0x7f4a1015ca40 2024-04-20T10:22:29.236 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.235+0000 7f4a16400640 1 -- 172.21.15.121:0/1845707889 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a10068980 con 0x7f4a1015ca40 2024-04-20T10:22:29.236 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.236+0000 7f4a07400640 1 -- 172.21.15.121:0/1845707889 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4a08012b90 con 0x7f4a1015ca40 2024-04-20T10:22:29.237 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.237+0000 7f4a07400640 1 --2- 172.21.15.121:0/1845707889 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f49ec06c8f0 0x7f49ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:29.237 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.237+0000 7f4a07400640 1 -- 172.21.15.121:0/1845707889 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f4a0808c2c0 con 0x7f4a1015ca40 2024-04-20T10:22:29.237 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.237+0000 7f4a0f400640 1 --2- 172.21.15.121:0/1845707889 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f49ec06c8f0 0x7f49ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:29.238 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.238+0000 7f4a0f400640 1 --2- 172.21.15.121:0/1845707889 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f49ec06c8f0 0x7f49ec06edb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f49f8007fd0 tx=0x7f49f8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:29.242 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.241+0000 7f4a07400640 1 -- 172.21.15.121:0/1845707889 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4a101f5c30 con 0x7f4a1015ca40 2024-04-20T10:22:29.440 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.440+0000 7f4a16400640 1 -- 172.21.15.121:0/1845707889 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4a100cfd90 con 0x7f4a1015ca40 2024-04-20T10:22:29.441 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.440+0000 7f4a07400640 1 -- 172.21.15.121:0/1845707889 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f4a0805a2e0 con 0x7f4a1015ca40 2024-04-20T10:22:29.441 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2024-04-20T10:22:29.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.443+0000 7f4a16400640 1 -- 172.21.15.121:0/1845707889 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f49ec06c8f0 msgr2=0x7f49ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:29.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.443+0000 7f4a16400640 1 --2- 172.21.15.121:0/1845707889 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f49ec06c8f0 0x7f49ec06edb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f49f8007fd0 tx=0x7f49f8009040 comp rx=0 tx=0).stop 2024-04-20T10:22:29.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.443+0000 7f4a16400640 1 -- 172.21.15.121:0/1845707889 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a1015ca40 msgr2=0x7f4a100b0fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:29.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.443+0000 7f4a16400640 1 --2- 172.21.15.121:0/1845707889 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a1015ca40 0x7f4a100b0fd0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4a0800d4f0 tx=0x7f4a0800d9b0 comp rx=0 tx=0).stop 2024-04-20T10:22:29.445 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.443+0000 7f4a16400640 1 -- 172.21.15.121:0/1845707889 shutdown_connections 2024-04-20T10:22:29.445 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.443+0000 7f4a16400640 1 --2- 172.21.15.121:0/1845707889 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a1015ca40 0x7f4a100b0fd0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:29.445 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.443+0000 7f4a16400640 1 --2- 172.21.15.121:0/1845707889 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f49ec06c8f0 0x7f49ec06edb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:29.445 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.443+0000 7f4a16400640 1 --2- 172.21.15.121:0/1845707889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a10156c20 0x7f4a101f44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:29.445 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.443+0000 7f4a16400640 1 --2- 172.21.15.121:0/1845707889 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a100ad8e0 0x7f4a101f56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:29.445 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.443+0000 7f4a16400640 1 -- 172.21.15.121:0/1845707889 >> 172.21.15.121:0/1845707889 conn(0x7f4a100b3350 msgr2=0x7f4a1015b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:29.445 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.444+0000 7f4a16400640 1 -- 172.21.15.121:0/1845707889 shutdown_connections 2024-04-20T10:22:29.445 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.444+0000 7f4a16400640 1 -- 172.21.15.121:0/1845707889 wait complete. 2024-04-20T10:22:29.464 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T10:22:29.673 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.672+0000 7fb547200640 1 Processor -- start 2024-04-20T10:22:29.673 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.672+0000 7fb547200640 1 -- start start 2024-04-20T10:22:29.674 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.674+0000 7fb547200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb54015b3b0 0x7fb54006eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:29.675 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.674+0000 7fb547200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb5400d0360 0x7fb5400d0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:29.675 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.674+0000 7fb547200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5400ade40 con 0x7fb54006dcb0 2024-04-20T10:22:29.675 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.674+0000 7fb547200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fb5400ae0e0 con 0x7fb54015b3b0 2024-04-20T10:22:29.675 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.674+0000 7fb547200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fb54015b790 con 0x7fb5400d0360 2024-04-20T10:22:29.675 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.674+0000 7fb53fe00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb54015b3b0 0x7fb54006eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:29.675 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.674+0000 7fb537e00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fb54006dcb0 0x7fb540071b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:56184/0 (socket says 172.21.15.121:56184) 2024-04-20T10:22:29.675 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.674+0000 7fb53fe00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fb54015b3b0 0x7fb54006eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59090/0 (socket says 172.21.15.121:59090) 2024-04-20T10:22:29.675 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.674+0000 7fb537e00640 1 -- 172.21.15.121:0/3003257950 learned_addr learned my addr 172.21.15.121:0/3003257950 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:29.676 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.675+0000 7fb545200640 1 --2- 172.21.15.121:0/3003257950 >> v2:172.21.15.123:3300/0 conn(0x7fb5400d0360 0x7fb5400d0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:29.676 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.675+0000 7fb53fe00640 1 -- 172.21.15.121:0/3003257950 >> v2:172.21.15.123:3300/0 conn(0x7fb5400d0360 msgr2=0x7fb5400d0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:29.676 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.675+0000 7fb53fe00640 1 --2- 172.21.15.121:0/3003257950 >> v2:172.21.15.123:3300/0 conn(0x7fb5400d0360 0x7fb5400d0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:29.676 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.675+0000 7fb53fe00640 1 -- 172.21.15.121:0/3003257950 >> v1:172.21.15.121:6789/0 conn(0x7fb54006dcb0 legacy=0x7fb540071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:29.676 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.675+0000 7fb53fe00640 1 -- 172.21.15.121:0/3003257950 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb540072260 con 0x7fb54015b3b0 2024-04-20T10:22:29.677 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.676+0000 7fb53fe00640 1 --2- 172.21.15.121:0/3003257950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb54015b3b0 0x7fb54006eaa0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb540073c80 tx=0x7fb53002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7a0c6dc3a3fcf52 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:29.678 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.676+0000 7fb53f400640 1 -- 172.21.15.121:0/3003257950 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb530035020 con 0x7fb54015b3b0 2024-04-20T10:22:29.678 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.676+0000 7fb53f400640 1 -- 172.21.15.121:0/3003257950 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb530031830 con 0x7fb54015b3b0 2024-04-20T10:22:29.678 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.676+0000 7fb53f400640 1 -- 172.21.15.121:0/3003257950 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb530031c00 con 0x7fb54015b3b0 2024-04-20T10:22:29.678 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.677+0000 7fb547200640 1 -- 172.21.15.121:0/3003257950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb54015b3b0 msgr2=0x7fb54006eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:29.678 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.677+0000 7fb547200640 1 --2- 172.21.15.121:0/3003257950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb54015b3b0 0x7fb54006eaa0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb540073c80 tx=0x7fb53002b8d0 comp rx=0 tx=0).stop 2024-04-20T10:22:29.678 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.677+0000 7fb547200640 1 -- 172.21.15.121:0/3003257950 shutdown_connections 2024-04-20T10:22:29.678 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.677+0000 7fb547200640 1 --2- 172.21.15.121:0/3003257950 >> v2:172.21.15.123:3300/0 conn(0x7fb5400d0360 0x7fb5400d0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:29.678 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.677+0000 7fb547200640 1 --2- 172.21.15.121:0/3003257950 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb54015b3b0 0x7fb54006eaa0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:29.678 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.677+0000 7fb547200640 1 -- 172.21.15.121:0/3003257950 >> 172.21.15.121:0/3003257950 conn(0x7fb5400b3350 msgr2=0x7fb5400b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:29.678 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.677+0000 7fb547200640 1 -- 172.21.15.121:0/3003257950 shutdown_connections 2024-04-20T10:22:29.678 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.677+0000 7fb547200640 1 -- 172.21.15.121:0/3003257950 wait complete. 2024-04-20T10:22:29.679 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.679+0000 7fb547200640 1 Processor -- start 2024-04-20T10:22:29.679 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.679+0000 7fb547200640 1 -- start start 2024-04-20T10:22:29.681 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.680+0000 7fb547200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb54006dcb0 0x7fb54015b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:29.681 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.680+0000 7fb547200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5400d0360 0x7fb5401f8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:29.681 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.680+0000 7fb547200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb54015b3b0 0x7fb5401f9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:29.681 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.680+0000 7fb547200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fb5400ae940 con 0x7fb54015b3b0 2024-04-20T10:22:29.681 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.680+0000 7fb547200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb5400ae660 con 0x7fb54006dcb0 2024-04-20T10:22:29.681 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.680+0000 7fb547200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fb5400aec20 con 0x7fb5400d0360 2024-04-20T10:22:29.682 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.680+0000 7fb545200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb54015b3b0 0x7fb5401f9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:29.682 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.680+0000 7fb545200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb54015b3b0 0x7fb5401f9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59092/0 (socket says 172.21.15.121:59092) 2024-04-20T10:22:29.682 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.680+0000 7fb545200640 1 -- 172.21.15.121:0/1455411997 learned_addr learned my addr 172.21.15.121:0/1455411997 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:29.682 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.681+0000 7fb53fe00640 1 --2- 172.21.15.121:0/1455411997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb54006dcb0 0x7fb54015b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:29.682 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.681+0000 7fb537e00640 1 --2- 172.21.15.121:0/1455411997 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5400d0360 0x7fb5401f8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:29.682 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.681+0000 7fb545200640 1 -- 172.21.15.121:0/1455411997 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5400d0360 msgr2=0x7fb5401f8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:29.682 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.681+0000 7fb545200640 1 --2- 172.21.15.121:0/1455411997 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5400d0360 0x7fb5401f8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:29.682 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.681+0000 7fb545200640 1 -- 172.21.15.121:0/1455411997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb54006dcb0 msgr2=0x7fb54015b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:29.682 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.681+0000 7fb545200640 1 --2- 172.21.15.121:0/1455411997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb54006dcb0 0x7fb54015b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:29.682 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.681+0000 7fb545200640 1 -- 172.21.15.121:0/1455411997 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb54015bf50 con 0x7fb54015b3b0 2024-04-20T10:22:29.683 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.681+0000 7fb545200640 1 --2- 172.21.15.121:0/1455411997 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb54015b3b0 0x7fb5401f9170 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb538007d90 tx=0x7fb53800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:29.683 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.681+0000 7fb53cc00640 1 -- 172.21.15.121:0/1455411997 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb538014420 con 0x7fb54015b3b0 2024-04-20T10:22:29.683 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.681+0000 7fb53cc00640 1 -- 172.21.15.121:0/1455411997 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb538019070 con 0x7fb54015b3b0 2024-04-20T10:22:29.683 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.681+0000 7fb53cc00640 1 -- 172.21.15.121:0/1455411997 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb538014820 con 0x7fb54015b3b0 2024-04-20T10:22:29.684 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.681+0000 7fb547200640 1 -- 172.21.15.121:0/1455411997 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5401fbe40 con 0x7fb54015b3b0 2024-04-20T10:22:29.684 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.682+0000 7fb547200640 1 -- 172.21.15.121:0/1455411997 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5401fc920 con 0x7fb54015b3b0 2024-04-20T10:22:29.684 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.683+0000 7fb547200640 1 -- 172.21.15.121:0/1455411997 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb540068980 con 0x7fb54015b3b0 2024-04-20T10:22:29.684 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.684+0000 7fb53cc00640 1 -- 172.21.15.121:0/1455411997 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb5380149c0 con 0x7fb54015b3b0 2024-04-20T10:22:29.685 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.684+0000 7fb53cc00640 1 --2- 172.21.15.121:0/1455411997 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fb51c06c920 0x7fb51c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:29.685 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.685+0000 7fb53cc00640 1 -- 172.21.15.121:0/1455411997 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fb53808c790 con 0x7fb54015b3b0 2024-04-20T10:22:29.685 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.685+0000 7fb53fe00640 1 --2- 172.21.15.121:0/1455411997 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fb51c06c920 0x7fb51c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:29.686 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.686+0000 7fb53fe00640 1 --2- 172.21.15.121:0/1455411997 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fb51c06c920 0x7fb51c06ede0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb530036000 tx=0x7fb53003c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:29.689 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.689+0000 7fb53cc00640 1 -- 172.21.15.121:0/1455411997 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb53805a7b0 con 0x7fb54015b3b0 2024-04-20T10:22:29.886 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.886+0000 7fb547200640 1 -- 172.21.15.121:0/1455411997 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb5401fcbb0 con 0x7fb54015b3b0 2024-04-20T10:22:29.887 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.887+0000 7fb53cc00640 1 -- 172.21.15.121:0/1455411997 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fb53805a0b0 con 0x7fb54015b3b0 2024-04-20T10:22:29.887 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 32 2024-04-20T10:22:29.890 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.889+0000 7fb547200640 1 -- 172.21.15.121:0/1455411997 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fb51c06c920 msgr2=0x7fb51c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:29.890 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.889+0000 7fb547200640 1 --2- 172.21.15.121:0/1455411997 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fb51c06c920 0x7fb51c06ede0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb530036000 tx=0x7fb53003c040 comp rx=0 tx=0).stop 2024-04-20T10:22:29.890 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.889+0000 7fb547200640 1 -- 172.21.15.121:0/1455411997 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb54015b3b0 msgr2=0x7fb5401f9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:29.890 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.889+0000 7fb547200640 1 --2- 172.21.15.121:0/1455411997 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb54015b3b0 0x7fb5401f9170 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb538007d90 tx=0x7fb53800e4b0 comp rx=0 tx=0).stop 2024-04-20T10:22:29.890 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.889+0000 7fb547200640 1 -- 172.21.15.121:0/1455411997 shutdown_connections 2024-04-20T10:22:29.890 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.889+0000 7fb547200640 1 --2- 172.21.15.121:0/1455411997 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fb54015b3b0 0x7fb5401f9170 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:29.890 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.889+0000 7fb547200640 1 --2- 172.21.15.121:0/1455411997 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fb5400d0360 0x7fb5401f8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:29.890 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.889+0000 7fb547200640 1 --2- 172.21.15.121:0/1455411997 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fb51c06c920 0x7fb51c06ede0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:29.890 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.889+0000 7fb547200640 1 --2- 172.21.15.121:0/1455411997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb54006dcb0 0x7fb54015b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:29.890 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.889+0000 7fb547200640 1 -- 172.21.15.121:0/1455411997 >> 172.21.15.121:0/1455411997 conn(0x7fb5400b3350 msgr2=0x7fb54006b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:29.891 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.889+0000 7fb547200640 1 -- 172.21.15.121:0/1455411997 shutdown_connections 2024-04-20T10:22:29.891 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:29.889+0000 7fb547200640 1 -- 172.21.15.121:0/1455411997 wait complete. 2024-04-20T10:22:29.908 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T10:22:30.105 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.104+0000 7fdb46a00640 1 Processor -- start 2024-04-20T10:22:30.105 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.104+0000 7fdb46a00640 1 -- start start 2024-04-20T10:22:30.106 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.106+0000 7fdb46a00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb40159e10 0x7fdb40071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:30.107 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.106+0000 7fdb46a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdb4015a1f0 0x7fdb40071fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:30.107 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.106+0000 7fdb46a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb4015cc90 con 0x7fdb4006da90 2024-04-20T10:22:30.107 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.106+0000 7fdb46a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fdb40072900 con 0x7fdb4015a1f0 2024-04-20T10:22:30.107 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.106+0000 7fdb46a00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fdb4015ca40 con 0x7fdb40159e10 2024-04-20T10:22:30.107 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.106+0000 7fdb44c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdb4015a1f0 0x7fdb40071fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:30.107 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.106+0000 7fdb44c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdb4015a1f0 0x7fdb40071fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59096/0 (socket says 172.21.15.121:59096) 2024-04-20T10:22:30.107 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.106+0000 7fdb44c00640 1 -- 172.21.15.121:0/913286369 learned_addr learned my addr 172.21.15.121:0/913286369 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:30.107 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.106+0000 7fdb45600640 1 --2- 172.21.15.121:0/913286369 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb40159e10 0x7fdb40071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:30.108 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.107+0000 7fdb44c00640 1 -- 172.21.15.121:0/913286369 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb40159e10 msgr2=0x7fdb40071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:30.108 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.107+0000 7fdb44c00640 1 --2- 172.21.15.121:0/913286369 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb40159e10 0x7fdb40071a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:30.108 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.107+0000 7fdb44c00640 1 -- 172.21.15.121:0/913286369 >> v1:172.21.15.123:6789/0 conn(0x7fdb4006da90 legacy=0x7fdb40160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:30.108 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.107+0000 7fdb44c00640 1 -- 172.21.15.121:0/913286369 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb4015cf70 con 0x7fdb4015a1f0 2024-04-20T10:22:30.109 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.108+0000 7fdb44c00640 1 --2- 172.21.15.121:0/913286369 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb4015a1f0 0x7fdb40071fc0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fdb4015cbc0 tx=0x7fdb2c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c698092b34a703a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:30.109 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.108+0000 7fdb3fe00640 1 -- 172.21.15.121:0/913286369 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c035020 con 0x7fdb4015a1f0 2024-04-20T10:22:30.109 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.108+0000 7fdb3fe00640 1 -- 172.21.15.121:0/913286369 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb2c031640 con 0x7fdb4015a1f0 2024-04-20T10:22:30.109 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.108+0000 7fdb3fe00640 1 -- 172.21.15.121:0/913286369 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c02d040 con 0x7fdb4015a1f0 2024-04-20T10:22:30.109 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.109+0000 7fdb46a00640 1 -- 172.21.15.121:0/913286369 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb4015a1f0 msgr2=0x7fdb40071fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:30.110 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.109+0000 7fdb46a00640 1 --2- 172.21.15.121:0/913286369 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb4015a1f0 0x7fdb40071fc0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fdb4015cbc0 tx=0x7fdb2c02b8f0 comp rx=0 tx=0).stop 2024-04-20T10:22:30.110 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.109+0000 7fdb46a00640 1 -- 172.21.15.121:0/913286369 shutdown_connections 2024-04-20T10:22:30.110 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.109+0000 7fdb46a00640 1 --2- 172.21.15.121:0/913286369 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb4015a1f0 0x7fdb40071fc0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:30.110 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.109+0000 7fdb46a00640 1 --2- 172.21.15.121:0/913286369 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb40159e10 0x7fdb40071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:30.110 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.109+0000 7fdb46a00640 1 -- 172.21.15.121:0/913286369 >> 172.21.15.121:0/913286369 conn(0x7fdb400d0040 msgr2=0x7fdb400d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:30.110 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.109+0000 7fdb46a00640 1 -- 172.21.15.121:0/913286369 shutdown_connections 2024-04-20T10:22:30.110 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.109+0000 7fdb46a00640 1 -- 172.21.15.121:0/913286369 wait complete. 2024-04-20T10:22:30.111 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.111+0000 7fdb46a00640 1 Processor -- start 2024-04-20T10:22:30.111 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.111+0000 7fdb46a00640 1 -- start start 2024-04-20T10:22:30.114 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.112+0000 7fdb46a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb4006da90 0x7fdb4015de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:30.114 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.112+0000 7fdb46a00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb40159e10 0x7fdb40152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:30.114 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.112+0000 7fdb46a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb4015a1f0 0x7fdb400b4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:30.114 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.112+0000 7fdb46a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4015d810 con 0x7fdb4006da90 2024-04-20T10:22:30.114 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.112+0000 7fdb46a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4015d530 con 0x7fdb4015a1f0 2024-04-20T10:22:30.114 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.112+0000 7fdb46a00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fdb4006b900 con 0x7fdb40159e10 2024-04-20T10:22:30.114 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.112+0000 7fdb45600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb4006da90 0x7fdb4015de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:30.114 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.112+0000 7fdb45600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb4006da90 0x7fdb4015de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59100/0 (socket says 172.21.15.121:59100) 2024-04-20T10:22:30.114 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.112+0000 7fdb45600640 1 -- 172.21.15.121:0/3946561786 learned_addr learned my addr 172.21.15.121:0/3946561786 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:30.114 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.112+0000 7fdb44c00640 1 --2- 172.21.15.121:0/3946561786 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb40159e10 0x7fdb40152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:30.115 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.113+0000 7fdb46000640 1 --2- 172.21.15.121:0/3946561786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb4015a1f0 0x7fdb400b4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:30.115 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.113+0000 7fdb45600640 1 -- 172.21.15.121:0/3946561786 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb40159e10 msgr2=0x7fdb40152500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:30.115 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.113+0000 7fdb45600640 1 --2- 172.21.15.121:0/3946561786 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb40159e10 0x7fdb40152500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:30.115 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.113+0000 7fdb45600640 1 -- 172.21.15.121:0/3946561786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb4015a1f0 msgr2=0x7fdb400b4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:30.115 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.113+0000 7fdb45600640 1 --2- 172.21.15.121:0/3946561786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb4015a1f0 0x7fdb400b4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:30.115 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.113+0000 7fdb45600640 1 -- 172.21.15.121:0/3946561786 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb4015e580 con 0x7fdb4006da90 2024-04-20T10:22:30.115 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.113+0000 7fdb44c00640 1 --2- 172.21.15.121:0/3946561786 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb40159e10 0x7fdb40152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:30.115 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.113+0000 7fdb46000640 1 --2- 172.21.15.121:0/3946561786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb4015a1f0 0x7fdb400b4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:30.115 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.113+0000 7fdb45600640 1 --2- 172.21.15.121:0/3946561786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb4006da90 0x7fdb4015de80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fdb3400d830 tx=0x7fdb3400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:30.115 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.113+0000 7fdb3d600640 1 -- 172.21.15.121:0/3946561786 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3400b6c0 con 0x7fdb4006da90 2024-04-20T10:22:30.115 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.113+0000 7fdb3d600640 1 -- 172.21.15.121:0/3946561786 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb34019070 con 0x7fdb4006da90 2024-04-20T10:22:30.115 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.113+0000 7fdb3d600640 1 -- 172.21.15.121:0/3946561786 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3400ba80 con 0x7fdb4006da90 2024-04-20T10:22:30.115 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.113+0000 7fdb46a00640 1 -- 172.21.15.121:0/3946561786 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb4015e860 con 0x7fdb4006da90 2024-04-20T10:22:30.115 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.113+0000 7fdb46a00640 1 -- 172.21.15.121:0/3946561786 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb400c95d0 con 0x7fdb4006da90 2024-04-20T10:22:30.116 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.114+0000 7fdb46a00640 1 -- 172.21.15.121:0/3946561786 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb40075b60 con 0x7fdb4006da90 2024-04-20T10:22:30.116 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.115+0000 7fdb3d600640 1 -- 172.21.15.121:0/3946561786 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdb3401e460 con 0x7fdb4006da90 2024-04-20T10:22:30.117 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.116+0000 7fdb3d600640 1 --2- 172.21.15.121:0/3946561786 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fdb2006c8f0 0x7fdb2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:30.117 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.116+0000 7fdb3d600640 1 -- 172.21.15.121:0/3946561786 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fdb3408c6d0 con 0x7fdb4006da90 2024-04-20T10:22:30.117 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.117+0000 7fdb44c00640 1 --2- 172.21.15.121:0/3946561786 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fdb2006c8f0 0x7fdb2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:30.117 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.117+0000 7fdb44c00640 1 --2- 172.21.15.121:0/3946561786 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fdb2006c8f0 0x7fdb2006edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c036000 tx=0x7fdb2c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:30.121 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.121+0000 7fdb3d600640 1 -- 172.21.15.121:0/3946561786 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdb3405a660 con 0x7fdb4006da90 2024-04-20T10:22:30.332 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.332+0000 7fdb46a00640 1 -- 172.21.15.121:0/3946561786 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdb4015c800 con 0x7fdb4006da90 2024-04-20T10:22:30.333 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.332+0000 7fdb3d600640 1 -- 172.21.15.121:0/3946561786 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7fdb34059f60 con 0x7fdb4006da90 2024-04-20T10:22:30.333 INFO:teuthology.orchestra.run.smithi121.stderr:listed 0 entries 2024-04-20T10:22:30.336 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.335+0000 7fdb46a00640 1 -- 172.21.15.121:0/3946561786 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fdb2006c8f0 msgr2=0x7fdb2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:30.336 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.335+0000 7fdb46a00640 1 --2- 172.21.15.121:0/3946561786 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fdb2006c8f0 0x7fdb2006edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c036000 tx=0x7fdb2c008030 comp rx=0 tx=0).stop 2024-04-20T10:22:30.336 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.335+0000 7fdb46a00640 1 -- 172.21.15.121:0/3946561786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb4006da90 msgr2=0x7fdb4015de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:30.336 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.335+0000 7fdb46a00640 1 --2- 172.21.15.121:0/3946561786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb4006da90 0x7fdb4015de80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fdb3400d830 tx=0x7fdb3400dcf0 comp rx=0 tx=0).stop 2024-04-20T10:22:30.336 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.335+0000 7fdb46a00640 1 -- 172.21.15.121:0/3946561786 shutdown_connections 2024-04-20T10:22:30.336 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.335+0000 7fdb46a00640 1 --2- 172.21.15.121:0/3946561786 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fdb2006c8f0 0x7fdb2006edb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:30.336 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.335+0000 7fdb46a00640 1 --2- 172.21.15.121:0/3946561786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb4015a1f0 0x7fdb400b4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:30.336 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.335+0000 7fdb46a00640 1 --2- 172.21.15.121:0/3946561786 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdb40159e10 0x7fdb40152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:30.336 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.335+0000 7fdb46a00640 1 --2- 172.21.15.121:0/3946561786 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdb4006da90 0x7fdb4015de80 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:30.336 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.335+0000 7fdb46a00640 1 -- 172.21.15.121:0/3946561786 >> 172.21.15.121:0/3946561786 conn(0x7fdb400d0040 msgr2=0x7fdb4006c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:30.336 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.335+0000 7fdb46a00640 1 -- 172.21.15.121:0/3946561786 shutdown_connections 2024-04-20T10:22:30.337 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.335+0000 7fdb46a00640 1 -- 172.21.15.121:0/3946561786 wait complete. 2024-04-20T10:22:30.354 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-04-20T10:22:30.354 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:22:30.410 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:22:30.614 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.613+0000 7f5a8f200640 1 Processor -- start 2024-04-20T10:22:30.614 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.613+0000 7f5a8f200640 1 -- start start 2024-04-20T10:22:30.616 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.615+0000 7f5a8f200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5a9006c960 0x7f5a9006cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:30.616 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.615+0000 7f5a8f200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a9006b900 con 0x7f5a9015a7b0 2024-04-20T10:22:30.616 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.615+0000 7f5a8f200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a9006bba0 con 0x7f5a90071c90 2024-04-20T10:22:30.616 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.615+0000 7f5a8f200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f5a900b50a0 con 0x7f5a9006c960 2024-04-20T10:22:30.616 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.615+0000 7f5a8d400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5a9006c960 0x7f5a9006cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:30.616 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.615+0000 7f5a8de00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f5a9015a7b0 0x7f5a9006da90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:56188/0 (socket says 172.21.15.121:56188) 2024-04-20T10:22:30.616 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.615+0000 7f5a8de00640 1 -- 172.21.15.121:0/3596135430 learned_addr learned my addr 172.21.15.121:0/3596135430 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:30.617 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.616+0000 7f5a8d400640 1 -- 172.21.15.121:0/3596135430 >> v1:172.21.15.123:6789/0 conn(0x7f5a90071c90 legacy=0x7f5a900b3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:30.617 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.616+0000 7f5a8d400640 1 -- 172.21.15.121:0/3596135430 >> v1:172.21.15.121:6789/0 conn(0x7f5a9015a7b0 legacy=0x7f5a9006da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:30.617 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.616+0000 7f5a8d400640 1 -- 172.21.15.121:0/3596135430 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a9006be80 con 0x7f5a9006c960 2024-04-20T10:22:30.617 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.617+0000 7f5a8d400640 1 --2- 172.21.15.121:0/3596135430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a9006c960 0x7f5a9006cdc0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c0097a0 tx=0x7f5a7c02dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3218a8a719ab710 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:30.618 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.617+0000 7f5a8ca00640 1 -- 172.21.15.121:0/3596135430 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c02f8a0 con 0x7f5a9006c960 2024-04-20T10:22:30.618 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.617+0000 7f5a8ca00640 1 -- 172.21.15.121:0/3596135430 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a7c02fa40 con 0x7f5a9006c960 2024-04-20T10:22:30.618 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.617+0000 7f5a8ca00640 1 -- 172.21.15.121:0/3596135430 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a7c02fe10 con 0x7f5a9006c960 2024-04-20T10:22:30.619 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.618+0000 7f5a8f200640 1 -- 172.21.15.121:0/3596135430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a9006c960 msgr2=0x7f5a9006cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:30.619 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.618+0000 7f5a8f200640 1 --2- 172.21.15.121:0/3596135430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a9006c960 0x7f5a9006cdc0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c0097a0 tx=0x7f5a7c02dae0 comp rx=0 tx=0).stop 2024-04-20T10:22:30.619 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.618+0000 7f5a8f200640 1 -- 172.21.15.121:0/3596135430 shutdown_connections 2024-04-20T10:22:30.619 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.618+0000 7f5a8f200640 1 --2- 172.21.15.121:0/3596135430 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a9006c960 0x7f5a9006cdc0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:30.619 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.618+0000 7f5a8f200640 1 -- 172.21.15.121:0/3596135430 >> 172.21.15.121:0/3596135430 conn(0x7f5a900d0040 msgr2=0x7f5a900d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:30.619 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.618+0000 7f5a8f200640 1 -- 172.21.15.121:0/3596135430 shutdown_connections 2024-04-20T10:22:30.619 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.618+0000 7f5a8f200640 1 -- 172.21.15.121:0/3596135430 wait complete. 2024-04-20T10:22:30.620 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.620+0000 7f5a8f200640 1 Processor -- start 2024-04-20T10:22:30.620 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.620+0000 7f5a8f200640 1 -- start start 2024-04-20T10:22:30.621 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.621+0000 7f5a8f200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a9006c960 0x7f5a90149970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:30.621 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.621+0000 7f5a8f200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a90071c90 0x7f5a9009ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:30.621 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.621+0000 7f5a8f200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a9015a7b0 0x7f5a901f3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:30.621 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.621+0000 7f5a8f200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9006e1a0 con 0x7f5a9015a7b0 2024-04-20T10:22:30.622 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.621+0000 7f5a8f200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5a9006c440 con 0x7f5a9006c960 2024-04-20T10:22:30.622 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.621+0000 7f5a8f200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f5a90152f90 con 0x7f5a90071c90 2024-04-20T10:22:30.622 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.621+0000 7f5a8e800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a9015a7b0 0x7f5a901f3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:30.622 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.621+0000 7f5a8e800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a9015a7b0 0x7f5a901f3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59118/0 (socket says 172.21.15.121:59118) 2024-04-20T10:22:30.622 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.621+0000 7f5a8e800640 1 -- 172.21.15.121:0/2094458836 learned_addr learned my addr 172.21.15.121:0/2094458836 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:30.623 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.621+0000 7f5a8e800640 1 -- 172.21.15.121:0/2094458836 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a90071c90 msgr2=0x7f5a9009ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:30.623 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.621+0000 7f5a8e800640 1 --2- 172.21.15.121:0/2094458836 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a90071c90 0x7f5a9009ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:30.623 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.621+0000 7f5a8e800640 1 -- 172.21.15.121:0/2094458836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a9006c960 msgr2=0x7f5a90149970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:30.623 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.621+0000 7f5a8e800640 1 --2- 172.21.15.121:0/2094458836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a9006c960 0x7f5a90149970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:30.623 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.621+0000 7f5a8e800640 1 -- 172.21.15.121:0/2094458836 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a90153270 con 0x7f5a9015a7b0 2024-04-20T10:22:30.623 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.622+0000 7f5a8e800640 1 --2- 172.21.15.121:0/2094458836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a9015a7b0 0x7f5a901f3700 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5a8800c780 tx=0x7f5a8800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:30.623 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.622+0000 7f5a86000640 1 -- 172.21.15.121:0/2094458836 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a88004b60 con 0x7f5a9015a7b0 2024-04-20T10:22:30.623 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.622+0000 7f5a86000640 1 -- 172.21.15.121:0/2094458836 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a88004d00 con 0x7f5a9015a7b0 2024-04-20T10:22:30.623 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.622+0000 7f5a86000640 1 -- 172.21.15.121:0/2094458836 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8800aa90 con 0x7f5a9015a7b0 2024-04-20T10:22:30.623 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.622+0000 7f5a8f200640 1 -- 172.21.15.121:0/2094458836 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a90153550 con 0x7f5a9015a7b0 2024-04-20T10:22:30.624 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.622+0000 7f5a8f200640 1 -- 172.21.15.121:0/2094458836 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a90149eb0 con 0x7f5a9015a7b0 2024-04-20T10:22:30.624 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.623+0000 7f5a8f200640 1 -- 172.21.15.121:0/2094458836 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a90075b60 con 0x7f5a9015a7b0 2024-04-20T10:22:30.624 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.624+0000 7f5a86000640 1 -- 172.21.15.121:0/2094458836 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5a8800ac30 con 0x7f5a9015a7b0 2024-04-20T10:22:30.625 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.625+0000 7f5a86000640 1 --2- 172.21.15.121:0/2094458836 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f5a7006c8f0 0x7f5a7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:30.625 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.625+0000 7f5a86000640 1 -- 172.21.15.121:0/2094458836 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f5a8808c260 con 0x7f5a9015a7b0 2024-04-20T10:22:30.626 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.625+0000 7f5a8de00640 1 --2- 172.21.15.121:0/2094458836 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f5a7006c8f0 0x7f5a7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:30.626 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.626+0000 7f5a8de00640 1 --2- 172.21.15.121:0/2094458836 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f5a7006c8f0 0x7f5a7006edb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5a80005880 tx=0x7f5a80007a20 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:30.630 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.630+0000 7f5a86000640 1 -- 172.21.15.121:0/2094458836 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5a88020070 con 0x7f5a9015a7b0 2024-04-20T10:22:30.834 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.834+0000 7f5a8f200640 1 -- 172.21.15.121:0/2094458836 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5a9015f320 con 0x7f5a9015a7b0 2024-04-20T10:22:30.837 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.837+0000 7f5a86000640 1 -- 172.21.15.121:0/2094458836 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18940 (secure 0 0 0) 0x7f5a8805a280 con 0x7f5a9015a7b0 2024-04-20T10:22:30.838 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:22:30.838 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":27,"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","created":"2024-04-20T10:20:38.076076+0000","modified":"2024-04-20T10:21:44.715898+0000","last_up_change":"2024-04-20T10:20:50.162700+0000","last_in_change":"2024-04-20T10:20:41.510382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:20:51.141365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:21:00.436937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:21:01.026957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:21:04.862693+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"629a7b8c-a40a-4a2a-a3cc-d9f357774310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6809","nonce":1044588835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6811","nonce":1044588835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6815","nonce":1044588835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6813","nonce":1044588835}]},"public_addr":"172.21.15.121:6809/1044588835","cluster_addr":"172.21.15.121:6811/1044588835","heartbeat_back_addr":"172.21.15.121:6815/1044588835","heartbeat_front_addr":"172.21.15.121:6813/1044588835","state":["exists","up"]},{"osd":1,"uuid":"f2570399-aa50-4ff3-bf5d-3d09207290db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6801","nonce":3505199624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6803","nonce":3505199624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6807","nonce":3505199624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6805","nonce":3505199624}]},"public_addr":"172.21.15.121:6801/3505199624","cluster_addr":"172.21.15.121:6803/3505199624","heartbeat_back_addr":"172.21.15.121:6807/3505199624","heartbeat_front_addr":"172.21.15.121:6805/3505199624","state":["exists","up"]},{"osd":2,"uuid":"50893807-dc30-49f4-b41f-739366626658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6817","nonce":3656724207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6819","nonce":3656724207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6827","nonce":3656724207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6822","nonce":3656724207}]},"public_addr":"172.21.15.121:6817/3656724207","cluster_addr":"172.21.15.121:6819/3656724207","heartbeat_back_addr":"172.21.15.121:6827/3656724207","heartbeat_front_addr":"172.21.15.121:6822/3656724207","state":["exists","up"]},{"osd":3,"uuid":"dadb01bc-5491-4ffc-8824-d48a057240b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6823","nonce":2232342832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6826","nonce":2232342832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6831","nonce":2232342832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6829","nonce":2232342832}]},"public_addr":"172.21.15.121:6823/2232342832","cluster_addr":"172.21.15.121:6826/2232342832","heartbeat_back_addr":"172.21.15.121:6831/2232342832","heartbeat_front_addr":"172.21.15.121:6829/2232342832","state":["exists","up"]},{"osd":4,"uuid":"b3f3b4a1-b375-4526-b69e-344ab24c2da1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6801","nonce":99353111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6803","nonce":99353111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6807","nonce":99353111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6805","nonce":99353111}]},"public_addr":"172.21.15.123:6801/99353111","cluster_addr":"172.21.15.123:6803/99353111","heartbeat_back_addr":"172.21.15.123:6807/99353111","heartbeat_front_addr":"172.21.15.123:6805/99353111","state":["exists","up"]},{"osd":5,"uuid":"2ee24a08-1624-4961-9642-d7b510282484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6818","nonce":2504374688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6822","nonce":2504374688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6830","nonce":2504374688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6826","nonce":2504374688}]},"public_addr":"172.21.15.123:6818/2504374688","cluster_addr":"172.21.15.123:6822/2504374688","heartbeat_back_addr":"172.21.15.123:6830/2504374688","heartbeat_front_addr":"172.21.15.123:6826/2504374688","state":["exists","up"]},{"osd":6,"uuid":"186118cc-6128-49de-acca-55260945c1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6819","nonce":4139299599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6823","nonce":4139299599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6829","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6831","nonce":4139299599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6827","nonce":4139299599}]},"public_addr":"172.21.15.123:6819/4139299599","cluster_addr":"172.21.15.123:6823/4139299599","heartbeat_back_addr":"172.21.15.123:6831/4139299599","heartbeat_front_addr":"172.21.15.123:6827/4139299599","state":["exists","up"]},{"osd":7,"uuid":"a00aac6e-4bd9-415a-a93d-ae85210865ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6809","nonce":4043981069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6811","nonce":4043981069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6815","nonce":4043981069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6813","nonce":4043981069}]},"public_addr":"172.21.15.123:6809/4043981069","cluster_addr":"172.21.15.123:6811/4043981069","heartbeat_back_addr":"172.21.15.123:6815/4043981069","heartbeat_front_addr":"172.21.15.123:6813/4043981069","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.904540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.958585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.616367+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.885244+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.103127+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.167248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.884348+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.862356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.23","mappings":[{"from":3,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:22:30.840 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.839+0000 7f5a8f200640 1 -- 172.21.15.121:0/2094458836 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f5a7006c8f0 msgr2=0x7f5a7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:30.840 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.839+0000 7f5a8f200640 1 --2- 172.21.15.121:0/2094458836 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f5a7006c8f0 0x7f5a7006edb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5a80005880 tx=0x7f5a80007a20 comp rx=0 tx=0).stop 2024-04-20T10:22:30.840 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.839+0000 7f5a8f200640 1 -- 172.21.15.121:0/2094458836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a9015a7b0 msgr2=0x7f5a901f3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:30.840 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.839+0000 7f5a8f200640 1 --2- 172.21.15.121:0/2094458836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a9015a7b0 0x7f5a901f3700 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5a8800c780 tx=0x7f5a8800cc40 comp rx=0 tx=0).stop 2024-04-20T10:22:30.841 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.840+0000 7f5a8f200640 1 -- 172.21.15.121:0/2094458836 shutdown_connections 2024-04-20T10:22:30.841 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.840+0000 7f5a8f200640 1 --2- 172.21.15.121:0/2094458836 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5a9015a7b0 0x7f5a901f3700 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:30.841 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.840+0000 7f5a8f200640 1 --2- 172.21.15.121:0/2094458836 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5a90071c90 0x7f5a9009ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:30.841 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.840+0000 7f5a8f200640 1 --2- 172.21.15.121:0/2094458836 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f5a7006c8f0 0x7f5a7006edb0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:30.841 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.840+0000 7f5a8f200640 1 --2- 172.21.15.121:0/2094458836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a9006c960 0x7f5a90149970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:30.841 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.840+0000 7f5a8f200640 1 -- 172.21.15.121:0/2094458836 >> 172.21.15.121:0/2094458836 conn(0x7f5a900d0040 msgr2=0x7f5a90159e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:30.841 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.840+0000 7f5a8f200640 1 -- 172.21.15.121:0/2094458836 shutdown_connections 2024-04-20T10:22:30.841 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:30.840+0000 7f5a8f200640 1 -- 172.21.15.121:0/2094458836 wait complete. 2024-04-20T10:22:30.859 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:20:51.141365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:21:00.436937+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:21:01.026957+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T10:21:04.862693+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T10:22:30.859 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:22:31.061 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.060+0000 7fc5de000640 1 Processor -- start 2024-04-20T10:22:31.061 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.061+0000 7fc5de000640 1 -- start start 2024-04-20T10:22:31.062 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.062+0000 7fc5de000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc5d80d4270 0x7fc5d80d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:31.063 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.062+0000 7fc5de000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5d80d3820 con 0x7fc5d80d4650 2024-04-20T10:22:31.063 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.062+0000 7fc5de000640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5d80d3b00 con 0x7fc5d80d0380 2024-04-20T10:22:31.063 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.062+0000 7fc5de000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fc5d80d40c0 con 0x7fc5d80d4270 2024-04-20T10:22:31.063 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.062+0000 7fc5cfe00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fc5d80d4650 0x7fc5d80b1290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:56194/0 (socket says 172.21.15.121:56194) 2024-04-20T10:22:31.063 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.062+0000 7fc5cfe00640 1 -- 172.21.15.121:0/3744359395 learned_addr learned my addr 172.21.15.121:0/3744359395 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:31.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.063+0000 7fc5d6a00640 1 -- 172.21.15.121:0/3744359395 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4258097904 0 0) 0x7fc5d80d3820 con 0x7fc5d80d4650 2024-04-20T10:22:31.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.063+0000 7fc5d6a00640 1 -- 172.21.15.121:0/3744359395 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc5bc003520 con 0x7fc5d80d4650 2024-04-20T10:22:31.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.063+0000 7fc5d7400640 1 --2- 172.21.15.121:0/3744359395 >> v2:172.21.15.123:3300/0 conn(0x7fc5d80d4270 0x7fc5d80d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:31.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.063+0000 7fc5d6a00640 1 -- 172.21.15.121:0/3744359395 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3419504609 0 0) 0x7fc5bc003520 con 0x7fc5d80d4650 2024-04-20T10:22:31.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.063+0000 7fc5d6a00640 1 -- 172.21.15.121:0/3744359395 >> v2:172.21.15.123:3300/0 conn(0x7fc5d80d4270 msgr2=0x7fc5d80d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:31.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.063+0000 7fc5d6a00640 1 --2- 172.21.15.121:0/3744359395 >> v2:172.21.15.123:3300/0 conn(0x7fc5d80d4270 0x7fc5d80d1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:31.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.063+0000 7fc5d6a00640 1 -- 172.21.15.121:0/3744359395 >> v1:172.21.15.123:6789/0 conn(0x7fc5d80d0380 legacy=0x7fc5d80d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:31.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.063+0000 7fc5d6a00640 1 -- 172.21.15.121:0/3744359395 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5d80d3de0 con 0x7fc5d80d4650 2024-04-20T10:22:31.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.063+0000 7fc5d6a00640 1 -- 172.21.15.121:0/3744359395 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7fc5c0001670 con 0x7fc5d80d4650 2024-04-20T10:22:31.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.063+0000 7fc5d6a00640 1 -- 172.21.15.121:0/3744359395 >> v1:172.21.15.121:6789/0 conn(0x7fc5d80d4650 legacy=0x7fc5d80b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:31.065 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.063+0000 7fc5d6a00640 1 --2- 172.21.15.121:0/3744359395 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5bc0039b0 0x7fc5bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:31.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.063+0000 7fc5d6a00640 1 --2- 172.21.15.121:0/3744359395 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc5bc004800 0x7fc5bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:31.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.063+0000 7fc5d6a00640 1 --2- 172.21.15.121:0/3744359395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5bc0051c0 0x7fc5bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:31.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.063+0000 7fc5d6a00640 1 -- 172.21.15.121:0/3744359395 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d80d40c0 con 0x7fc5bc0039b0 2024-04-20T10:22:31.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.063+0000 7fc5d6a00640 1 -- 172.21.15.121:0/3744359395 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc5bc00c110 con 0x7fc5bc0051c0 2024-04-20T10:22:31.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.063+0000 7fc5d6a00640 1 -- 172.21.15.121:0/3744359395 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fc5bc00c310 con 0x7fc5bc004800 2024-04-20T10:22:31.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.064+0000 7fc5d7400640 1 --2- 172.21.15.121:0/3744359395 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5bc0039b0 0x7fc5bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:31.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.064+0000 7fc5d7400640 1 -- 172.21.15.121:0/3744359395 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc5bc004800 msgr2=0x7fc5bc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:31.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.064+0000 7fc5d7400640 1 --2- 172.21.15.121:0/3744359395 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc5bc004800 0x7fc5bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:31.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.064+0000 7fc5d7400640 1 -- 172.21.15.121:0/3744359395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5bc0051c0 msgr2=0x7fc5bc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:31.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.064+0000 7fc5d7e00640 1 --2- 172.21.15.121:0/3744359395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5bc0051c0 0x7fc5bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:31.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.064+0000 7fc5d7400640 1 --2- 172.21.15.121:0/3744359395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5bc0051c0 0x7fc5bc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:31.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.064+0000 7fc5d7400640 1 -- 172.21.15.121:0/3744359395 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5bc003520 con 0x7fc5bc0039b0 2024-04-20T10:22:31.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.064+0000 7fc5d7e00640 1 --2- 172.21.15.121:0/3744359395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5bc0051c0 0x7fc5bc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:31.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.064+0000 7fc5d7e00640 1 -- 172.21.15.121:0/3744359395 reap_dead start 2024-04-20T10:22:31.068 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.065+0000 7fc5d7400640 1 --2- 172.21.15.121:0/3744359395 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5bc0039b0 0x7fc5bc003dd0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800ba30 tx=0x7fc5bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7297aecb3e4c671d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:31.068 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.065+0000 7fc5d6a00640 1 -- 172.21.15.121:0/3744359395 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8036020 con 0x7fc5bc0039b0 2024-04-20T10:22:31.068 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.065+0000 7fc5d6a00640 1 -- 172.21.15.121:0/3744359395 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5c8032640 con 0x7fc5bc0039b0 2024-04-20T10:22:31.068 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.065+0000 7fc5d6a00640 1 -- 172.21.15.121:0/3744359395 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c800b600 con 0x7fc5bc0039b0 2024-04-20T10:22:31.068 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.066+0000 7fc5de000640 1 -- 172.21.15.121:0/3744359395 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5bc0039b0 msgr2=0x7fc5bc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:31.068 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.066+0000 7fc5de000640 1 --2- 172.21.15.121:0/3744359395 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5bc0039b0 0x7fc5bc003dd0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800ba30 tx=0x7fc5bc005830 comp rx=0 tx=0).stop 2024-04-20T10:22:31.068 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.066+0000 7fc5de000640 1 -- 172.21.15.121:0/3744359395 shutdown_connections 2024-04-20T10:22:31.068 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.066+0000 7fc5de000640 1 --2- 172.21.15.121:0/3744359395 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5bc0039b0 0x7fc5bc003dd0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:31.068 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.066+0000 7fc5de000640 1 -- 172.21.15.121:0/3744359395 >> 172.21.15.121:0/3744359395 conn(0x7fc5d80b99c0 msgr2=0x7fc5d80b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:31.068 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.066+0000 7fc5de000640 1 -- 172.21.15.121:0/3744359395 shutdown_connections 2024-04-20T10:22:31.068 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.066+0000 7fc5de000640 1 -- 172.21.15.121:0/3744359395 wait complete. 2024-04-20T10:22:31.069 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.068+0000 7fc5de000640 1 Processor -- start 2024-04-20T10:22:31.070 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.068+0000 7fc5de000640 1 -- start start 2024-04-20T10:22:31.070 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.069+0000 7fc5de000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5bc0039b0 0x7fc5d80cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:31.070 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.069+0000 7fc5de000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc5d81f7960 0x7fc5d80c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:31.070 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.069+0000 7fc5de000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5d815ba60 0x7fc5d80b5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:31.070 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.069+0000 7fc5de000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d80d4ae0 con 0x7fc5bc0039b0 2024-04-20T10:22:31.070 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.069+0000 7fc5de000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d80d53b0 con 0x7fc5d815ba60 2024-04-20T10:22:31.070 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.069+0000 7fc5de000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fc5d80d4c60 con 0x7fc5d81f7960 2024-04-20T10:22:31.070 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.069+0000 7fc5d7400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5bc0039b0 0x7fc5d80cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:31.070 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.069+0000 7fc5d7400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5bc0039b0 0x7fc5d80cb6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59132/0 (socket says 172.21.15.121:59132) 2024-04-20T10:22:31.070 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.069+0000 7fc5d7400640 1 -- 172.21.15.121:0/60413246 learned_addr learned my addr 172.21.15.121:0/60413246 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:31.071 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.070+0000 7fc5cfe00640 1 --2- 172.21.15.121:0/60413246 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc5d81f7960 0x7fc5d80c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:31.072 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.070+0000 7fc5d7400640 1 -- 172.21.15.121:0/60413246 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc5d81f7960 msgr2=0x7fc5d80c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:31.072 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.070+0000 7fc5d7e00640 1 --2- 172.21.15.121:0/60413246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5d815ba60 0x7fc5d80b5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:31.072 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.070+0000 7fc5d7400640 1 --2- 172.21.15.121:0/60413246 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc5d81f7960 0x7fc5d80c94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:31.072 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.070+0000 7fc5d7400640 1 -- 172.21.15.121:0/60413246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5d815ba60 msgr2=0x7fc5d80b5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:31.072 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.070+0000 7fc5d7400640 1 --2- 172.21.15.121:0/60413246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5d815ba60 0x7fc5d80b5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:31.072 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.070+0000 7fc5d7400640 1 -- 172.21.15.121:0/60413246 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5d80cbbf0 con 0x7fc5bc0039b0 2024-04-20T10:22:31.072 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.070+0000 7fc5d7400640 1 --2- 172.21.15.121:0/60413246 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5bc0039b0 0x7fc5d80cb6b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc5c802bf90 tx=0x7fc5c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:31.072 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.070+0000 7fc5cf400640 1 -- 172.21.15.121:0/60413246 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8036040 con 0x7fc5bc0039b0 2024-04-20T10:22:31.072 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.070+0000 7fc5cf400640 1 -- 172.21.15.121:0/60413246 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5c8030a50 con 0x7fc5bc0039b0 2024-04-20T10:22:31.072 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.070+0000 7fc5de000640 1 -- 172.21.15.121:0/60413246 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5d80c9180 con 0x7fc5bc0039b0 2024-04-20T10:22:31.072 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.070+0000 7fc5cf400640 1 -- 172.21.15.121:0/60413246 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c800b600 con 0x7fc5bc0039b0 2024-04-20T10:22:31.072 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.071+0000 7fc5de000640 1 -- 172.21.15.121:0/60413246 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5d81f80d0 con 0x7fc5bc0039b0 2024-04-20T10:22:31.073 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.072+0000 7fc5de000640 1 -- 172.21.15.121:0/60413246 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5d8068980 con 0x7fc5bc0039b0 2024-04-20T10:22:31.073 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.073+0000 7fc5cf400640 1 -- 172.21.15.121:0/60413246 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc5c8039020 con 0x7fc5bc0039b0 2024-04-20T10:22:31.074 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.073+0000 7fc5cf400640 1 --2- 172.21.15.121:0/60413246 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fc5b406c8f0 0x7fc5b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:31.074 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.074+0000 7fc5cf400640 1 -- 172.21.15.121:0/60413246 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc5c80ae940 con 0x7fc5bc0039b0 2024-04-20T10:22:31.074 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.074+0000 7fc5cfe00640 1 --2- 172.21.15.121:0/60413246 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fc5b406c8f0 0x7fc5b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:31.075 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.074+0000 7fc5cfe00640 1 --2- 172.21.15.121:0/60413246 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fc5b406c8f0 0x7fc5b406edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc5d80ce040 tx=0x7fc5c0009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:31.078 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.078+0000 7fc5cf400640 1 -- 172.21.15.121:0/60413246 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc5c807c9e0 con 0x7fc5bc0039b0 2024-04-20T10:22:31.269 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.268+0000 7fc5de000640 1 -- 172.21.15.121:0/60413246 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc5d80c9a00 con 0x7fc5bc0039b0 2024-04-20T10:22:31.269 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.269+0000 7fc5cf400640 1 -- 172.21.15.121:0/60413246 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7fc5c807c2e0 con 0x7fc5bc0039b0 2024-04-20T10:22:31.269 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2024-04-20T10:22:31.272 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.272+0000 7fc5de000640 1 -- 172.21.15.121:0/60413246 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fc5b406c8f0 msgr2=0x7fc5b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:31.272 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.272+0000 7fc5de000640 1 --2- 172.21.15.121:0/60413246 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fc5b406c8f0 0x7fc5b406edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc5d80ce040 tx=0x7fc5c0009040 comp rx=0 tx=0).stop 2024-04-20T10:22:31.272 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.272+0000 7fc5de000640 1 -- 172.21.15.121:0/60413246 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5bc0039b0 msgr2=0x7fc5d80cb6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:31.273 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.272+0000 7fc5de000640 1 --2- 172.21.15.121:0/60413246 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5bc0039b0 0x7fc5d80cb6b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc5c802bf90 tx=0x7fc5c8004730 comp rx=0 tx=0).stop 2024-04-20T10:22:31.273 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.272+0000 7fc5de000640 1 -- 172.21.15.121:0/60413246 shutdown_connections 2024-04-20T10:22:31.273 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.272+0000 7fc5de000640 1 --2- 172.21.15.121:0/60413246 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fc5b406c8f0 0x7fc5b406edb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:31.273 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.272+0000 7fc5de000640 1 --2- 172.21.15.121:0/60413246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5d815ba60 0x7fc5d80b5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:31.273 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.272+0000 7fc5de000640 1 --2- 172.21.15.121:0/60413246 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc5d81f7960 0x7fc5d80c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:31.273 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.272+0000 7fc5de000640 1 --2- 172.21.15.121:0/60413246 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc5bc0039b0 0x7fc5d80cb6b0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:31.273 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.272+0000 7fc5de000640 1 -- 172.21.15.121:0/60413246 >> 172.21.15.121:0/60413246 conn(0x7fc5d80b99c0 msgr2=0x7fc5d80b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:31.273 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.272+0000 7fc5de000640 1 -- 172.21.15.121:0/60413246 shutdown_connections 2024-04-20T10:22:31.273 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.272+0000 7fc5de000640 1 -- 172.21.15.121:0/60413246 wait complete. 2024-04-20T10:22:31.290 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:22:31.486 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.485+0000 7fbda2c00640 1 Processor -- start 2024-04-20T10:22:31.486 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.485+0000 7fbda2c00640 1 -- start start 2024-04-20T10:22:31.488 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.487+0000 7fbda2c00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd9c0d0a70 0x7fbd9c06e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:31.488 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.487+0000 7fbda2c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fbd9c06dbb0 0x7fbd9c0ad680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:31.488 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.487+0000 7fbda2c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd9c06f060 con 0x7fbd9c15e8a0 2024-04-20T10:22:31.488 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.487+0000 7fbda2c00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fbd9c06e7c0 con 0x7fbd9c06dbb0 2024-04-20T10:22:31.488 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.487+0000 7fbda2c00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fbd9c06eaa0 con 0x7fbd9c0d0a70 2024-04-20T10:22:31.488 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.487+0000 7fbda0c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fbd9c06dbb0 0x7fbd9c0ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:31.488 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.487+0000 7fbda0c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fbd9c06dbb0 0x7fbd9c0ad680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59142/0 (socket says 172.21.15.121:59142) 2024-04-20T10:22:31.488 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.487+0000 7fbda0c00640 1 -- 172.21.15.121:0/2926709930 learned_addr learned my addr 172.21.15.121:0/2926709930 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:31.489 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.488+0000 7fbd93400640 1 --2- 172.21.15.121:0/2926709930 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd9c0d0a70 0x7fbd9c06e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:31.489 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.488+0000 7fbda0c00640 1 -- 172.21.15.121:0/2926709930 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd9c0d0a70 msgr2=0x7fbd9c06e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:31.489 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.488+0000 7fbda0c00640 1 --2- 172.21.15.121:0/2926709930 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd9c0d0a70 0x7fbd9c06e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:31.489 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.488+0000 7fbda0c00640 1 -- 172.21.15.121:0/2926709930 >> v1:172.21.15.123:6789/0 conn(0x7fbd9c15e8a0 legacy=0x7fbd9c0d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:31.489 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.488+0000 7fbda0c00640 1 -- 172.21.15.121:0/2926709930 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd9c06ed80 con 0x7fbd9c06dbb0 2024-04-20T10:22:31.490 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.489+0000 7fbda0c00640 1 --2- 172.21.15.121:0/2926709930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd9c06dbb0 0x7fbd9c0ad680 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbd94009e50 tx=0x7fbd9402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27bb85ec7216e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:31.490 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.489+0000 7fbd9b400640 1 -- 172.21.15.121:0/2926709930 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9402ca70 con 0x7fbd9c06dbb0 2024-04-20T10:22:31.490 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.489+0000 7fbd9b400640 1 -- 172.21.15.121:0/2926709930 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd9402cc10 con 0x7fbd9c06dbb0 2024-04-20T10:22:31.490 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.489+0000 7fbd9b400640 1 -- 172.21.15.121:0/2926709930 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd940343f0 con 0x7fbd9c06dbb0 2024-04-20T10:22:31.491 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.490+0000 7fbda2c00640 1 -- 172.21.15.121:0/2926709930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd9c06dbb0 msgr2=0x7fbd9c0ad680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:31.491 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.490+0000 7fbda2c00640 1 --2- 172.21.15.121:0/2926709930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd9c06dbb0 0x7fbd9c0ad680 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbd94009e50 tx=0x7fbd9402bed0 comp rx=0 tx=0).stop 2024-04-20T10:22:31.491 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.490+0000 7fbda2c00640 1 -- 172.21.15.121:0/2926709930 shutdown_connections 2024-04-20T10:22:31.491 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.490+0000 7fbda2c00640 1 --2- 172.21.15.121:0/2926709930 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd9c06dbb0 0x7fbd9c0ad680 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:31.491 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.490+0000 7fbda2c00640 1 --2- 172.21.15.121:0/2926709930 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd9c0d0a70 0x7fbd9c06e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:31.491 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.490+0000 7fbda2c00640 1 -- 172.21.15.121:0/2926709930 >> 172.21.15.121:0/2926709930 conn(0x7fbd9c0b3350 msgr2=0x7fbd9c0b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:31.491 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.490+0000 7fbda2c00640 1 -- 172.21.15.121:0/2926709930 shutdown_connections 2024-04-20T10:22:31.492 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.490+0000 7fbda2c00640 1 -- 172.21.15.121:0/2926709930 wait complete. 2024-04-20T10:22:31.492 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.492+0000 7fbda2c00640 1 Processor -- start 2024-04-20T10:22:31.493 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.492+0000 7fbda2c00640 1 -- start start 2024-04-20T10:22:31.494 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.493+0000 7fbda2c00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd9c06dbb0 0x7fbd9c099a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:31.495 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.493+0000 7fbda2c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd9c0d0a70 0x7fbd9c1549c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:31.495 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.493+0000 7fbda2c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd9c15e8a0 0x7fbd9c154f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:31.495 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.493+0000 7fbda2c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c06fef0 con 0x7fbd9c15e8a0 2024-04-20T10:22:31.495 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.493+0000 7fbda2c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c06fd70 con 0x7fbd9c0d0a70 2024-04-20T10:22:31.495 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.493+0000 7fbda2c00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fbd9c0adbc0 con 0x7fbd9c06dbb0 2024-04-20T10:22:31.495 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.494+0000 7fbda0c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd9c15e8a0 0x7fbd9c154f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:31.495 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.494+0000 7fbda0c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd9c15e8a0 0x7fbd9c154f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59158/0 (socket says 172.21.15.121:59158) 2024-04-20T10:22:31.495 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.494+0000 7fbda0c00640 1 -- 172.21.15.121:0/2829083365 learned_addr learned my addr 172.21.15.121:0/2829083365 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:31.495 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.494+0000 7fbd9be00640 1 --2- 172.21.15.121:0/2829083365 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd9c06dbb0 0x7fbd9c099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:31.495 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.494+0000 7fbd93400640 1 --2- 172.21.15.121:0/2829083365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd9c0d0a70 0x7fbd9c1549c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:31.495 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.494+0000 7fbda0c00640 1 -- 172.21.15.121:0/2829083365 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd9c06dbb0 msgr2=0x7fbd9c099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:31.495 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.494+0000 7fbda0c00640 1 --2- 172.21.15.121:0/2829083365 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd9c06dbb0 0x7fbd9c099a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:31.495 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.494+0000 7fbda0c00640 1 -- 172.21.15.121:0/2829083365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd9c0d0a70 msgr2=0x7fbd9c1549c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:31.495 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.494+0000 7fbda0c00640 1 --2- 172.21.15.121:0/2829083365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd9c0d0a70 0x7fbd9c1549c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:31.496 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.494+0000 7fbda0c00640 1 -- 172.21.15.121:0/2829083365 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd9c0b1130 con 0x7fbd9c15e8a0 2024-04-20T10:22:31.496 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.495+0000 7fbda0c00640 1 --2- 172.21.15.121:0/2829083365 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd9c15e8a0 0x7fbd9c154f00 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbd94009e50 tx=0x7fbd94004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:31.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.495+0000 7fbd98c00640 1 -- 172.21.15.121:0/2829083365 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94043020 con 0x7fbd9c15e8a0 2024-04-20T10:22:31.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.495+0000 7fbd98c00640 1 -- 172.21.15.121:0/2829083365 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd9403e450 con 0x7fbd9c15e8a0 2024-04-20T10:22:31.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.495+0000 7fbd98c00640 1 -- 172.21.15.121:0/2829083365 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94009510 con 0x7fbd9c15e8a0 2024-04-20T10:22:31.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.495+0000 7fbda2c00640 1 -- 172.21.15.121:0/2829083365 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd9c0b1410 con 0x7fbd9c15e8a0 2024-04-20T10:22:31.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.495+0000 7fbda2c00640 1 -- 172.21.15.121:0/2829083365 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd9c0b03c0 con 0x7fbd9c15e8a0 2024-04-20T10:22:31.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.496+0000 7fbda2c00640 1 -- 172.21.15.121:0/2829083365 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd9c06e160 con 0x7fbd9c15e8a0 2024-04-20T10:22:31.497 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.497+0000 7fbd98c00640 1 -- 172.21.15.121:0/2829083365 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbd94046020 con 0x7fbd9c15e8a0 2024-04-20T10:22:31.498 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.498+0000 7fbd98c00640 1 --2- 172.21.15.121:0/2829083365 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fbd7806c920 0x7fbd7806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:31.499 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.498+0000 7fbd98c00640 1 -- 172.21.15.121:0/2829083365 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fbd940ae870 con 0x7fbd9c15e8a0 2024-04-20T10:22:31.499 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.498+0000 7fbd9be00640 1 --2- 172.21.15.121:0/2829083365 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fbd7806c920 0x7fbd7806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:31.500 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.499+0000 7fbd9be00640 1 --2- 172.21.15.121:0/2829083365 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fbd7806c920 0x7fbd7806ede0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c006fd0 tx=0x7fbd8c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:31.503 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.503+0000 7fbd98c00640 1 -- 172.21.15.121:0/2829083365 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbd9404d030 con 0x7fbd9c15e8a0 2024-04-20T10:22:31.701 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.701+0000 7fbda2c00640 1 -- 172.21.15.121:0/2829083365 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbd9c068980 con 0x7fbd9c15e8a0 2024-04-20T10:22:31.702 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.702+0000 7fbd98c00640 1 -- 172.21.15.121:0/2829083365 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fbd9407c800 con 0x7fbd9c15e8a0 2024-04-20T10:22:31.703 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 32 2024-04-20T10:22:31.705 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.704+0000 7fbda2c00640 1 -- 172.21.15.121:0/2829083365 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fbd7806c920 msgr2=0x7fbd7806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:31.705 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.704+0000 7fbda2c00640 1 --2- 172.21.15.121:0/2829083365 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fbd7806c920 0x7fbd7806ede0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c006fd0 tx=0x7fbd8c008040 comp rx=0 tx=0).stop 2024-04-20T10:22:31.705 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.704+0000 7fbda2c00640 1 -- 172.21.15.121:0/2829083365 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd9c15e8a0 msgr2=0x7fbd9c154f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:31.705 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.704+0000 7fbda2c00640 1 --2- 172.21.15.121:0/2829083365 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd9c15e8a0 0x7fbd9c154f00 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbd94009e50 tx=0x7fbd94004a40 comp rx=0 tx=0).stop 2024-04-20T10:22:31.706 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.705+0000 7fbda2c00640 1 -- 172.21.15.121:0/2829083365 shutdown_connections 2024-04-20T10:22:31.706 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.705+0000 7fbda2c00640 1 --2- 172.21.15.121:0/2829083365 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbd9c15e8a0 0x7fbd9c154f00 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:31.706 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.705+0000 7fbda2c00640 1 --2- 172.21.15.121:0/2829083365 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fbd7806c920 0x7fbd7806ede0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:31.706 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.705+0000 7fbda2c00640 1 --2- 172.21.15.121:0/2829083365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbd9c0d0a70 0x7fbd9c1549c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:31.706 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.705+0000 7fbda2c00640 1 --2- 172.21.15.121:0/2829083365 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbd9c06dbb0 0x7fbd9c099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:31.706 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.705+0000 7fbda2c00640 1 -- 172.21.15.121:0/2829083365 >> 172.21.15.121:0/2829083365 conn(0x7fbd9c0b3350 msgr2=0x7fbd9c15c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:31.706 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.705+0000 7fbda2c00640 1 -- 172.21.15.121:0/2829083365 shutdown_connections 2024-04-20T10:22:31.706 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.705+0000 7fbda2c00640 1 -- 172.21.15.121:0/2829083365 wait complete. 2024-04-20T10:22:31.723 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:22:31.930 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.929+0000 7f1e76200640 1 Processor -- start 2024-04-20T10:22:31.930 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.929+0000 7f1e76200640 1 -- start start 2024-04-20T10:22:31.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.931+0000 7f1e76200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1e7015e770 0x7f1e700b3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:31.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.931+0000 7f1e76200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1e70162210 0x7f1e70162670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:31.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.931+0000 7f1e76200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1e700b6400 0x7f1e700b6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:31.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.931+0000 7f1e76200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f1e7015eb50 con 0x7f1e70162210 2024-04-20T10:22:31.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.931+0000 7f1e76200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f1e700d3590 con 0x7f1e7015e770 2024-04-20T10:22:31.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.931+0000 7f1e76200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f1e700d3870 con 0x7f1e700b6400 2024-04-20T10:22:31.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.931+0000 7f1e6fe00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1e70162210 0x7f1e70162670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:31.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.931+0000 7f1e6fe00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1e70162210 0x7f1e70162670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59174/0 (socket says 172.21.15.121:59174) 2024-04-20T10:22:31.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.931+0000 7f1e6fe00640 1 -- 172.21.15.121:0/3318584 learned_addr learned my addr 172.21.15.121:0/3318584 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:31.933 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.932+0000 7f1e6fe00640 1 -- 172.21.15.121:0/3318584 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1e700b6400 msgr2=0x7f1e700b6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:31.933 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.932+0000 7f1e6fe00640 1 --2- 172.21.15.121:0/3318584 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1e700b6400 0x7f1e700b6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:31.933 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.932+0000 7f1e74e00640 1 --2- 172.21.15.121:0/3318584 >> v2:172.21.15.123:3300/0 conn(0x7f1e7015e770 0x7f1e700b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:31.933 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.932+0000 7f1e6fe00640 1 -- 172.21.15.121:0/3318584 >> v2:172.21.15.123:3300/0 conn(0x7f1e7015e770 msgr2=0x7f1e700b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:31.933 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.932+0000 7f1e6fe00640 1 --2- 172.21.15.121:0/3318584 >> v2:172.21.15.123:3300/0 conn(0x7f1e7015e770 0x7f1e700b3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:31.933 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.932+0000 7f1e6fe00640 1 -- 172.21.15.121:0/3318584 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e700d3b50 con 0x7f1e70162210 2024-04-20T10:22:31.933 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.932+0000 7f1e74e00640 1 --2- 172.21.15.121:0/3318584 >> v2:172.21.15.123:3300/0 conn(0x7f1e7015e770 0x7f1e700b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:31.934 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.933+0000 7f1e6fe00640 1 --2- 172.21.15.121:0/3318584 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e70162210 0x7f1e70162670 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1e7008e630 tx=0x7f1e5c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b723c3763ecbd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:31.935 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.933+0000 7f1e6f400640 1 -- 172.21.15.121:0/3318584 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c02e070 con 0x7f1e70162210 2024-04-20T10:22:31.935 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.933+0000 7f1e6f400640 1 -- 172.21.15.121:0/3318584 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e5c037070 con 0x7f1e70162210 2024-04-20T10:22:31.935 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.933+0000 7f1e6f400640 1 -- 172.21.15.121:0/3318584 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c031a00 con 0x7f1e70162210 2024-04-20T10:22:31.935 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.934+0000 7f1e76200640 1 -- 172.21.15.121:0/3318584 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e70162210 msgr2=0x7f1e70162670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:31.935 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.934+0000 7f1e76200640 1 --2- 172.21.15.121:0/3318584 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e70162210 0x7f1e70162670 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1e7008e630 tx=0x7f1e5c02b880 comp rx=0 tx=0).stop 2024-04-20T10:22:31.935 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.934+0000 7f1e76200640 1 -- 172.21.15.121:0/3318584 shutdown_connections 2024-04-20T10:22:31.935 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.934+0000 7f1e76200640 1 --2- 172.21.15.121:0/3318584 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1e700b6400 0x7f1e700b6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:31.935 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.934+0000 7f1e76200640 1 --2- 172.21.15.121:0/3318584 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e70162210 0x7f1e70162670 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:31.935 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.934+0000 7f1e76200640 1 --2- 172.21.15.121:0/3318584 >> v2:172.21.15.123:3300/0 conn(0x7f1e7015e770 0x7f1e700b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:31.935 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.934+0000 7f1e76200640 1 -- 172.21.15.121:0/3318584 >> 172.21.15.121:0/3318584 conn(0x7f1e7009d6d0 msgr2=0x7f1e700b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:31.935 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.934+0000 7f1e76200640 1 -- 172.21.15.121:0/3318584 shutdown_connections 2024-04-20T10:22:31.935 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.934+0000 7f1e76200640 1 -- 172.21.15.121:0/3318584 wait complete. 2024-04-20T10:22:31.936 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.936+0000 7f1e76200640 1 Processor -- start 2024-04-20T10:22:31.936 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.936+0000 7f1e76200640 1 -- start start 2024-04-20T10:22:31.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.937+0000 7f1e76200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1e700b6400 0x7f1e7015a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:31.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.937+0000 7f1e76200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e7015e770 0x7f1e70160030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:31.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.937+0000 7f1e76200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e70162210 0x7f1e700d25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:31.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.937+0000 7f1e76200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1e700b39d0 con 0x7f1e70162210 2024-04-20T10:22:31.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.937+0000 7f1e76200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1e700d3e30 con 0x7f1e7015e770 2024-04-20T10:22:31.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.937+0000 7f1e76200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f1e700b3b50 con 0x7f1e700b6400 2024-04-20T10:22:31.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.937+0000 7f1e75800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e70162210 0x7f1e700d25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:31.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.937+0000 7f1e75800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e70162210 0x7f1e700d25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59178/0 (socket says 172.21.15.121:59178) 2024-04-20T10:22:31.938 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.937+0000 7f1e75800640 1 -- 172.21.15.121:0/427176051 learned_addr learned my addr 172.21.15.121:0/427176051 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:31.939 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.938+0000 7f1e74e00640 1 --2- 172.21.15.121:0/427176051 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1e700b6400 0x7f1e7015a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:31.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.938+0000 7f1e6fe00640 1 --2- 172.21.15.121:0/427176051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e7015e770 0x7f1e70160030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:31.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.938+0000 7f1e75800640 1 -- 172.21.15.121:0/427176051 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1e700b6400 msgr2=0x7f1e7015a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:31.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.938+0000 7f1e75800640 1 --2- 172.21.15.121:0/427176051 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1e700b6400 0x7f1e7015a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:31.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.938+0000 7f1e75800640 1 -- 172.21.15.121:0/427176051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e7015e770 msgr2=0x7f1e70160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:31.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.938+0000 7f1e75800640 1 --2- 172.21.15.121:0/427176051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e7015e770 0x7f1e70160030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:31.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.938+0000 7f1e75800640 1 -- 172.21.15.121:0/427176051 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e7006e750 con 0x7f1e70162210 2024-04-20T10:22:31.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.938+0000 7f1e6fe00640 1 --2- 172.21.15.121:0/427176051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e7015e770 0x7f1e70160030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:31.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.938+0000 7f1e74e00640 1 --2- 172.21.15.121:0/427176051 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1e700b6400 0x7f1e7015a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:31.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.938+0000 7f1e75800640 1 --2- 172.21.15.121:0/427176051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e70162210 0x7f1e700d25c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1e6800b970 tx=0x7f1e6800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:31.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.938+0000 7f1e6cc00640 1 -- 172.21.15.121:0/427176051 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e68019070 con 0x7f1e70162210 2024-04-20T10:22:31.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.938+0000 7f1e6cc00640 1 -- 172.21.15.121:0/427176051 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e6800cb60 con 0x7f1e70162210 2024-04-20T10:22:31.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.938+0000 7f1e6cc00640 1 -- 172.21.15.121:0/427176051 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e68014440 con 0x7f1e70162210 2024-04-20T10:22:31.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.938+0000 7f1e76200640 1 -- 172.21.15.121:0/427176051 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e7006ea30 con 0x7f1e70162210 2024-04-20T10:22:31.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.939+0000 7f1e76200640 1 -- 172.21.15.121:0/427176051 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e7006bbe0 con 0x7f1e70162210 2024-04-20T10:22:31.941 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.940+0000 7f1e76200640 1 -- 172.21.15.121:0/427176051 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e70075b30 con 0x7f1e70162210 2024-04-20T10:22:31.941 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.941+0000 7f1e6cc00640 1 -- 172.21.15.121:0/427176051 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1e6801c070 con 0x7f1e70162210 2024-04-20T10:22:31.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.942+0000 7f1e6cc00640 1 --2- 172.21.15.121:0/427176051 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f1e5006c8f0 0x7f1e5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:31.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.942+0000 7f1e6cc00640 1 -- 172.21.15.121:0/427176051 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f1e6808bce0 con 0x7f1e70162210 2024-04-20T10:22:31.942 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.942+0000 7f1e74e00640 1 --2- 172.21.15.121:0/427176051 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f1e5006c8f0 0x7f1e5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:31.943 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.943+0000 7f1e74e00640 1 --2- 172.21.15.121:0/427176051 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f1e5006c8f0 0x7f1e5006edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1e60006fd0 tx=0x7f1e60008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:31.946 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:31.946+0000 7f1e6cc00640 1 -- 172.21.15.121:0/427176051 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1e68059cf0 con 0x7f1e70162210 2024-04-20T10:22:32.144 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.143+0000 7f1e76200640 1 -- 172.21.15.121:0/427176051 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1e700cb170 con 0x7f1e70162210 2024-04-20T10:22:32.144 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.144+0000 7f1e6cc00640 1 -- 172.21.15.121:0/427176051 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f1e680595f0 con 0x7f1e70162210 2024-04-20T10:22:32.145 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2024-04-20T10:22:32.147 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.147+0000 7f1e76200640 1 -- 172.21.15.121:0/427176051 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f1e5006c8f0 msgr2=0x7f1e5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:32.147 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.147+0000 7f1e76200640 1 --2- 172.21.15.121:0/427176051 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f1e5006c8f0 0x7f1e5006edb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1e60006fd0 tx=0x7f1e60008040 comp rx=0 tx=0).stop 2024-04-20T10:22:32.148 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.147+0000 7f1e76200640 1 -- 172.21.15.121:0/427176051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e70162210 msgr2=0x7f1e700d25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:32.148 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.147+0000 7f1e76200640 1 --2- 172.21.15.121:0/427176051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e70162210 0x7f1e700d25c0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1e6800b970 tx=0x7f1e6800be30 comp rx=0 tx=0).stop 2024-04-20T10:22:32.148 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.147+0000 7f1e76200640 1 -- 172.21.15.121:0/427176051 shutdown_connections 2024-04-20T10:22:32.148 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.147+0000 7f1e76200640 1 --2- 172.21.15.121:0/427176051 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1e70162210 0x7f1e700d25c0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:32.148 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.147+0000 7f1e76200640 1 --2- 172.21.15.121:0/427176051 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f1e5006c8f0 0x7f1e5006edb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:32.148 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.147+0000 7f1e76200640 1 --2- 172.21.15.121:0/427176051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e7015e770 0x7f1e70160030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:32.148 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.147+0000 7f1e76200640 1 --2- 172.21.15.121:0/427176051 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1e700b6400 0x7f1e7015a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:32.148 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.147+0000 7f1e76200640 1 -- 172.21.15.121:0/427176051 >> 172.21.15.121:0/427176051 conn(0x7f1e7009d6d0 msgr2=0x7f1e700b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:32.148 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.147+0000 7f1e76200640 1 -- 172.21.15.121:0/427176051 shutdown_connections 2024-04-20T10:22:32.148 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.147+0000 7f1e76200640 1 -- 172.21.15.121:0/427176051 wait complete. 2024-04-20T10:22:32.165 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T10:22:32.371 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.370+0000 7f40fbe00640 1 Processor -- start 2024-04-20T10:22:32.371 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.371+0000 7f40fbe00640 1 -- start start 2024-04-20T10:22:32.373 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.372+0000 7f40fbe00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f40f4156c20 0x7f40f406b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:32.373 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.372+0000 7f40fbe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f40f40d0360 0x7f40f40d07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:32.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.372+0000 7f40fbe00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40f4141c60 con 0x7f40f415ca40 2024-04-20T10:22:32.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.372+0000 7f40fbe00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f40f40d0d10 con 0x7f40f4156c20 2024-04-20T10:22:32.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.372+0000 7f40fbe00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f40f40adab0 con 0x7f40f40d0360 2024-04-20T10:22:32.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.373+0000 7f40f9400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f40f4156c20 0x7f40f406b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:32.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.373+0000 7f40f8a00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f40f415ca40 0x7f40f40ad3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:56202/0 (socket says 172.21.15.121:56202) 2024-04-20T10:22:32.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.373+0000 7f40f8a00640 1 -- 172.21.15.121:0/14177966 learned_addr learned my addr 172.21.15.121:0/14177966 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:32.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.373+0000 7f40f9400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f40f4156c20 0x7f40f406b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59184/0 (socket says 172.21.15.121:59184) 2024-04-20T10:22:32.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.373+0000 7f40f9e00640 1 --2- 172.21.15.121:0/14177966 >> v2:172.21.15.123:3300/0 conn(0x7f40f40d0360 0x7f40f40d07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:32.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.373+0000 7f40f9400640 1 -- 172.21.15.121:0/14177966 >> v2:172.21.15.123:3300/0 conn(0x7f40f40d0360 msgr2=0x7f40f40d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:32.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.373+0000 7f40f9400640 1 --2- 172.21.15.121:0/14177966 >> v2:172.21.15.123:3300/0 conn(0x7f40f40d0360 0x7f40f40d07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:32.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.373+0000 7f40f9400640 1 -- 172.21.15.121:0/14177966 >> v1:172.21.15.121:6789/0 conn(0x7f40f415ca40 legacy=0x7f40f40ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:32.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.373+0000 7f40f9400640 1 -- 172.21.15.121:0/14177966 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f415cf90 con 0x7f40f4156c20 2024-04-20T10:22:32.375 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.373+0000 7f40f3e00640 1 -- 172.21.15.121:0/14177966 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3577211291 0 0) 0x7f40f4141c60 con 0x7f40f415ca40 2024-04-20T10:22:32.375 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.374+0000 7f40f9400640 1 --2- 172.21.15.121:0/14177966 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40f4156c20 0x7f40f406b900 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f40f406ff40 tx=0x7f40e402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e855ea471fbbcdfa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:32.375 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.375+0000 7f40f3e00640 1 -- 172.21.15.121:0/14177966 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e4035020 con 0x7f40f4156c20 2024-04-20T10:22:32.375 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.375+0000 7f40f3e00640 1 -- 172.21.15.121:0/14177966 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40e4031830 con 0x7f40f4156c20 2024-04-20T10:22:32.375 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.375+0000 7f40f3e00640 1 -- 172.21.15.121:0/14177966 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e4031c00 con 0x7f40f4156c20 2024-04-20T10:22:32.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.375+0000 7f40fbe00640 1 -- 172.21.15.121:0/14177966 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40f4156c20 msgr2=0x7f40f406b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:32.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.375+0000 7f40fbe00640 1 --2- 172.21.15.121:0/14177966 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40f4156c20 0x7f40f406b900 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f40f406ff40 tx=0x7f40e402b8d0 comp rx=0 tx=0).stop 2024-04-20T10:22:32.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.375+0000 7f40fbe00640 1 -- 172.21.15.121:0/14177966 shutdown_connections 2024-04-20T10:22:32.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.375+0000 7f40fbe00640 1 --2- 172.21.15.121:0/14177966 >> v2:172.21.15.123:3300/0 conn(0x7f40f40d0360 0x7f40f40d07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:32.377 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.375+0000 7f40fbe00640 1 --2- 172.21.15.121:0/14177966 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40f4156c20 0x7f40f406b900 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:32.377 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.375+0000 7f40fbe00640 1 -- 172.21.15.121:0/14177966 >> 172.21.15.121:0/14177966 conn(0x7f40f40b3350 msgr2=0x7f40f40b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:32.377 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.375+0000 7f40fbe00640 1 -- 172.21.15.121:0/14177966 shutdown_connections 2024-04-20T10:22:32.377 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.376+0000 7f40fbe00640 1 -- 172.21.15.121:0/14177966 wait complete. 2024-04-20T10:22:32.378 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.377+0000 7f40fbe00640 1 Processor -- start 2024-04-20T10:22:32.378 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.377+0000 7f40fbe00640 1 -- start start 2024-04-20T10:22:32.380 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.378+0000 7f40fbe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40f40d0360 0x7f40f41f5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:32.380 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.378+0000 7f40fbe00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40f4156c20 0x7f40f40cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:32.380 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.378+0000 7f40fbe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40f415ca40 0x7f40f40ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:32.380 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.378+0000 7f40fbe00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f40f415d830 con 0x7f40f415ca40 2024-04-20T10:22:32.380 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.378+0000 7f40fbe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f40f415d550 con 0x7f40f40d0360 2024-04-20T10:22:32.381 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.378+0000 7f40fbe00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f40f406be40 con 0x7f40f4156c20 2024-04-20T10:22:32.381 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.379+0000 7f40f9e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40f415ca40 0x7f40f40ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:32.381 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.379+0000 7f40f9e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40f415ca40 0x7f40f40ce0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59198/0 (socket says 172.21.15.121:59198) 2024-04-20T10:22:32.381 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.379+0000 7f40f9e00640 1 -- 172.21.15.121:0/2172065912 learned_addr learned my addr 172.21.15.121:0/2172065912 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:32.381 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.379+0000 7f40f9400640 1 --2- 172.21.15.121:0/2172065912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40f40d0360 0x7f40f41f5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:32.381 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.379+0000 7f40f8a00640 1 --2- 172.21.15.121:0/2172065912 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40f4156c20 0x7f40f40cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:32.381 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.379+0000 7f40f9e00640 1 -- 172.21.15.121:0/2172065912 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40f4156c20 msgr2=0x7f40f40cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:32.381 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.379+0000 7f40f9e00640 1 --2- 172.21.15.121:0/2172065912 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40f4156c20 0x7f40f40cdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:32.381 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.379+0000 7f40f9e00640 1 -- 172.21.15.121:0/2172065912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40f40d0360 msgr2=0x7f40f41f5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:32.381 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.379+0000 7f40f9e00640 1 --2- 172.21.15.121:0/2172065912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40f40d0360 0x7f40f41f5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:32.381 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.379+0000 7f40f9e00640 1 -- 172.21.15.121:0/2172065912 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f406f200 con 0x7f40f415ca40 2024-04-20T10:22:32.381 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.379+0000 7f40f9400640 1 --2- 172.21.15.121:0/2172065912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40f40d0360 0x7f40f41f5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:32.381 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.379+0000 7f40f9e00640 1 --2- 172.21.15.121:0/2172065912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40f415ca40 0x7f40f40ce0c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f40ec007d90 tx=0x7f40ec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:32.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.380+0000 7f40f1600640 1 -- 172.21.15.121:0/2172065912 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec014420 con 0x7f40f415ca40 2024-04-20T10:22:32.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.380+0000 7f40f1600640 1 -- 172.21.15.121:0/2172065912 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40ec019070 con 0x7f40f415ca40 2024-04-20T10:22:32.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.380+0000 7f40f1600640 1 -- 172.21.15.121:0/2172065912 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec014820 con 0x7f40f415ca40 2024-04-20T10:22:32.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.380+0000 7f40fbe00640 1 -- 172.21.15.121:0/2172065912 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40f40ce6f0 con 0x7f40f415ca40 2024-04-20T10:22:32.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.380+0000 7f40fbe00640 1 -- 172.21.15.121:0/2172065912 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40f40cecb0 con 0x7f40f415ca40 2024-04-20T10:22:32.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.381+0000 7f40fbe00640 1 -- 172.21.15.121:0/2172065912 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40f4068980 con 0x7f40f415ca40 2024-04-20T10:22:32.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.382+0000 7f40f1600640 1 -- 172.21.15.121:0/2172065912 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f40ec0149c0 con 0x7f40f415ca40 2024-04-20T10:22:32.384 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.383+0000 7f40f1600640 1 --2- 172.21.15.121:0/2172065912 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f40d406c8f0 0x7f40d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:32.384 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.383+0000 7f40f1600640 1 -- 172.21.15.121:0/2172065912 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f40ec08c890 con 0x7f40f415ca40 2024-04-20T10:22:32.384 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.383+0000 7f40f9400640 1 --2- 172.21.15.121:0/2172065912 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f40d406c8f0 0x7f40d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:32.384 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.384+0000 7f40f9400640 1 --2- 172.21.15.121:0/2172065912 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f40d406c8f0 0x7f40d406edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f40e4036000 tx=0x7f40e4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:32.388 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.388+0000 7f40f1600640 1 -- 172.21.15.121:0/2172065912 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f40ec05a820 con 0x7f40f415ca40 2024-04-20T10:22:32.590 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.590+0000 7f40fbe00640 1 -- 172.21.15.121:0/2172065912 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f40f40cfd90 con 0x7f40f415ca40 2024-04-20T10:22:32.591 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.590+0000 7f40f1600640 1 -- 172.21.15.121:0/2172065912 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f40ec05a120 con 0x7f40f415ca40 2024-04-20T10:22:32.591 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 32 2024-04-20T10:22:32.594 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.593+0000 7f40fbe00640 1 -- 172.21.15.121:0/2172065912 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f40d406c8f0 msgr2=0x7f40d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:32.594 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.593+0000 7f40fbe00640 1 --2- 172.21.15.121:0/2172065912 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f40d406c8f0 0x7f40d406edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f40e4036000 tx=0x7f40e4008030 comp rx=0 tx=0).stop 2024-04-20T10:22:32.594 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.593+0000 7f40fbe00640 1 -- 172.21.15.121:0/2172065912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40f415ca40 msgr2=0x7f40f40ce0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:32.594 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.593+0000 7f40fbe00640 1 --2- 172.21.15.121:0/2172065912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40f415ca40 0x7f40f40ce0c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f40ec007d90 tx=0x7f40ec00e4b0 comp rx=0 tx=0).stop 2024-04-20T10:22:32.594 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.593+0000 7f40fbe00640 1 -- 172.21.15.121:0/2172065912 shutdown_connections 2024-04-20T10:22:32.594 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.593+0000 7f40fbe00640 1 --2- 172.21.15.121:0/2172065912 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f40f415ca40 0x7f40f40ce0c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:32.594 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.593+0000 7f40fbe00640 1 --2- 172.21.15.121:0/2172065912 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f40f4156c20 0x7f40f40cdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:32.594 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.593+0000 7f40fbe00640 1 --2- 172.21.15.121:0/2172065912 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f40d406c8f0 0x7f40d406edb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:32.594 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.593+0000 7f40fbe00640 1 --2- 172.21.15.121:0/2172065912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40f40d0360 0x7f40f41f5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:32.594 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.593+0000 7f40fbe00640 1 -- 172.21.15.121:0/2172065912 >> 172.21.15.121:0/2172065912 conn(0x7f40f40b3350 msgr2=0x7f40f415ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:32.594 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.593+0000 7f40fbe00640 1 -- 172.21.15.121:0/2172065912 shutdown_connections 2024-04-20T10:22:32.595 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.593+0000 7f40fbe00640 1 -- 172.21.15.121:0/2172065912 wait complete. 2024-04-20T10:22:32.613 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T10:22:32.811 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.810+0000 7f4568a00640 1 Processor -- start 2024-04-20T10:22:32.811 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.811+0000 7f4568a00640 1 -- start start 2024-04-20T10:22:32.813 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.812+0000 7f4568a00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f456415b400 0x7f4564072d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:32.813 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.812+0000 7f4568a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f456415eec0 0x7f456415f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:32.813 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.812+0000 7f4568a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f456415f840 0x7f456415b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:32.813 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.812+0000 7f4568a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f456406d280 con 0x7f456415f840 2024-04-20T10:22:32.813 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.812+0000 7f4568a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f456406ccc0 con 0x7f456415eec0 2024-04-20T10:22:32.813 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.812+0000 7f4568a00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f456406cfa0 con 0x7f456415b400 2024-04-20T10:22:32.813 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.812+0000 7f4563200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f456415f840 0x7f456415b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:32.813 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.813+0000 7f4563200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f456415f840 0x7f456415b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59204/0 (socket says 172.21.15.121:59204) 2024-04-20T10:22:32.813 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.813+0000 7f4563200640 1 -- 172.21.15.121:0/4030933900 learned_addr learned my addr 172.21.15.121:0/4030933900 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:32.814 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.813+0000 7f4563200640 1 -- 172.21.15.121:0/4030933900 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f456415b400 msgr2=0x7f4564072d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:32.814 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.813+0000 7f4563200640 1 --2- 172.21.15.121:0/4030933900 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f456415b400 0x7f4564072d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:32.814 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.813+0000 7f4563200640 1 -- 172.21.15.121:0/4030933900 >> v2:172.21.15.123:3300/0 conn(0x7f456415eec0 msgr2=0x7f456415f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:32.815 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.813+0000 7f4559e00640 1 --2- 172.21.15.121:0/4030933900 >> v2:172.21.15.123:3300/0 conn(0x7f456415eec0 0x7f456415f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:32.815 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.813+0000 7f4563200640 1 --2- 172.21.15.121:0/4030933900 >> v2:172.21.15.123:3300/0 conn(0x7f456415eec0 0x7f456415f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:32.815 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.813+0000 7f4563200640 1 -- 172.21.15.121:0/4030933900 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f456406e440 con 0x7f456415f840 2024-04-20T10:22:32.815 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.814+0000 7f4563200640 1 --2- 172.21.15.121:0/4030933900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f456415f840 0x7f456415b7e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f45640d1140 tx=0x7f455c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd88f6c47ff711c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:32.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.814+0000 7f4561e00640 1 -- 172.21.15.121:0/4030933900 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455c035020 con 0x7f456415f840 2024-04-20T10:22:32.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.815+0000 7f4561e00640 1 -- 172.21.15.121:0/4030933900 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f455c031640 con 0x7f456415f840 2024-04-20T10:22:32.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.815+0000 7f4561e00640 1 -- 172.21.15.121:0/4030933900 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455c02d040 con 0x7f456415f840 2024-04-20T10:22:32.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.815+0000 7f4568a00640 1 -- 172.21.15.121:0/4030933900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f456415f840 msgr2=0x7f456415b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:32.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.815+0000 7f4568a00640 1 --2- 172.21.15.121:0/4030933900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f456415f840 0x7f456415b7e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f45640d1140 tx=0x7f455c02b8f0 comp rx=0 tx=0).stop 2024-04-20T10:22:32.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.815+0000 7f4568a00640 1 -- 172.21.15.121:0/4030933900 shutdown_connections 2024-04-20T10:22:32.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.815+0000 7f4568a00640 1 --2- 172.21.15.121:0/4030933900 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f456415f840 0x7f456415b7e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:32.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.815+0000 7f4568a00640 1 --2- 172.21.15.121:0/4030933900 >> v2:172.21.15.123:3300/0 conn(0x7f456415eec0 0x7f456415f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:32.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.815+0000 7f4568a00640 1 --2- 172.21.15.121:0/4030933900 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f456415b400 0x7f4564072d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:32.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.815+0000 7f4568a00640 1 -- 172.21.15.121:0/4030933900 >> 172.21.15.121:0/4030933900 conn(0x7f45640d0040 msgr2=0x7f45640d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:32.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.815+0000 7f4568a00640 1 -- 172.21.15.121:0/4030933900 shutdown_connections 2024-04-20T10:22:32.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.815+0000 7f4568a00640 1 -- 172.21.15.121:0/4030933900 wait complete. 2024-04-20T10:22:32.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.817+0000 7f4568a00640 1 Processor -- start 2024-04-20T10:22:32.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.817+0000 7f4568a00640 1 -- start start 2024-04-20T10:22:32.819 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.818+0000 7f4568a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f456415b400 0x7f4564148310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:32.819 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.818+0000 7f4568a00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f456415eec0 0x7f4564149750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:32.819 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.818+0000 7f4568a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f456415f840 0x7f456414ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:32.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.818+0000 7f4568a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4564073240 con 0x7f456415f840 2024-04-20T10:22:32.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.818+0000 7f4568a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f456415fc90 con 0x7f456415b400 2024-04-20T10:22:32.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.818+0000 7f4568a00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f456415bd20 con 0x7f456415eec0 2024-04-20T10:22:32.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.818+0000 7f4563200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f456415f840 0x7f456414ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:32.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.818+0000 7f4563200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f456415f840 0x7f456414ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59214/0 (socket says 172.21.15.121:59214) 2024-04-20T10:22:32.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.818+0000 7f4563200640 1 -- 172.21.15.121:0/2601760777 learned_addr learned my addr 172.21.15.121:0/2601760777 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:32.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.819+0000 7f4559e00640 1 --2- 172.21.15.121:0/2601760777 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f456415eec0 0x7f4564149750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:32.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.819+0000 7f4562800640 1 --2- 172.21.15.121:0/2601760777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f456415b400 0x7f4564148310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:32.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.819+0000 7f4563200640 1 -- 172.21.15.121:0/2601760777 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f456415eec0 msgr2=0x7f4564149750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:32.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.819+0000 7f4563200640 1 --2- 172.21.15.121:0/2601760777 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f456415eec0 0x7f4564149750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:32.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.819+0000 7f4563200640 1 -- 172.21.15.121:0/2601760777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f456415b400 msgr2=0x7f4564148310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:32.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.819+0000 7f4563200640 1 --2- 172.21.15.121:0/2601760777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f456415b400 0x7f4564148310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:32.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.819+0000 7f4563200640 1 -- 172.21.15.121:0/2601760777 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4564137a40 con 0x7f456415f840 2024-04-20T10:22:32.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.819+0000 7f4563200640 1 --2- 172.21.15.121:0/2601760777 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f456415f840 0x7f456414ac70 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f455c0097f0 tx=0x7f455c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:32.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.820+0000 7f455b400640 1 -- 172.21.15.121:0/2601760777 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455c040070 con 0x7f456415f840 2024-04-20T10:22:32.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.820+0000 7f455b400640 1 -- 172.21.15.121:0/2601760777 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f455c039cd0 con 0x7f456415f840 2024-04-20T10:22:32.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.820+0000 7f455b400640 1 -- 172.21.15.121:0/2601760777 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455c035040 con 0x7f456415f840 2024-04-20T10:22:32.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.820+0000 7f4568a00640 1 -- 172.21.15.121:0/2601760777 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4564137760 con 0x7f456415f840 2024-04-20T10:22:32.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.820+0000 7f4568a00640 1 -- 172.21.15.121:0/2601760777 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4564148f50 con 0x7f456415f840 2024-04-20T10:22:32.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.821+0000 7f4568a00640 1 -- 172.21.15.121:0/2601760777 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4564075b60 con 0x7f456415f840 2024-04-20T10:22:32.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.822+0000 7f455b400640 1 -- 172.21.15.121:0/2601760777 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f455c038020 con 0x7f456415f840 2024-04-20T10:22:32.823 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.823+0000 7f455b400640 1 --2- 172.21.15.121:0/2601760777 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f454006c8f0 0x7f454006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:32.823 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.823+0000 7f455b400640 1 -- 172.21.15.121:0/2601760777 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f455c0ad950 con 0x7f456415f840 2024-04-20T10:22:32.823 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.823+0000 7f4562800640 1 --2- 172.21.15.121:0/2601760777 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f454006c8f0 0x7f454006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:32.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.824+0000 7f4562800640 1 --2- 172.21.15.121:0/2601760777 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f454006c8f0 0x7f454006edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4554009f20 tx=0x7f45540094f0 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:32.828 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:32.828+0000 7f455b400640 1 -- 172.21.15.121:0/2601760777 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f455c07ba10 con 0x7f456415f840 2024-04-20T10:22:33.042 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.042+0000 7f4568a00640 1 -- 172.21.15.121:0/2601760777 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f45640cfdb0 con 0x7f456415f840 2024-04-20T10:22:33.043 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.043+0000 7f455b400640 1 -- 172.21.15.121:0/2601760777 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f455c07b310 con 0x7f456415f840 2024-04-20T10:22:33.043 INFO:teuthology.orchestra.run.smithi121.stderr:listed 0 entries 2024-04-20T10:22:33.046 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.045+0000 7f4568a00640 1 -- 172.21.15.121:0/2601760777 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f454006c8f0 msgr2=0x7f454006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:33.046 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.045+0000 7f4568a00640 1 --2- 172.21.15.121:0/2601760777 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f454006c8f0 0x7f454006edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4554009f20 tx=0x7f45540094f0 comp rx=0 tx=0).stop 2024-04-20T10:22:33.046 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.045+0000 7f4568a00640 1 -- 172.21.15.121:0/2601760777 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f456415f840 msgr2=0x7f456414ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:33.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.045+0000 7f4568a00640 1 --2- 172.21.15.121:0/2601760777 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f456415f840 0x7f456414ac70 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f455c0097f0 tx=0x7f455c0046c0 comp rx=0 tx=0).stop 2024-04-20T10:22:33.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.046+0000 7f4568a00640 1 -- 172.21.15.121:0/2601760777 shutdown_connections 2024-04-20T10:22:33.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.046+0000 7f4568a00640 1 --2- 172.21.15.121:0/2601760777 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f456415f840 0x7f456414ac70 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:33.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.046+0000 7f4568a00640 1 --2- 172.21.15.121:0/2601760777 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f456415eec0 0x7f4564149750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:33.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.046+0000 7f4568a00640 1 --2- 172.21.15.121:0/2601760777 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f454006c8f0 0x7f454006edb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:33.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.046+0000 7f4568a00640 1 --2- 172.21.15.121:0/2601760777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f456415b400 0x7f4564148310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:33.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.046+0000 7f4568a00640 1 -- 172.21.15.121:0/2601760777 >> 172.21.15.121:0/2601760777 conn(0x7f45640d0040 msgr2=0x7f45640b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:33.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.046+0000 7f4568a00640 1 -- 172.21.15.121:0/2601760777 shutdown_connections 2024-04-20T10:22:33.047 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.046+0000 7f4568a00640 1 -- 172.21.15.121:0/2601760777 wait complete. 2024-04-20T10:22:33.064 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi123.front.sepia.ceph.com... 2024-04-20T10:22:33.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:22:33.064 DEBUG:teuthology.orchestra.run.smithi123:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-04-20T10:22:33.151 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:22:33.151 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2024-04-20T10:22:33.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:22:33.151 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-20T10:22:33.200 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-20T10:22:33.200 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:22:33.400 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.399+0000 7f8697000640 1 Processor -- start 2024-04-20T10:22:33.400 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.399+0000 7f8697000640 1 -- start start 2024-04-20T10:22:33.401 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.400+0000 7f8697000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8690156c20 0x7f869006b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:33.401 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.401+0000 7f8697000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86900d0360 0x7f86900d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:33.401 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.401+0000 7f8697000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8690141c60 con 0x7f869015ca40 2024-04-20T10:22:33.402 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.401+0000 7f8697000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f86900adab0 con 0x7f8690156c20 2024-04-20T10:22:33.402 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.401+0000 7f8697000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f869015cf90 con 0x7f86900d0360 2024-04-20T10:22:33.402 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.401+0000 7f868fe00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8690156c20 0x7f869006b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:33.402 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.401+0000 7f868fe00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f8690156c20 0x7f869006b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59218/0 (socket says 172.21.15.121:59218) 2024-04-20T10:22:33.402 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.401+0000 7f868f400640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f869015ca40 0x7f86900ad3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:56204/0 (socket says 172.21.15.121:56204) 2024-04-20T10:22:33.402 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.401+0000 7f868fe00640 1 -- 172.21.15.121:0/965881802 learned_addr learned my addr 172.21.15.121:0/965881802 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:33.403 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.401+0000 7f8695000640 1 --2- 172.21.15.121:0/965881802 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86900d0360 0x7f86900d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:33.403 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.401+0000 7f868fe00640 1 -- 172.21.15.121:0/965881802 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86900d0360 msgr2=0x7f86900d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:33.403 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.401+0000 7f868fe00640 1 --2- 172.21.15.121:0/965881802 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86900d0360 0x7f86900d07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:33.403 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.402+0000 7f868fe00640 1 -- 172.21.15.121:0/965881802 >> v1:172.21.15.121:6789/0 conn(0x7f869015ca40 legacy=0x7f86900ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:33.403 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.402+0000 7f868fe00640 1 -- 172.21.15.121:0/965881802 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f869015d270 con 0x7f8690156c20 2024-04-20T10:22:33.403 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.403+0000 7f868fe00640 1 --2- 172.21.15.121:0/965881802 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8690156c20 0x7f869006b900 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8680009870 tx=0x7f868002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=683abb242ebf3790 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:33.404 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.403+0000 7f868ea00640 1 -- 172.21.15.121:0/965881802 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8680034070 con 0x7f8690156c20 2024-04-20T10:22:33.404 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.403+0000 7f868ea00640 1 -- 172.21.15.121:0/965881802 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8680030840 con 0x7f8690156c20 2024-04-20T10:22:33.404 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.403+0000 7f868ea00640 1 -- 172.21.15.121:0/965881802 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8680030c10 con 0x7f8690156c20 2024-04-20T10:22:33.405 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.404+0000 7f8697000640 1 -- 172.21.15.121:0/965881802 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8690156c20 msgr2=0x7f869006b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:33.405 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.404+0000 7f8697000640 1 --2- 172.21.15.121:0/965881802 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8690156c20 0x7f869006b900 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8680009870 tx=0x7f868002b980 comp rx=0 tx=0).stop 2024-04-20T10:22:33.405 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.404+0000 7f8697000640 1 -- 172.21.15.121:0/965881802 shutdown_connections 2024-04-20T10:22:33.405 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.404+0000 7f8697000640 1 --2- 172.21.15.121:0/965881802 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86900d0360 0x7f86900d07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:33.405 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.404+0000 7f8697000640 1 --2- 172.21.15.121:0/965881802 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8690156c20 0x7f869006b900 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:33.405 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.404+0000 7f8697000640 1 -- 172.21.15.121:0/965881802 >> 172.21.15.121:0/965881802 conn(0x7f86900b3350 msgr2=0x7f86900b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:33.405 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.404+0000 7f8697000640 1 -- 172.21.15.121:0/965881802 shutdown_connections 2024-04-20T10:22:33.405 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.404+0000 7f8697000640 1 -- 172.21.15.121:0/965881802 wait complete. 2024-04-20T10:22:33.406 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.405+0000 7f8697000640 1 Processor -- start 2024-04-20T10:22:33.406 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.406+0000 7f8697000640 1 -- start start 2024-04-20T10:22:33.408 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.406+0000 7f8697000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86900d0360 0x7f86901f37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:33.408 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.407+0000 7f8697000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8690156c20 0x7f869014f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:33.408 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.407+0000 7f8697000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f869015ca40 0x7f8690151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:33.408 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.407+0000 7f8697000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f869006be40 con 0x7f869015ca40 2024-04-20T10:22:33.408 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.407+0000 7f8697000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f869015d830 con 0x7f8690156c20 2024-04-20T10:22:33.408 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.407+0000 7f8697000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f869006bfc0 con 0x7f86900d0360 2024-04-20T10:22:33.408 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.407+0000 7f8695000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f869015ca40 0x7f8690151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:33.408 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.407+0000 7f8695000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f869015ca40 0x7f8690151c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59232/0 (socket says 172.21.15.121:59232) 2024-04-20T10:22:33.408 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.407+0000 7f8695000640 1 -- 172.21.15.121:0/2127201635 learned_addr learned my addr 172.21.15.121:0/2127201635 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:33.408 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.407+0000 7f8695000640 1 -- 172.21.15.121:0/2127201635 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86900d0360 msgr2=0x7f86901f37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:33.409 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.407+0000 7f8695000640 1 --2- 172.21.15.121:0/2127201635 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86900d0360 0x7f86901f37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:33.409 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.407+0000 7f8695000640 1 -- 172.21.15.121:0/2127201635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8690156c20 msgr2=0x7f869014f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:33.409 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.407+0000 7f8695000640 1 --2- 172.21.15.121:0/2127201635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8690156c20 0x7f869014f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:33.409 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.407+0000 7f8695000640 1 -- 172.21.15.121:0/2127201635 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86901524f0 con 0x7f869015ca40 2024-04-20T10:22:33.409 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.408+0000 7f8695000640 1 --2- 172.21.15.121:0/2127201635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f869015ca40 0x7f8690151c10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8688007dc0 tx=0x7f868800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:33.410 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.408+0000 7f8687e00640 1 -- 172.21.15.121:0/2127201635 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8688014420 con 0x7f869015ca40 2024-04-20T10:22:33.410 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.408+0000 7f8687e00640 1 -- 172.21.15.121:0/2127201635 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8688019070 con 0x7f869015ca40 2024-04-20T10:22:33.410 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.408+0000 7f8687e00640 1 -- 172.21.15.121:0/2127201635 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8688014820 con 0x7f869015ca40 2024-04-20T10:22:33.410 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.408+0000 7f8697000640 1 -- 172.21.15.121:0/2127201635 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86901f2cf0 con 0x7f869015ca40 2024-04-20T10:22:33.410 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.408+0000 7f8697000640 1 -- 172.21.15.121:0/2127201635 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86901527d0 con 0x7f869015ca40 2024-04-20T10:22:33.410 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.409+0000 7f8697000640 1 -- 172.21.15.121:0/2127201635 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8690068980 con 0x7f869015ca40 2024-04-20T10:22:33.410 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.410+0000 7f8687e00640 1 -- 172.21.15.121:0/2127201635 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f86880149c0 con 0x7f869015ca40 2024-04-20T10:22:33.411 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.411+0000 7f8687e00640 1 --2- 172.21.15.121:0/2127201635 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f867006c8f0 0x7f867006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:33.411 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.411+0000 7f8687e00640 1 -- 172.21.15.121:0/2127201635 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f868808c820 con 0x7f869015ca40 2024-04-20T10:22:33.412 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.411+0000 7f868fe00640 1 --2- 172.21.15.121:0/2127201635 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f867006c8f0 0x7f867006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:33.412 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.412+0000 7f868fe00640 1 --2- 172.21.15.121:0/2127201635 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f867006c8f0 0x7f867006edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8680036000 tx=0x7f868003b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:33.416 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.416+0000 7f8687e00640 1 -- 172.21.15.121:0/2127201635 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f868805a7c0 con 0x7f869015ca40 2024-04-20T10:22:33.656 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.656+0000 7f8697000640 1 -- 172.21.15.121:0/2127201635 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86900d0a00 con 0x7f869015ca40 2024-04-20T10:22:33.658 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.657+0000 7f8687e00640 1 -- 172.21.15.121:0/2127201635 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5204 (secure 0 0 0) 0x7f868805a0c0 con 0x7f869015ca40 2024-04-20T10:22:33.658 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:22:33.658 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1376743826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1376743826},{"type":"v1","addr":"172.21.15.123:6833","nonce":1376743826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/999167058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":999167058},{"type":"v1","addr":"172.21.15.121:6835","nonce":999167058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2160736935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2160736935},{"type":"v1","addr":"172.21.15.121:6837","nonce":2160736935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:21:02.048590+0000","modified":"2024-04-20T10:21:29.775269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6835/3336775250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3336775250},{"type":"v1","addr":"172.21.15.123:6835","nonce":3336775250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1}]} 2024-04-20T10:22:33.658 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 8 2024-04-20T10:22:33.661 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.660+0000 7f8697000640 1 -- 172.21.15.121:0/2127201635 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f867006c8f0 msgr2=0x7f867006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:33.661 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.660+0000 7f8697000640 1 --2- 172.21.15.121:0/2127201635 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f867006c8f0 0x7f867006edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8680036000 tx=0x7f868003b040 comp rx=0 tx=0).stop 2024-04-20T10:22:33.661 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.660+0000 7f8697000640 1 -- 172.21.15.121:0/2127201635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f869015ca40 msgr2=0x7f8690151c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:33.661 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.660+0000 7f8697000640 1 --2- 172.21.15.121:0/2127201635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f869015ca40 0x7f8690151c10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8688007dc0 tx=0x7f868800e4b0 comp rx=0 tx=0).stop 2024-04-20T10:22:33.661 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.660+0000 7f8697000640 1 -- 172.21.15.121:0/2127201635 shutdown_connections 2024-04-20T10:22:33.661 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.660+0000 7f8697000640 1 --2- 172.21.15.121:0/2127201635 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f869015ca40 0x7f8690151c10 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:33.661 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.660+0000 7f8697000640 1 --2- 172.21.15.121:0/2127201635 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f867006c8f0 0x7f867006edb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:33.661 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.660+0000 7f8697000640 1 --2- 172.21.15.121:0/2127201635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8690156c20 0x7f869014f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:33.661 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.660+0000 7f8697000640 1 --2- 172.21.15.121:0/2127201635 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86900d0360 0x7f86901f37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:33.661 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.660+0000 7f8697000640 1 -- 172.21.15.121:0/2127201635 >> 172.21.15.121:0/2127201635 conn(0x7f86900b3350 msgr2=0x7f86900afce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:33.662 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.660+0000 7f8697000640 1 -- 172.21.15.121:0/2127201635 shutdown_connections 2024-04-20T10:22:33.662 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.660+0000 7f8697000640 1 -- 172.21.15.121:0/2127201635 wait complete. 2024-04-20T10:22:33.679 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:22:33.881 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.880+0000 7f92ae000640 1 Processor -- start 2024-04-20T10:22:33.881 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.881+0000 7f92ae000640 1 -- start start 2024-04-20T10:22:33.883 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.882+0000 7f92ae000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f92a815c310 0x7f92a80716e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:33.884 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.882+0000 7f92ae000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f92a80cff70 0x7f92a8071c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:33.884 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.882+0000 7f92ae000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f92a80c7320 0x7f92a80c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:33.884 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.882+0000 7f92ae000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f92a815c6f0 con 0x7f92a80cff70 2024-04-20T10:22:33.884 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.882+0000 7f92ae000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f92a815eee0 con 0x7f92a815c310 2024-04-20T10:22:33.884 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.882+0000 7f92ae000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f92a815f1c0 con 0x7f92a80c7320 2024-04-20T10:22:33.884 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.882+0000 7f92a6a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f92a80cff70 0x7f92a8071c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:33.884 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.883+0000 7f92a6a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f92a80cff70 0x7f92a8071c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59242/0 (socket says 172.21.15.121:59242) 2024-04-20T10:22:33.884 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.883+0000 7f92a6a00640 1 -- 172.21.15.121:0/1296317798 learned_addr learned my addr 172.21.15.121:0/1296317798 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:33.884 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.883+0000 7f92a7400640 1 --2- 172.21.15.121:0/1296317798 >> v2:172.21.15.123:3300/0 conn(0x7f92a815c310 0x7f92a80716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:33.884 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.883+0000 7f92a7e00640 1 --2- 172.21.15.121:0/1296317798 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f92a80c7320 0x7f92a80c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:33.884 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.883+0000 7f92a6a00640 1 -- 172.21.15.121:0/1296317798 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f92a80c7320 msgr2=0x7f92a80c77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:33.884 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.883+0000 7f92a6a00640 1 --2- 172.21.15.121:0/1296317798 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f92a80c7320 0x7f92a80c77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:33.884 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.883+0000 7f92a6a00640 1 -- 172.21.15.121:0/1296317798 >> v2:172.21.15.123:3300/0 conn(0x7f92a815c310 msgr2=0x7f92a80716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:33.884 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.883+0000 7f92a6a00640 1 --2- 172.21.15.121:0/1296317798 >> v2:172.21.15.123:3300/0 conn(0x7f92a815c310 0x7f92a80716e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:33.884 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.883+0000 7f92a6a00640 1 -- 172.21.15.121:0/1296317798 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a815f4a0 con 0x7f92a80cff70 2024-04-20T10:22:33.885 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.883+0000 7f92a7e00640 1 --2- 172.21.15.121:0/1296317798 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f92a80c7320 0x7f92a80c77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:33.886 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.884+0000 7f92a6a00640 1 --2- 172.21.15.121:0/1296317798 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92a80cff70 0x7f92a8071c20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f92a80d3450 tx=0x7f929002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=312afa173466c57 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:33.886 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.884+0000 7f92a6000640 1 -- 172.21.15.121:0/1296317798 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9290035020 con 0x7f92a80cff70 2024-04-20T10:22:33.886 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.884+0000 7f92a6000640 1 -- 172.21.15.121:0/1296317798 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9290031870 con 0x7f92a80cff70 2024-04-20T10:22:33.886 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.884+0000 7f92a6000640 1 -- 172.21.15.121:0/1296317798 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9290031c40 con 0x7f92a80cff70 2024-04-20T10:22:33.886 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.885+0000 7f92ae000640 1 -- 172.21.15.121:0/1296317798 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92a80cff70 msgr2=0x7f92a8071c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:33.886 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.885+0000 7f92ae000640 1 --2- 172.21.15.121:0/1296317798 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92a80cff70 0x7f92a8071c20 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f92a80d3450 tx=0x7f929002b910 comp rx=0 tx=0).stop 2024-04-20T10:22:33.886 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.885+0000 7f92ae000640 1 -- 172.21.15.121:0/1296317798 shutdown_connections 2024-04-20T10:22:33.886 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.885+0000 7f92ae000640 1 --2- 172.21.15.121:0/1296317798 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f92a80c7320 0x7f92a80c77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:33.886 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.885+0000 7f92ae000640 1 --2- 172.21.15.121:0/1296317798 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92a80cff70 0x7f92a8071c20 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:33.886 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.885+0000 7f92ae000640 1 --2- 172.21.15.121:0/1296317798 >> v2:172.21.15.123:3300/0 conn(0x7f92a815c310 0x7f92a80716e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:33.886 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.885+0000 7f92ae000640 1 -- 172.21.15.121:0/1296317798 >> 172.21.15.121:0/1296317798 conn(0x7f92a80d23a0 msgr2=0x7f92a80d27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:33.887 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.885+0000 7f92ae000640 1 -- 172.21.15.121:0/1296317798 shutdown_connections 2024-04-20T10:22:33.887 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.885+0000 7f92ae000640 1 -- 172.21.15.121:0/1296317798 wait complete. 2024-04-20T10:22:33.887 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.887+0000 7f92ae000640 1 Processor -- start 2024-04-20T10:22:33.887 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.887+0000 7f92ae000640 1 -- start start 2024-04-20T10:22:33.889 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.888+0000 7f92ae000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f92a80c7320 0x7f92a81f4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:33.889 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.888+0000 7f92ae000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92a80cff70 0x7f92a8161b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:33.890 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.888+0000 7f92ae000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92a815c310 0x7f92a80cdd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:33.890 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.888+0000 7f92ae000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f92a80c7e70 con 0x7f92a80cff70 2024-04-20T10:22:33.890 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.888+0000 7f92ae000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f92a80c7cf0 con 0x7f92a815c310 2024-04-20T10:22:33.890 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.888+0000 7f92ae000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f92a80b1f20 con 0x7f92a80c7320 2024-04-20T10:22:33.890 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.888+0000 7f92a6a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92a80cff70 0x7f92a8161b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:33.890 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.888+0000 7f92a6a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92a80cff70 0x7f92a8161b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59250/0 (socket says 172.21.15.121:59250) 2024-04-20T10:22:33.890 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.888+0000 7f92a6a00640 1 -- 172.21.15.121:0/2587349546 learned_addr learned my addr 172.21.15.121:0/2587349546 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:33.890 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.889+0000 7f92a7400640 1 --2- 172.21.15.121:0/2587349546 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f92a80c7320 0x7f92a81f4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:33.890 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.889+0000 7f92a6a00640 1 -- 172.21.15.121:0/2587349546 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f92a80c7320 msgr2=0x7f92a81f4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:33.890 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.889+0000 7f92a6a00640 1 --2- 172.21.15.121:0/2587349546 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f92a80c7320 0x7f92a81f4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:33.890 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.889+0000 7f92a6a00640 1 -- 172.21.15.121:0/2587349546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92a815c310 msgr2=0x7f92a80cdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:33.890 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.889+0000 7f92a7e00640 1 --2- 172.21.15.121:0/2587349546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92a815c310 0x7f92a80cdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:33.891 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.889+0000 7f92a6a00640 1 --2- 172.21.15.121:0/2587349546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92a815c310 0x7f92a80cdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:33.891 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.889+0000 7f92a6a00640 1 -- 172.21.15.121:0/2587349546 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a81f5370 con 0x7f92a80cff70 2024-04-20T10:22:33.891 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.889+0000 7f92a6a00640 1 --2- 172.21.15.121:0/2587349546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92a80cff70 0x7f92a8161b40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f929002b950 tx=0x7f92900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:33.891 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.890+0000 7f929f400640 1 -- 172.21.15.121:0/2587349546 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9290040070 con 0x7f92a80cff70 2024-04-20T10:22:33.892 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.890+0000 7f929f400640 1 -- 172.21.15.121:0/2587349546 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f929003a420 con 0x7f92a80cff70 2024-04-20T10:22:33.892 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.890+0000 7f929f400640 1 -- 172.21.15.121:0/2587349546 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9290035040 con 0x7f92a80cff70 2024-04-20T10:22:33.892 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.890+0000 7f92ae000640 1 -- 172.21.15.121:0/2587349546 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92a8162080 con 0x7f92a80cff70 2024-04-20T10:22:33.892 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.890+0000 7f92ae000640 1 -- 172.21.15.121:0/2587349546 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92a8160d80 con 0x7f92a80cff70 2024-04-20T10:22:33.892 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.891+0000 7f92ae000640 1 -- 172.21.15.121:0/2587349546 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92a8068980 con 0x7f92a80cff70 2024-04-20T10:22:33.892 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.892+0000 7f929f400640 1 -- 172.21.15.121:0/2587349546 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f929004b060 con 0x7f92a80cff70 2024-04-20T10:22:33.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.893+0000 7f929f400640 1 --2- 172.21.15.121:0/2587349546 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f928406c920 0x7f928406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:33.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.893+0000 7f929f400640 1 -- 172.21.15.121:0/2587349546 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f92900ade10 con 0x7f92a80cff70 2024-04-20T10:22:33.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.893+0000 7f92a7400640 1 --2- 172.21.15.121:0/2587349546 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f928406c920 0x7f928406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:33.894 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.894+0000 7f92a7400640 1 --2- 172.21.15.121:0/2587349546 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f928406c920 0x7f928406ede0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f92a80ce770 tx=0x7f9298008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:33.898 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:33.898+0000 7f929f400640 1 -- 172.21.15.121:0/2587349546 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f92a81f5370 con 0x7f92a80cff70 2024-04-20T10:22:34.101 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.100+0000 7f92ae000640 1 -- 172.21.15.121:0/2587349546 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f92a806f090 con 0x7f92a80cff70 2024-04-20T10:22:34.104 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.103+0000 7f929f400640 1 -- 172.21.15.121:0/2587349546 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18940 (secure 0 0 0) 0x7f929007bed0 con 0x7f92a80cff70 2024-04-20T10:22:34.104 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:22:34.105 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":27,"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","created":"2024-04-20T10:20:38.076076+0000","modified":"2024-04-20T10:21:44.715898+0000","last_up_change":"2024-04-20T10:20:50.162700+0000","last_in_change":"2024-04-20T10:20:41.510382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:20:51.141365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:21:00.436937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:21:01.026957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:21:04.862693+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"629a7b8c-a40a-4a2a-a3cc-d9f357774310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6809","nonce":1044588835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6811","nonce":1044588835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6815","nonce":1044588835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6813","nonce":1044588835}]},"public_addr":"172.21.15.121:6809/1044588835","cluster_addr":"172.21.15.121:6811/1044588835","heartbeat_back_addr":"172.21.15.121:6815/1044588835","heartbeat_front_addr":"172.21.15.121:6813/1044588835","state":["exists","up"]},{"osd":1,"uuid":"f2570399-aa50-4ff3-bf5d-3d09207290db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6801","nonce":3505199624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6803","nonce":3505199624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6807","nonce":3505199624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6805","nonce":3505199624}]},"public_addr":"172.21.15.121:6801/3505199624","cluster_addr":"172.21.15.121:6803/3505199624","heartbeat_back_addr":"172.21.15.121:6807/3505199624","heartbeat_front_addr":"172.21.15.121:6805/3505199624","state":["exists","up"]},{"osd":2,"uuid":"50893807-dc30-49f4-b41f-739366626658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6817","nonce":3656724207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6819","nonce":3656724207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6827","nonce":3656724207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6822","nonce":3656724207}]},"public_addr":"172.21.15.121:6817/3656724207","cluster_addr":"172.21.15.121:6819/3656724207","heartbeat_back_addr":"172.21.15.121:6827/3656724207","heartbeat_front_addr":"172.21.15.121:6822/3656724207","state":["exists","up"]},{"osd":3,"uuid":"dadb01bc-5491-4ffc-8824-d48a057240b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6823","nonce":2232342832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6826","nonce":2232342832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6831","nonce":2232342832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6829","nonce":2232342832}]},"public_addr":"172.21.15.121:6823/2232342832","cluster_addr":"172.21.15.121:6826/2232342832","heartbeat_back_addr":"172.21.15.121:6831/2232342832","heartbeat_front_addr":"172.21.15.121:6829/2232342832","state":["exists","up"]},{"osd":4,"uuid":"b3f3b4a1-b375-4526-b69e-344ab24c2da1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6801","nonce":99353111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6803","nonce":99353111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6807","nonce":99353111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6805","nonce":99353111}]},"public_addr":"172.21.15.123:6801/99353111","cluster_addr":"172.21.15.123:6803/99353111","heartbeat_back_addr":"172.21.15.123:6807/99353111","heartbeat_front_addr":"172.21.15.123:6805/99353111","state":["exists","up"]},{"osd":5,"uuid":"2ee24a08-1624-4961-9642-d7b510282484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6818","nonce":2504374688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6822","nonce":2504374688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6830","nonce":2504374688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6826","nonce":2504374688}]},"public_addr":"172.21.15.123:6818/2504374688","cluster_addr":"172.21.15.123:6822/2504374688","heartbeat_back_addr":"172.21.15.123:6830/2504374688","heartbeat_front_addr":"172.21.15.123:6826/2504374688","state":["exists","up"]},{"osd":6,"uuid":"186118cc-6128-49de-acca-55260945c1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6819","nonce":4139299599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6823","nonce":4139299599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6829","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6831","nonce":4139299599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6827","nonce":4139299599}]},"public_addr":"172.21.15.123:6819/4139299599","cluster_addr":"172.21.15.123:6823/4139299599","heartbeat_back_addr":"172.21.15.123:6831/4139299599","heartbeat_front_addr":"172.21.15.123:6827/4139299599","state":["exists","up"]},{"osd":7,"uuid":"a00aac6e-4bd9-415a-a93d-ae85210865ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6809","nonce":4043981069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6811","nonce":4043981069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6815","nonce":4043981069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6813","nonce":4043981069}]},"public_addr":"172.21.15.123:6809/4043981069","cluster_addr":"172.21.15.123:6811/4043981069","heartbeat_back_addr":"172.21.15.123:6815/4043981069","heartbeat_front_addr":"172.21.15.123:6813/4043981069","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.904540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.958585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.616367+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.885244+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.103127+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.167248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.884348+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.862356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.23","mappings":[{"from":3,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:22:34.107 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.106+0000 7f92ae000640 1 -- 172.21.15.121:0/2587349546 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f928406c920 msgr2=0x7f928406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:34.107 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.106+0000 7f92ae000640 1 --2- 172.21.15.121:0/2587349546 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f928406c920 0x7f928406ede0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f92a80ce770 tx=0x7f9298008040 comp rx=0 tx=0).stop 2024-04-20T10:22:34.107 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.106+0000 7f92ae000640 1 -- 172.21.15.121:0/2587349546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92a80cff70 msgr2=0x7f92a8161b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:34.107 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.106+0000 7f92ae000640 1 --2- 172.21.15.121:0/2587349546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92a80cff70 0x7f92a8161b40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f929002b950 tx=0x7f92900046c0 comp rx=0 tx=0).stop 2024-04-20T10:22:34.108 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.106+0000 7f92ae000640 1 -- 172.21.15.121:0/2587349546 shutdown_connections 2024-04-20T10:22:34.108 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.106+0000 7f92ae000640 1 --2- 172.21.15.121:0/2587349546 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f928406c920 0x7f928406ede0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:34.108 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.106+0000 7f92ae000640 1 --2- 172.21.15.121:0/2587349546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92a815c310 0x7f92a80cdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:34.108 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.106+0000 7f92ae000640 1 --2- 172.21.15.121:0/2587349546 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f92a80cff70 0x7f92a8161b40 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:34.108 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.106+0000 7f92ae000640 1 --2- 172.21.15.121:0/2587349546 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f92a80c7320 0x7f92a81f4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:34.108 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.106+0000 7f92ae000640 1 -- 172.21.15.121:0/2587349546 >> 172.21.15.121:0/2587349546 conn(0x7f92a80d23a0 msgr2=0x7f92a80d1180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:34.108 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.107+0000 7f92ae000640 1 -- 172.21.15.121:0/2587349546 shutdown_connections 2024-04-20T10:22:34.108 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.107+0000 7f92ae000640 1 -- 172.21.15.121:0/2587349546 wait complete. 2024-04-20T10:22:34.127 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:20:51.141365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:21:00.436937+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:21:01.026957+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T10:21:04.862693+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T10:22:34.128 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:22:34.334 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.333+0000 7f244d800640 1 Processor -- start 2024-04-20T10:22:34.334 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.333+0000 7f244d800640 1 -- start start 2024-04-20T10:22:34.336 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.335+0000 7f244d800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f244815e8a0 0x7f244806e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:34.336 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.335+0000 7f244d800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f244806dbb0 0x7f24480d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:34.336 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.335+0000 7f244d800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f244806f060 con 0x7f24480d08a0 2024-04-20T10:22:34.338 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.335+0000 7f244d800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f244806e7c0 con 0x7f244815e8a0 2024-04-20T10:22:34.338 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.335+0000 7f244d800640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f244806eaa0 con 0x7f244806dbb0 2024-04-20T10:22:34.338 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.335+0000 7f2447400640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f24480d08a0 0x7f244806b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:56214/0 (socket says 172.21.15.121:56214) 2024-04-20T10:22:34.338 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.335+0000 7f2447400640 1 -- 172.21.15.121:0/3958392196 learned_addr learned my addr 172.21.15.121:0/3958392196 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:34.338 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.336+0000 7f2446a00640 1 --2- 172.21.15.121:0/3958392196 >> v2:172.21.15.123:3300/0 conn(0x7f244815e8a0 0x7f244806e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:34.338 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.336+0000 7f243fe00640 1 --2- 172.21.15.121:0/3958392196 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f244806dbb0 0x7f24480d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:34.338 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.336+0000 7f2446000640 1 -- 172.21.15.121:0/3958392196 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3029483296 0 0) 0x7f244806f060 con 0x7f24480d08a0 2024-04-20T10:22:34.339 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.336+0000 7f2446000640 1 -- 172.21.15.121:0/3958392196 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f242c003520 con 0x7f24480d08a0 2024-04-20T10:22:34.339 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.336+0000 7f2446000640 1 -- 172.21.15.121:0/3958392196 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3597314945 0 0) 0x7f242c003520 con 0x7f24480d08a0 2024-04-20T10:22:34.339 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.336+0000 7f2446000640 1 -- 172.21.15.121:0/3958392196 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f244806dbb0 msgr2=0x7f24480d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:34.339 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.336+0000 7f2446000640 1 --2- 172.21.15.121:0/3958392196 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f244806dbb0 0x7f24480d0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:34.339 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.336+0000 7f2446000640 1 -- 172.21.15.121:0/3958392196 >> v2:172.21.15.123:3300/0 conn(0x7f244815e8a0 msgr2=0x7f244806e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:34.339 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.336+0000 7f2446000640 1 --2- 172.21.15.121:0/3958392196 >> v2:172.21.15.123:3300/0 conn(0x7f244815e8a0 0x7f244806e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:34.339 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.336+0000 7f2446000640 1 -- 172.21.15.121:0/3958392196 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f244806ed80 con 0x7f24480d08a0 2024-04-20T10:22:34.339 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.336+0000 7f2446000640 1 -- 172.21.15.121:0/3958392196 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f24400019f0 con 0x7f24480d08a0 2024-04-20T10:22:34.339 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.336+0000 7f2446000640 1 -- 172.21.15.121:0/3958392196 >> v1:172.21.15.121:6789/0 conn(0x7f24480d08a0 legacy=0x7f244806b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:34.340 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.336+0000 7f2446000640 1 --2- 172.21.15.121:0/3958392196 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f244806dbb0 0x7f242c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:34.340 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.337+0000 7f2446000640 1 --2- 172.21.15.121:0/3958392196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f242c003fe0 0x7f242c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:34.340 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.337+0000 7f2446000640 1 --2- 172.21.15.121:0/3958392196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f242c004d40 0x7f242c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:34.340 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.337+0000 7f2446000640 1 -- 172.21.15.121:0/3958392196 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f244806e7c0 con 0x7f242c004d40 2024-04-20T10:22:34.340 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.337+0000 7f2446000640 1 -- 172.21.15.121:0/3958392196 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f244806eaa0 con 0x7f242c003fe0 2024-04-20T10:22:34.340 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.337+0000 7f2446000640 1 -- 172.21.15.121:0/3958392196 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f242c00bac0 con 0x7f244806dbb0 2024-04-20T10:22:34.340 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.337+0000 7f2447400640 1 --2- 172.21.15.121:0/3958392196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f242c004d40 0x7f242c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:34.340 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.337+0000 7f2446a00640 1 --2- 172.21.15.121:0/3958392196 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f244806dbb0 0x7f242c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:34.340 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.337+0000 7f2447400640 1 -- 172.21.15.121:0/3958392196 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f244806dbb0 msgr2=0x7f242c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:34.340 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.337+0000 7f2447400640 1 --2- 172.21.15.121:0/3958392196 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f244806dbb0 0x7f242c0039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:34.341 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.337+0000 7f2447400640 1 -- 172.21.15.121:0/3958392196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f242c003fe0 msgr2=0x7f242c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:34.341 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.337+0000 7f2447400640 1 --2- 172.21.15.121:0/3958392196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f242c003fe0 0x7f242c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:34.341 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.337+0000 7f2447400640 1 -- 172.21.15.121:0/3958392196 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f242c003520 con 0x7f242c004d40 2024-04-20T10:22:34.342 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.338+0000 7f2447400640 1 --2- 172.21.15.121:0/3958392196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f242c004d40 0x7f242c0051c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f244000a920 tx=0x7f242c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992c80778c9cb1b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:34.342 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.339+0000 7f2446000640 1 -- 172.21.15.121:0/3958392196 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244000a4c0 con 0x7f242c004d40 2024-04-20T10:22:34.342 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.339+0000 7f2446000640 1 -- 172.21.15.121:0/3958392196 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2440034650 con 0x7f242c004d40 2024-04-20T10:22:34.342 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.339+0000 7f2446000640 1 -- 172.21.15.121:0/3958392196 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2440034a80 con 0x7f242c004d40 2024-04-20T10:22:34.342 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.339+0000 7f244d800640 1 -- 172.21.15.121:0/3958392196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f242c004d40 msgr2=0x7f242c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:34.342 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.339+0000 7f244d800640 1 --2- 172.21.15.121:0/3958392196 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f242c004d40 0x7f242c0051c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f244000a920 tx=0x7f242c005830 comp rx=0 tx=0).stop 2024-04-20T10:22:34.342 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.339+0000 7f2447400640 1 -- 172.21.15.121:0/3958392196 reap_dead start 2024-04-20T10:22:34.343 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.339+0000 7f244d800640 1 -- 172.21.15.121:0/3958392196 shutdown_connections 2024-04-20T10:22:34.343 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.339+0000 7f244d800640 1 -- 172.21.15.121:0/3958392196 >> 172.21.15.121:0/3958392196 conn(0x7f24480b3350 msgr2=0x7f24480b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:34.343 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.339+0000 7f244d800640 1 -- 172.21.15.121:0/3958392196 shutdown_connections 2024-04-20T10:22:34.343 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.339+0000 7f244d800640 1 -- 172.21.15.121:0/3958392196 wait complete. 2024-04-20T10:22:34.343 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.341+0000 7f244d800640 1 Processor -- start 2024-04-20T10:22:34.343 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.341+0000 7f244d800640 1 -- start start 2024-04-20T10:22:34.345 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.342+0000 7f244d800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f242c004d40 0x7f244814da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:34.345 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.342+0000 7f244d800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f244814efe0 0x7f2448149ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:34.345 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.342+0000 7f244d800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f244814a780 0x7f24480b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:34.345 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.342+0000 7f244d800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f244806fef0 con 0x7f242c004d40 2024-04-20T10:22:34.345 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.342+0000 7f244d800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f244806fd70 con 0x7f244814a780 2024-04-20T10:22:34.345 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.342+0000 7f244d800640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f24480ae690 con 0x7f244814efe0 2024-04-20T10:22:34.345 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.343+0000 7f2446a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f242c004d40 0x7f244814da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:34.345 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.343+0000 7f2446a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f242c004d40 0x7f244814da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59278/0 (socket says 172.21.15.121:59278) 2024-04-20T10:22:34.345 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.343+0000 7f2446a00640 1 -- 172.21.15.121:0/261293949 learned_addr learned my addr 172.21.15.121:0/261293949 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:34.345 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.343+0000 7f2446a00640 1 -- 172.21.15.121:0/261293949 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f244814efe0 msgr2=0x7f2448149ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:34.346 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.343+0000 7f2446a00640 1 --2- 172.21.15.121:0/261293949 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f244814efe0 0x7f2448149ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:34.346 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.343+0000 7f2446a00640 1 -- 172.21.15.121:0/261293949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f244814a780 msgr2=0x7f24480b0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:34.346 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.343+0000 7f2446a00640 1 --2- 172.21.15.121:0/261293949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f244814a780 0x7f24480b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:34.346 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.343+0000 7f2446a00640 1 -- 172.21.15.121:0/261293949 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24480aed10 con 0x7f242c004d40 2024-04-20T10:22:34.346 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.343+0000 7f2446a00640 1 --2- 172.21.15.121:0/261293949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f242c004d40 0x7f244814da00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f243800e810 tx=0x7f243800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:34.346 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.344+0000 7f243ea00640 1 -- 172.21.15.121:0/261293949 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2438017070 con 0x7f242c004d40 2024-04-20T10:22:34.346 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.344+0000 7f243ea00640 1 -- 172.21.15.121:0/261293949 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f243800c650 con 0x7f242c004d40 2024-04-20T10:22:34.346 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.344+0000 7f244d800640 1 -- 172.21.15.121:0/261293949 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24480aeff0 con 0x7f242c004d40 2024-04-20T10:22:34.346 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.344+0000 7f243ea00640 1 -- 172.21.15.121:0/261293949 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243800f040 con 0x7f242c004d40 2024-04-20T10:22:34.346 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.344+0000 7f244d800640 1 -- 172.21.15.121:0/261293949 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f244814f460 con 0x7f242c004d40 2024-04-20T10:22:34.347 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.345+0000 7f244d800640 1 -- 172.21.15.121:0/261293949 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2448068980 con 0x7f242c004d40 2024-04-20T10:22:34.347 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.346+0000 7f243ea00640 1 -- 172.21.15.121:0/261293949 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f243801a070 con 0x7f242c004d40 2024-04-20T10:22:34.347 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.347+0000 7f243ea00640 1 --2- 172.21.15.121:0/261293949 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f242406c8f0 0x7f242406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:34.347 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.347+0000 7f243ea00640 1 -- 172.21.15.121:0/261293949 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f243808bb80 con 0x7f242c004d40 2024-04-20T10:22:34.348 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.347+0000 7f243fe00640 1 --2- 172.21.15.121:0/261293949 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f242406c8f0 0x7f242406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:34.348 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.348+0000 7f243fe00640 1 --2- 172.21.15.121:0/261293949 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f242406c8f0 0x7f242406edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f24481f7650 tx=0x7f2430008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:34.352 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.351+0000 7f243ea00640 1 -- 172.21.15.121:0/261293949 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2438059ba0 con 0x7f242c004d40 2024-04-20T10:22:34.547 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.546+0000 7f244d800640 1 -- 172.21.15.121:0/261293949 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f244806d160 con 0x7f242c004d40 2024-04-20T10:22:34.548 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.547+0000 7f243ea00640 1 -- 172.21.15.121:0/261293949 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f24380594a0 con 0x7f242c004d40 2024-04-20T10:22:34.548 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2024-04-20T10:22:34.551 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.550+0000 7f244d800640 1 -- 172.21.15.121:0/261293949 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f242406c8f0 msgr2=0x7f242406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:34.551 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.550+0000 7f244d800640 1 --2- 172.21.15.121:0/261293949 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f242406c8f0 0x7f242406edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f24481f7650 tx=0x7f2430008040 comp rx=0 tx=0).stop 2024-04-20T10:22:34.551 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.550+0000 7f244d800640 1 -- 172.21.15.121:0/261293949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f242c004d40 msgr2=0x7f244814da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:34.551 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.550+0000 7f244d800640 1 --2- 172.21.15.121:0/261293949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f242c004d40 0x7f244814da00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f243800e810 tx=0x7f243800ecd0 comp rx=0 tx=0).stop 2024-04-20T10:22:34.551 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.550+0000 7f244d800640 1 -- 172.21.15.121:0/261293949 shutdown_connections 2024-04-20T10:22:34.551 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.550+0000 7f244d800640 1 --2- 172.21.15.121:0/261293949 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f242406c8f0 0x7f242406edb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:34.551 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.550+0000 7f244d800640 1 --2- 172.21.15.121:0/261293949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f244814a780 0x7f24480b0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:34.551 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.550+0000 7f244d800640 1 --2- 172.21.15.121:0/261293949 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f244814efe0 0x7f2448149ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:34.551 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.550+0000 7f244d800640 1 --2- 172.21.15.121:0/261293949 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f242c004d40 0x7f244814da00 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:34.551 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.550+0000 7f244d800640 1 -- 172.21.15.121:0/261293949 >> 172.21.15.121:0/261293949 conn(0x7f24480b3350 msgr2=0x7f24480b1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:34.551 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.550+0000 7f244d800640 1 -- 172.21.15.121:0/261293949 shutdown_connections 2024-04-20T10:22:34.551 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.550+0000 7f244d800640 1 -- 172.21.15.121:0/261293949 wait complete. 2024-04-20T10:22:34.569 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:22:34.766 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.765+0000 7feaffe00640 1 Processor -- start 2024-04-20T10:22:34.766 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.765+0000 7feaffe00640 1 -- start start 2024-04-20T10:22:34.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.767+0000 7feaffe00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7feb0015b1f0 0x7feb0015f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:34.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.767+0000 7feaffe00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb0015b5d0 con 0x7feb000d0010 2024-04-20T10:22:34.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.767+0000 7feaffe00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb0015fc70 con 0x7feb0006ca10 2024-04-20T10:22:34.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.767+0000 7feaffe00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7feb0006e730 con 0x7feb0015b1f0 2024-04-20T10:22:34.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.767+0000 7feafea00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7feb0015b1f0 0x7feb0015f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:34.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.767+0000 7feafea00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7feb0015b1f0 0x7feb0015f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59284/0 (socket says 172.21.15.121:59284) 2024-04-20T10:22:34.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.767+0000 7feafea00640 1 -- 172.21.15.121:0/1179670702 learned_addr learned my addr 172.21.15.121:0/1179670702 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:34.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.767+0000 7feaf7e00640 1 --1- 172.21.15.121:0/1179670702 >> v1:172.21.15.121:6789/0 conn(0x7feb000d0010 0x7feb000adc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:56218/0 (socket says 172.21.15.121:56218) 2024-04-20T10:22:34.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.767+0000 7feafea00640 1 -- 172.21.15.121:0/1179670702 >> v1:172.21.15.123:6789/0 conn(0x7feb0006ca10 legacy=0x7feb0006ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:34.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.767+0000 7feafea00640 1 -- 172.21.15.121:0/1179670702 >> v1:172.21.15.121:6789/0 conn(0x7feb000d0010 legacy=0x7feb000adc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:34.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.768+0000 7feafe000640 1 -- 172.21.15.121:0/1179670702 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1385226321 0 0) 0x7feb0015b5d0 con 0x7feb000d0010 2024-04-20T10:22:34.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.768+0000 7feafea00640 1 -- 172.21.15.121:0/1179670702 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb000ae310 con 0x7feb0015b1f0 2024-04-20T10:22:34.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.769+0000 7feafea00640 1 --2- 172.21.15.121:0/1179670702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb0015b1f0 0x7feb0015f730 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7feaf00098a0 tx=0x7feaf002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3673e2710a8a113 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:34.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.769+0000 7feafe000640 1 -- 172.21.15.121:0/1179670702 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf0035020 con 0x7feb0015b1f0 2024-04-20T10:22:34.770 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.769+0000 7feafe000640 1 -- 172.21.15.121:0/1179670702 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaf0031640 con 0x7feb0015b1f0 2024-04-20T10:22:34.770 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.769+0000 7feafe000640 1 -- 172.21.15.121:0/1179670702 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf002d040 con 0x7feb0015b1f0 2024-04-20T10:22:34.771 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.769+0000 7feaffe00640 1 -- 172.21.15.121:0/1179670702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb0015b1f0 msgr2=0x7feb0015f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:34.771 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.769+0000 7feaffe00640 1 --2- 172.21.15.121:0/1179670702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb0015b1f0 0x7feb0015f730 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7feaf00098a0 tx=0x7feaf002b8f0 comp rx=0 tx=0).stop 2024-04-20T10:22:34.771 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.770+0000 7feaffe00640 1 -- 172.21.15.121:0/1179670702 shutdown_connections 2024-04-20T10:22:34.771 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.770+0000 7feaffe00640 1 --2- 172.21.15.121:0/1179670702 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb0015b1f0 0x7feb0015f730 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:34.771 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.770+0000 7feaffe00640 1 -- 172.21.15.121:0/1179670702 >> 172.21.15.121:0/1179670702 conn(0x7feb000b5270 msgr2=0x7feb000b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:34.771 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.770+0000 7feaffe00640 1 -- 172.21.15.121:0/1179670702 shutdown_connections 2024-04-20T10:22:34.771 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.770+0000 7feaffe00640 1 -- 172.21.15.121:0/1179670702 wait complete. 2024-04-20T10:22:34.772 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.771+0000 7feaffe00640 1 Processor -- start 2024-04-20T10:22:34.772 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.771+0000 7feaffe00640 1 -- start start 2024-04-20T10:22:34.774 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.772+0000 7feaffe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb0006ca10 0x7feb0014e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:34.774 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.772+0000 7feaffe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb000d0010 0x7feb0014efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:34.774 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.772+0000 7feaffe00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb0015b1f0 0x7feb001549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:34.774 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.772+0000 7feaffe00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7feb000d0b90 con 0x7feb000d0010 2024-04-20T10:22:34.775 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.772+0000 7feaffe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feb000d08b0 con 0x7feb0006ca10 2024-04-20T10:22:34.775 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.772+0000 7feaffe00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7feb000d0e70 con 0x7feb0015b1f0 2024-04-20T10:22:34.775 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.773+0000 7feaf7e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb000d0010 0x7feb0014efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:34.775 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.773+0000 7feaf7e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb000d0010 0x7feb0014efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59296/0 (socket says 172.21.15.121:59296) 2024-04-20T10:22:34.775 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.773+0000 7feaf7e00640 1 -- 172.21.15.121:0/3295681297 learned_addr learned my addr 172.21.15.121:0/3295681297 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:34.775 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.773+0000 7feaff400640 1 --2- 172.21.15.121:0/3295681297 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb0015b1f0 0x7feb001549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:34.775 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.773+0000 7feafea00640 1 --2- 172.21.15.121:0/3295681297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb0006ca10 0x7feb0014e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:34.775 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.773+0000 7feaf7e00640 1 -- 172.21.15.121:0/3295681297 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb0015b1f0 msgr2=0x7feb001549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:34.775 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.773+0000 7feaf7e00640 1 --2- 172.21.15.121:0/3295681297 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb0015b1f0 0x7feb001549b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:34.776 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.773+0000 7feaf7e00640 1 -- 172.21.15.121:0/3295681297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb0006ca10 msgr2=0x7feb0014e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:34.776 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.773+0000 7feaf7e00640 1 --2- 172.21.15.121:0/3295681297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb0006ca10 0x7feb0014e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:34.776 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.773+0000 7feaf7e00640 1 -- 172.21.15.121:0/3295681297 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb0014fa30 con 0x7feb000d0010 2024-04-20T10:22:34.776 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.773+0000 7feaff400640 1 --2- 172.21.15.121:0/3295681297 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb0015b1f0 0x7feb001549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:34.776 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.773+0000 7feafea00640 1 --2- 172.21.15.121:0/3295681297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb0006ca10 0x7feb0014e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:34.776 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.773+0000 7feaf7e00640 1 --2- 172.21.15.121:0/3295681297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb000d0010 0x7feb0014efe0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7feae800e780 tx=0x7feae800ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:34.776 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.774+0000 7feaf6a00640 1 -- 172.21.15.121:0/3295681297 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae801a070 con 0x7feb000d0010 2024-04-20T10:22:34.776 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.774+0000 7feaf6a00640 1 -- 172.21.15.121:0/3295681297 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feae8006bf0 con 0x7feb000d0010 2024-04-20T10:22:34.776 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.774+0000 7feaf6a00640 1 -- 172.21.15.121:0/3295681297 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae80090d0 con 0x7feb000d0010 2024-04-20T10:22:34.776 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.774+0000 7feaffe00640 1 -- 172.21.15.121:0/3295681297 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb0014fd10 con 0x7feb000d0010 2024-04-20T10:22:34.776 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.774+0000 7feaffe00640 1 -- 172.21.15.121:0/3295681297 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb001502d0 con 0x7feb000d0010 2024-04-20T10:22:34.777 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.775+0000 7feaffe00640 1 -- 172.21.15.121:0/3295681297 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb00075b30 con 0x7feb000d0010 2024-04-20T10:22:34.777 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.776+0000 7feaf6a00640 1 -- 172.21.15.121:0/3295681297 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feae8009270 con 0x7feb000d0010 2024-04-20T10:22:34.777 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.777+0000 7feaf6a00640 1 --2- 172.21.15.121:0/3295681297 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7feadc06c920 0x7feadc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:34.777 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.777+0000 7feaf6a00640 1 -- 172.21.15.121:0/3295681297 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7feae808d390 con 0x7feb000d0010 2024-04-20T10:22:34.777 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.777+0000 7feafea00640 1 --2- 172.21.15.121:0/3295681297 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7feadc06c920 0x7feadc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:34.778 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.778+0000 7feafea00640 1 --2- 172.21.15.121:0/3295681297 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7feadc06c920 0x7feadc06ede0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7feaf0036000 tx=0x7feaf003c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:34.782 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.781+0000 7feaf6a00640 1 -- 172.21.15.121:0/3295681297 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7feae805b3b0 con 0x7feb000d0010 2024-04-20T10:22:34.976 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.975+0000 7feaffe00640 1 -- 172.21.15.121:0/3295681297 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7feb000af060 con 0x7feb000d0010 2024-04-20T10:22:34.977 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.977+0000 7feaf6a00640 1 -- 172.21.15.121:0/3295681297 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7feae805acb0 con 0x7feb000d0010 2024-04-20T10:22:34.977 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 32 2024-04-20T10:22:34.980 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.979+0000 7feaffe00640 1 -- 172.21.15.121:0/3295681297 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7feadc06c920 msgr2=0x7feadc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:34.980 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.979+0000 7feaffe00640 1 --2- 172.21.15.121:0/3295681297 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7feadc06c920 0x7feadc06ede0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7feaf0036000 tx=0x7feaf003c040 comp rx=0 tx=0).stop 2024-04-20T10:22:34.980 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.979+0000 7feaffe00640 1 -- 172.21.15.121:0/3295681297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb000d0010 msgr2=0x7feb0014efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:34.980 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.979+0000 7feaffe00640 1 --2- 172.21.15.121:0/3295681297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb000d0010 0x7feb0014efe0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7feae800e780 tx=0x7feae800ec40 comp rx=0 tx=0).stop 2024-04-20T10:22:34.980 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.979+0000 7feaffe00640 1 -- 172.21.15.121:0/3295681297 shutdown_connections 2024-04-20T10:22:34.980 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.979+0000 7feaffe00640 1 --2- 172.21.15.121:0/3295681297 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7feb0015b1f0 0x7feb001549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:34.980 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.979+0000 7feaffe00640 1 --2- 172.21.15.121:0/3295681297 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7feb000d0010 0x7feb0014efe0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:34.980 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.979+0000 7feaffe00640 1 --2- 172.21.15.121:0/3295681297 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7feadc06c920 0x7feadc06ede0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:34.980 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.979+0000 7feaffe00640 1 --2- 172.21.15.121:0/3295681297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb0006ca10 0x7feb0014e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:34.980 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.979+0000 7feaffe00640 1 -- 172.21.15.121:0/3295681297 >> 172.21.15.121:0/3295681297 conn(0x7feb000b5270 msgr2=0x7feb001561f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:34.980 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.979+0000 7feaffe00640 1 -- 172.21.15.121:0/3295681297 shutdown_connections 2024-04-20T10:22:34.980 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:34.979+0000 7feaffe00640 1 -- 172.21.15.121:0/3295681297 wait complete. 2024-04-20T10:22:34.997 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:22:35.191 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.190+0000 7fbef1800640 1 Processor -- start 2024-04-20T10:22:35.191 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.191+0000 7fbef1800640 1 -- start start 2024-04-20T10:22:35.193 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.192+0000 7fbef1800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fbeec0d4270 0x7fbeec0d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:35.193 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.192+0000 7fbef1800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbeec0d4650 0x7fbeec0b1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:35.193 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.192+0000 7fbef1800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbeec0d3820 con 0x7fbeec0b17d0 2024-04-20T10:22:35.193 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.192+0000 7fbef1800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fbeec0d40c0 con 0x7fbeec0d4270 2024-04-20T10:22:35.193 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.192+0000 7fbef1800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fbeec0d3b00 con 0x7fbeec0d4650 2024-04-20T10:22:35.193 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.192+0000 7fbeeaa00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fbeec0d4270 0x7fbeec0d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:35.193 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.192+0000 7fbeeb400640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fbeec0b17d0 0x7fbeec0d0380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:56234/0 (socket says 172.21.15.121:56234) 2024-04-20T10:22:35.193 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.192+0000 7fbeeb400640 1 -- 172.21.15.121:0/2078929119 learned_addr learned my addr 172.21.15.121:0/2078929119 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:35.194 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.193+0000 7fbee2000640 1 --2- 172.21.15.121:0/2078929119 >> v2:172.21.15.123:3300/0 conn(0x7fbeec0d4650 0x7fbeec0b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:35.194 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.193+0000 7fbeeaa00640 1 -- 172.21.15.121:0/2078929119 >> v2:172.21.15.123:3300/0 conn(0x7fbeec0d4650 msgr2=0x7fbeec0b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:35.194 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.193+0000 7fbeeaa00640 1 --2- 172.21.15.121:0/2078929119 >> v2:172.21.15.123:3300/0 conn(0x7fbeec0d4650 0x7fbeec0b1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:35.194 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.193+0000 7fbeeaa00640 1 -- 172.21.15.121:0/2078929119 >> v1:172.21.15.121:6789/0 conn(0x7fbeec0b17d0 legacy=0x7fbeec0d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:35.194 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.193+0000 7fbeeaa00640 1 -- 172.21.15.121:0/2078929119 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeec0d3de0 con 0x7fbeec0d4270 2024-04-20T10:22:35.195 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.194+0000 7fbeeaa00640 1 --2- 172.21.15.121:0/2078929119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbeec0d4270 0x7fbeec0d1a00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fbeec0b8060 tx=0x7fbedc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9982fd1b6764a575 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:35.195 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.194+0000 7fbeea000640 1 -- 172.21.15.121:0/2078929119 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc035020 con 0x7fbeec0d4270 2024-04-20T10:22:35.195 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.194+0000 7fbeea000640 1 -- 172.21.15.121:0/2078929119 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbedc031830 con 0x7fbeec0d4270 2024-04-20T10:22:35.195 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.194+0000 7fbeea000640 1 -- 172.21.15.121:0/2078929119 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc031c00 con 0x7fbeec0d4270 2024-04-20T10:22:35.196 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.195+0000 7fbef1800640 1 -- 172.21.15.121:0/2078929119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbeec0d4270 msgr2=0x7fbeec0d1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:35.196 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.195+0000 7fbef1800640 1 --2- 172.21.15.121:0/2078929119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbeec0d4270 0x7fbeec0d1a00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fbeec0b8060 tx=0x7fbedc02b8d0 comp rx=0 tx=0).stop 2024-04-20T10:22:35.196 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.195+0000 7fbef1800640 1 -- 172.21.15.121:0/2078929119 shutdown_connections 2024-04-20T10:22:35.196 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.195+0000 7fbef1800640 1 --2- 172.21.15.121:0/2078929119 >> v2:172.21.15.123:3300/0 conn(0x7fbeec0d4650 0x7fbeec0b1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:35.196 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.195+0000 7fbef1800640 1 --2- 172.21.15.121:0/2078929119 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbeec0d4270 0x7fbeec0d1a00 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:35.196 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.196+0000 7fbef1800640 1 -- 172.21.15.121:0/2078929119 >> 172.21.15.121:0/2078929119 conn(0x7fbeec0b99c0 msgr2=0x7fbeec0b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:35.196 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.196+0000 7fbef1800640 1 -- 172.21.15.121:0/2078929119 shutdown_connections 2024-04-20T10:22:35.196 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.196+0000 7fbef1800640 1 -- 172.21.15.121:0/2078929119 wait complete. 2024-04-20T10:22:35.198 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.198+0000 7fbef1800640 1 Processor -- start 2024-04-20T10:22:35.198 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.198+0000 7fbef1800640 1 -- start start 2024-04-20T10:22:35.201 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.199+0000 7fbef1800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbeec0b17d0 0x7fbeec1f7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:35.201 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.199+0000 7fbef1800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbeec0d4270 0x7fbeec0ce460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:35.201 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.199+0000 7fbef1800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbeec0d4650 0x7fbeec15b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:35.201 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.199+0000 7fbef1800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec0d4ae0 con 0x7fbeec0d4270 2024-04-20T10:22:35.201 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.199+0000 7fbef1800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec0d53b0 con 0x7fbeec0d4650 2024-04-20T10:22:35.201 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.199+0000 7fbef1800640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fbeec0d4c60 con 0x7fbeec0b17d0 2024-04-20T10:22:35.201 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.200+0000 7fbee2000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbeec0d4270 0x7fbeec0ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:35.201 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.200+0000 7fbee2000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbeec0d4270 0x7fbeec0ce460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59306/0 (socket says 172.21.15.121:59306) 2024-04-20T10:22:35.201 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.200+0000 7fbee2000640 1 -- 172.21.15.121:0/1011406880 learned_addr learned my addr 172.21.15.121:0/1011406880 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:35.201 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.200+0000 7fbeeaa00640 1 --2- 172.21.15.121:0/1011406880 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbeec0b17d0 0x7fbeec1f7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:35.202 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.200+0000 7fbeeb400640 1 --2- 172.21.15.121:0/1011406880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbeec0d4650 0x7fbeec15b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:35.202 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.200+0000 7fbee2000640 1 -- 172.21.15.121:0/1011406880 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbeec0b17d0 msgr2=0x7fbeec1f7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:35.202 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.200+0000 7fbee2000640 1 --2- 172.21.15.121:0/1011406880 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbeec0b17d0 0x7fbeec1f7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:35.202 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.200+0000 7fbee2000640 1 -- 172.21.15.121:0/1011406880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbeec0d4650 msgr2=0x7fbeec15b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:35.202 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.200+0000 7fbee2000640 1 --2- 172.21.15.121:0/1011406880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbeec0d4650 0x7fbeec15b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:35.202 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.200+0000 7fbee2000640 1 -- 172.21.15.121:0/1011406880 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeec0ce9a0 con 0x7fbeec0d4270 2024-04-20T10:22:35.202 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.201+0000 7fbee2000640 1 --2- 172.21.15.121:0/1011406880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbeec0d4270 0x7fbeec0ce460 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fbed4007e70 tx=0x7fbed400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:35.203 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.201+0000 7fbee3400640 1 -- 172.21.15.121:0/1011406880 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed4014620 con 0x7fbeec0d4270 2024-04-20T10:22:35.203 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.201+0000 7fbee3400640 1 -- 172.21.15.121:0/1011406880 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbed4019070 con 0x7fbeec0d4270 2024-04-20T10:22:35.203 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.201+0000 7fbef1800640 1 -- 172.21.15.121:0/1011406880 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeec1f8310 con 0x7fbeec0d4270 2024-04-20T10:22:35.203 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.201+0000 7fbee3400640 1 -- 172.21.15.121:0/1011406880 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed4014a00 con 0x7fbeec0d4270 2024-04-20T10:22:35.203 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.201+0000 7fbef1800640 1 -- 172.21.15.121:0/1011406880 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeec1f88d0 con 0x7fbeec0d4270 2024-04-20T10:22:35.203 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.202+0000 7fbef1800640 1 -- 172.21.15.121:0/1011406880 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeec068980 con 0x7fbeec0d4270 2024-04-20T10:22:35.204 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.203+0000 7fbee3400640 1 -- 172.21.15.121:0/1011406880 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbed4014ba0 con 0x7fbeec0d4270 2024-04-20T10:22:35.205 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.204+0000 7fbee3400640 1 --2- 172.21.15.121:0/1011406880 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fbec806c8f0 0x7fbec806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:35.205 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.204+0000 7fbee3400640 1 -- 172.21.15.121:0/1011406880 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fbed408c770 con 0x7fbeec0d4270 2024-04-20T10:22:35.205 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.204+0000 7fbeeaa00640 1 --2- 172.21.15.121:0/1011406880 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fbec806c8f0 0x7fbec806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:35.205 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.205+0000 7fbeeaa00640 1 --2- 172.21.15.121:0/1011406880 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fbec806c8f0 0x7fbec806edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbedc036000 tx=0x7fbedc008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:35.210 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.210+0000 7fbee3400640 1 -- 172.21.15.121:0/1011406880 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbeec0ce9a0 con 0x7fbeec0d4270 2024-04-20T10:22:35.410 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.409+0000 7fbef1800640 1 -- 172.21.15.121:0/1011406880 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbeec160d90 con 0x7fbeec0d4270 2024-04-20T10:22:35.410 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.410+0000 7fbee3400640 1 -- 172.21.15.121:0/1011406880 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fbed405a830 con 0x7fbeec0d4270 2024-04-20T10:22:35.411 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 64 2024-04-20T10:22:35.414 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.413+0000 7fbef1800640 1 -- 172.21.15.121:0/1011406880 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fbec806c8f0 msgr2=0x7fbec806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:35.414 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.413+0000 7fbef1800640 1 --2- 172.21.15.121:0/1011406880 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fbec806c8f0 0x7fbec806edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbedc036000 tx=0x7fbedc008030 comp rx=0 tx=0).stop 2024-04-20T10:22:35.414 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.413+0000 7fbef1800640 1 -- 172.21.15.121:0/1011406880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbeec0d4270 msgr2=0x7fbeec0ce460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:35.414 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.413+0000 7fbef1800640 1 --2- 172.21.15.121:0/1011406880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbeec0d4270 0x7fbeec0ce460 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fbed4007e70 tx=0x7fbed400e4b0 comp rx=0 tx=0).stop 2024-04-20T10:22:35.414 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.413+0000 7fbef1800640 1 -- 172.21.15.121:0/1011406880 shutdown_connections 2024-04-20T10:22:35.414 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.413+0000 7fbef1800640 1 --2- 172.21.15.121:0/1011406880 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fbec806c8f0 0x7fbec806edb0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:35.414 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.413+0000 7fbef1800640 1 --2- 172.21.15.121:0/1011406880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbeec0d4650 0x7fbeec15b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:35.414 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.413+0000 7fbef1800640 1 --2- 172.21.15.121:0/1011406880 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fbeec0d4270 0x7fbeec0ce460 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:35.414 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.413+0000 7fbef1800640 1 --2- 172.21.15.121:0/1011406880 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fbeec0b17d0 0x7fbeec1f7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:35.414 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.413+0000 7fbef1800640 1 -- 172.21.15.121:0/1011406880 >> 172.21.15.121:0/1011406880 conn(0x7fbeec0b99c0 msgr2=0x7fbeec15f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:35.415 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.413+0000 7fbef1800640 1 -- 172.21.15.121:0/1011406880 shutdown_connections 2024-04-20T10:22:35.415 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.413+0000 7fbef1800640 1 -- 172.21.15.121:0/1011406880 wait complete. 2024-04-20T10:22:35.432 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T10:22:35.630 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.630+0000 7f9333200640 1 Processor -- start 2024-04-20T10:22:35.630 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.630+0000 7f9333200640 1 -- start start 2024-04-20T10:22:35.632 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.631+0000 7f9333200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f933406ca10 0x7f933406cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:35.632 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.631+0000 7f9333200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f933415b5d0 con 0x7f933415b1f0 2024-04-20T10:22:35.632 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.631+0000 7f9333200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f933406e730 con 0x7f93340d0010 2024-04-20T10:22:35.632 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.631+0000 7f9333200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f93340ae310 con 0x7f933406ca10 2024-04-20T10:22:35.632 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.632+0000 7f9331e00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f933415b1f0 0x7f933415f730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:56238/0 (socket says 172.21.15.121:56238) 2024-04-20T10:22:35.632 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.632+0000 7f9331e00640 1 -- 172.21.15.121:0/2776078375 learned_addr learned my addr 172.21.15.121:0/2776078375 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:35.634 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.632+0000 7f9332800640 1 --2- 172.21.15.121:0/2776078375 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f933406ca10 0x7f933406cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:35.634 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.632+0000 7f9330a00640 1 -- 172.21.15.121:0/2776078375 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3233330713 0 0) 0x7f933415b5d0 con 0x7f933415b1f0 2024-04-20T10:22:35.635 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.632+0000 7f9330a00640 1 -- 172.21.15.121:0/2776078375 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9318003520 con 0x7f933415b1f0 2024-04-20T10:22:35.635 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.633+0000 7f9330a00640 1 -- 172.21.15.121:0/2776078375 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1694567962 0 0) 0x7f9318003520 con 0x7f933415b1f0 2024-04-20T10:22:35.635 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.633+0000 7f9330a00640 1 -- 172.21.15.121:0/2776078375 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f933406ca10 msgr2=0x7f933406cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:35.635 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.633+0000 7f9330a00640 1 --2- 172.21.15.121:0/2776078375 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f933406ca10 0x7f933406cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:35.635 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.633+0000 7f9330a00640 1 -- 172.21.15.121:0/2776078375 >> v1:172.21.15.123:6789/0 conn(0x7f93340d0010 legacy=0x7f93340adc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:35.635 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.633+0000 7f9330a00640 1 -- 172.21.15.121:0/2776078375 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93340d05d0 con 0x7f933415b1f0 2024-04-20T10:22:35.635 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.633+0000 7f9330a00640 1 -- 172.21.15.121:0/2776078375 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f9324001670 con 0x7f933415b1f0 2024-04-20T10:22:35.635 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.633+0000 7f9330a00640 1 -- 172.21.15.121:0/2776078375 >> v1:172.21.15.121:6789/0 conn(0x7f933415b1f0 legacy=0x7f933415f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:35.635 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.633+0000 7f9330a00640 1 --2- 172.21.15.121:0/2776078375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93180039e0 0x7f9318003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:35.635 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.633+0000 7f9330a00640 1 --2- 172.21.15.121:0/2776078375 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f933406ca10 0x7f9318004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:35.635 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.633+0000 7f9330a00640 1 --2- 172.21.15.121:0/2776078375 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9318004d40 0x7f93180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:35.635 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.633+0000 7f9330a00640 1 -- 172.21.15.121:0/2776078375 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f93340ae310 con 0x7f9318004d40 2024-04-20T10:22:35.635 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.633+0000 7f9330a00640 1 -- 172.21.15.121:0/2776078375 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f931800baf0 con 0x7f93180039e0 2024-04-20T10:22:35.636 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.633+0000 7f9330a00640 1 -- 172.21.15.121:0/2776078375 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f931800bcf0 con 0x7f933406ca10 2024-04-20T10:22:35.636 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.633+0000 7f9332800640 1 --2- 172.21.15.121:0/2776078375 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9318004d40 0x7f93180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:35.636 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.633+0000 7f9331400640 1 --2- 172.21.15.121:0/2776078375 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f933406ca10 0x7f9318004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:35.636 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.633+0000 7f9331e00640 1 --2- 172.21.15.121:0/2776078375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93180039e0 0x7f9318003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:35.636 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.633+0000 7f9332800640 1 -- 172.21.15.121:0/2776078375 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f933406ca10 msgr2=0x7f9318004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:35.636 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.633+0000 7f9332800640 1 --2- 172.21.15.121:0/2776078375 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f933406ca10 0x7f9318004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:35.636 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.634+0000 7f9332800640 1 -- 172.21.15.121:0/2776078375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93180039e0 msgr2=0x7f9318003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:35.636 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.634+0000 7f9332800640 1 --2- 172.21.15.121:0/2776078375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93180039e0 0x7f9318003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:35.636 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.634+0000 7f9332800640 1 -- 172.21.15.121:0/2776078375 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9318003520 con 0x7f9318004d40 2024-04-20T10:22:35.636 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.634+0000 7f9331400640 1 --2- 172.21.15.121:0/2776078375 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f933406ca10 0x7f9318004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:35.636 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.634+0000 7f9331e00640 1 --2- 172.21.15.121:0/2776078375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93180039e0 0x7f9318003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:35.637 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.635+0000 7f9332800640 1 --2- 172.21.15.121:0/2776078375 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9318004d40 0x7f93180051c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f932c009fc0 tx=0x7f9318005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3773f2b460b00a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:35.638 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.635+0000 7f9330a00640 1 -- 172.21.15.121:0/2776078375 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932c039070 con 0x7f9318004d40 2024-04-20T10:22:35.638 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.635+0000 7f9330a00640 1 -- 172.21.15.121:0/2776078375 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f932c02cc20 con 0x7f9318004d40 2024-04-20T10:22:35.638 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.635+0000 7f9330a00640 1 -- 172.21.15.121:0/2776078375 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932c009c10 con 0x7f9318004d40 2024-04-20T10:22:35.638 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.636+0000 7f9333200640 1 -- 172.21.15.121:0/2776078375 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9318004d40 msgr2=0x7f93180051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:35.638 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.636+0000 7f9333200640 1 --2- 172.21.15.121:0/2776078375 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9318004d40 0x7f93180051c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f932c009fc0 tx=0x7f9318005830 comp rx=0 tx=0).stop 2024-04-20T10:22:35.638 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.636+0000 7f9332800640 1 -- 172.21.15.121:0/2776078375 reap_dead start 2024-04-20T10:22:35.638 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.636+0000 7f9333200640 1 -- 172.21.15.121:0/2776078375 shutdown_connections 2024-04-20T10:22:35.638 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.636+0000 7f9333200640 1 -- 172.21.15.121:0/2776078375 >> 172.21.15.121:0/2776078375 conn(0x7f93340b5270 msgr2=0x7f93340b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:35.638 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.636+0000 7f9333200640 1 -- 172.21.15.121:0/2776078375 shutdown_connections 2024-04-20T10:22:35.638 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.636+0000 7f9333200640 1 -- 172.21.15.121:0/2776078375 wait complete. 2024-04-20T10:22:35.639 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.638+0000 7f9333200640 1 Processor -- start 2024-04-20T10:22:35.639 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.638+0000 7f9333200640 1 -- start start 2024-04-20T10:22:35.641 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.639+0000 7f9333200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9318004d40 0x7f93341517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:35.641 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.639+0000 7f9333200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9334149150 0x7f933414ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:35.641 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.639+0000 7f9333200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9334147500 0x7f933414c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:35.641 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.639+0000 7f9333200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f93340d0e70 con 0x7f9334149150 2024-04-20T10:22:35.641 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.639+0000 7f9333200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f93340d0b90 con 0x7f9334147500 2024-04-20T10:22:35.641 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.639+0000 7f9333200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f933406cea0 con 0x7f9318004d40 2024-04-20T10:22:35.641 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.640+0000 7f9331400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9334149150 0x7f933414ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:35.641 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.640+0000 7f9331400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9334149150 0x7f933414ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59332/0 (socket says 172.21.15.121:59332) 2024-04-20T10:22:35.641 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.640+0000 7f9331400640 1 -- 172.21.15.121:0/1388261089 learned_addr learned my addr 172.21.15.121:0/1388261089 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:35.641 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.640+0000 7f9331400640 1 -- 172.21.15.121:0/1388261089 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9318004d40 msgr2=0x7f93341517f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:35.641 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.640+0000 7f9331400640 1 --2- 172.21.15.121:0/1388261089 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9318004d40 0x7f93341517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:35.641 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.640+0000 7f9331400640 1 -- 172.21.15.121:0/1388261089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9334147500 msgr2=0x7f933414c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:35.641 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.640+0000 7f9331400640 1 --2- 172.21.15.121:0/1388261089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9334147500 0x7f933414c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:35.642 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.640+0000 7f9331400640 1 -- 172.21.15.121:0/1388261089 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93341504b0 con 0x7f9334149150 2024-04-20T10:22:35.642 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.640+0000 7f9331400640 1 --2- 172.21.15.121:0/1388261089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9334149150 0x7f933414ff70 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f931c00d7a0 tx=0x7f931c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:35.642 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.641+0000 7f932a000640 1 -- 172.21.15.121:0/1388261089 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c005270 con 0x7f9334149150 2024-04-20T10:22:35.643 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.641+0000 7f932a000640 1 -- 172.21.15.121:0/1388261089 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f931c017070 con 0x7f9334149150 2024-04-20T10:22:35.643 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.641+0000 7f932a000640 1 -- 172.21.15.121:0/1388261089 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c00b630 con 0x7f9334149150 2024-04-20T10:22:35.643 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.641+0000 7f9333200640 1 -- 172.21.15.121:0/1388261089 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9334150750 con 0x7f9334149150 2024-04-20T10:22:35.643 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.641+0000 7f9333200640 1 -- 172.21.15.121:0/1388261089 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9334150d10 con 0x7f9334149150 2024-04-20T10:22:35.643 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.642+0000 7f9333200640 1 -- 172.21.15.121:0/1388261089 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9334075b30 con 0x7f9334149150 2024-04-20T10:22:35.643 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.643+0000 7f932a000640 1 -- 172.21.15.121:0/1388261089 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f931c00b7d0 con 0x7f9334149150 2024-04-20T10:22:35.644 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.644+0000 7f932a000640 1 --2- 172.21.15.121:0/1388261089 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f931006c920 0x7f931006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:35.644 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.644+0000 7f932a000640 1 -- 172.21.15.121:0/1388261089 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f931c08be20 con 0x7f9334149150 2024-04-20T10:22:35.645 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.644+0000 7f9331e00640 1 --2- 172.21.15.121:0/1388261089 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f931006c920 0x7f931006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:35.645 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.645+0000 7f9331e00640 1 --2- 172.21.15.121:0/1388261089 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f931006c920 0x7f931006ede0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f93240017e0 tx=0x7f9324009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:35.649 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.648+0000 7f932a000640 1 -- 172.21.15.121:0/1388261089 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f931c059db0 con 0x7f9334149150 2024-04-20T10:22:35.840 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.840+0000 7f9333200640 1 -- 172.21.15.121:0/1388261089 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f9334072860 con 0x7f9334149150 2024-04-20T10:22:35.841 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.841+0000 7f932a000640 1 -- 172.21.15.121:0/1388261089 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f931c001230 con 0x7f9334149150 2024-04-20T10:22:35.842 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 32 2024-04-20T10:22:35.845 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.843+0000 7f9333200640 1 -- 172.21.15.121:0/1388261089 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f931006c920 msgr2=0x7f931006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:35.845 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.843+0000 7f9333200640 1 --2- 172.21.15.121:0/1388261089 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f931006c920 0x7f931006ede0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f93240017e0 tx=0x7f9324009040 comp rx=0 tx=0).stop 2024-04-20T10:22:35.845 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.843+0000 7f9333200640 1 -- 172.21.15.121:0/1388261089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9334149150 msgr2=0x7f933414ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:35.845 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.843+0000 7f9333200640 1 --2- 172.21.15.121:0/1388261089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9334149150 0x7f933414ff70 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f931c00d7a0 tx=0x7f931c00dc60 comp rx=0 tx=0).stop 2024-04-20T10:22:35.845 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.844+0000 7f9333200640 1 -- 172.21.15.121:0/1388261089 shutdown_connections 2024-04-20T10:22:35.845 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.844+0000 7f9333200640 1 --2- 172.21.15.121:0/1388261089 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f931006c920 0x7f931006ede0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:35.845 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.844+0000 7f9333200640 1 --2- 172.21.15.121:0/1388261089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9334147500 0x7f933414c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:35.845 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.844+0000 7f9333200640 1 --2- 172.21.15.121:0/1388261089 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f9334149150 0x7f933414ff70 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:35.845 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.844+0000 7f9333200640 1 --2- 172.21.15.121:0/1388261089 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f9318004d40 0x7f93341517f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:35.845 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.844+0000 7f9333200640 1 -- 172.21.15.121:0/1388261089 >> 172.21.15.121:0/1388261089 conn(0x7f93340b5270 msgr2=0x7f93340b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:35.845 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.844+0000 7f9333200640 1 -- 172.21.15.121:0/1388261089 shutdown_connections 2024-04-20T10:22:35.845 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:35.844+0000 7f9333200640 1 -- 172.21.15.121:0/1388261089 wait complete. 2024-04-20T10:22:35.863 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:22:36.057 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.056+0000 7fcc01200640 1 Processor -- start 2024-04-20T10:22:36.057 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.056+0000 7fcc01200640 1 -- start start 2024-04-20T10:22:36.059 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.058+0000 7fcc01200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fcbfc0b2f70 0x7fcbfc0b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:36.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.058+0000 7fcc01200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcbfc06b900 0x7fcbfc06bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:36.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.058+0000 7fcc01200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbfc071a80 con 0x7fcbfc0adf60 2024-04-20T10:22:36.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.058+0000 7fcc01200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fcbfc0ae3f0 con 0x7fcbfc0b2f70 2024-04-20T10:22:36.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.058+0000 7fcc01200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fcbfc06c240 con 0x7fcbfc06b900 2024-04-20T10:22:36.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.058+0000 7fcbfaa00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fcbfc0adf60 0x7fcbfc15f5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:56244/0 (socket says 172.21.15.121:56244) 2024-04-20T10:22:36.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.058+0000 7fcbfaa00640 1 -- 172.21.15.121:0/2699885877 learned_addr learned my addr 172.21.15.121:0/2699885877 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:36.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.058+0000 7fcbfb400640 1 --2- 172.21.15.121:0/2699885877 >> v2:172.21.15.121:3300/0 conn(0x7fcbfc0b2f70 0x7fcbfc0b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:36.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.059+0000 7fcbfbe00640 1 --2- 172.21.15.121:0/2699885877 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcbfc06b900 0x7fcbfc06bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:36.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.059+0000 7fcbfa000640 1 -- 172.21.15.121:0/2699885877 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1005890721 0 0) 0x7fcbfc071a80 con 0x7fcbfc0adf60 2024-04-20T10:22:36.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.059+0000 7fcbfa000640 1 -- 172.21.15.121:0/2699885877 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcbe0003520 con 0x7fcbfc0adf60 2024-04-20T10:22:36.060 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.059+0000 7fcbfb400640 1 -- 172.21.15.121:0/2699885877 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcbfc06b900 msgr2=0x7fcbfc06bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:36.061 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.059+0000 7fcbfb400640 1 --2- 172.21.15.121:0/2699885877 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcbfc06b900 0x7fcbfc06bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:36.061 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.059+0000 7fcbfa000640 1 -- 172.21.15.121:0/2699885877 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2773594757 0 0) 0x7fcbe0003520 con 0x7fcbfc0adf60 2024-04-20T10:22:36.061 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.059+0000 7fcbfb400640 1 -- 172.21.15.121:0/2699885877 >> v1:172.21.15.121:6789/0 conn(0x7fcbfc0adf60 legacy=0x7fcbfc15f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:36.061 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.059+0000 7fcbfb400640 1 -- 172.21.15.121:0/2699885877 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbfc071cf0 con 0x7fcbfc0b2f70 2024-04-20T10:22:36.062 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.060+0000 7fcbfb400640 1 --2- 172.21.15.121:0/2699885877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbfc0b2f70 0x7fcbfc0b4320 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fcbec0097a0 tx=0x7fcbec02b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=496c64013082bd33 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:36.062 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.060+0000 7fcbfa000640 1 -- 172.21.15.121:0/2699885877 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbec037070 con 0x7fcbfc0b2f70 2024-04-20T10:22:36.062 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.060+0000 7fcbfa000640 1 -- 172.21.15.121:0/2699885877 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbec030510 con 0x7fcbfc0b2f70 2024-04-20T10:22:36.062 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.060+0000 7fcbfa000640 1 -- 172.21.15.121:0/2699885877 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8001670 con 0x7fcbfc0b2f70 2024-04-20T10:22:36.063 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.061+0000 7fcc01200640 1 -- 172.21.15.121:0/2699885877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbfc0b2f70 msgr2=0x7fcbfc0b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:36.063 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.061+0000 7fcc01200640 1 --2- 172.21.15.121:0/2699885877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbfc0b2f70 0x7fcbfc0b4320 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fcbec0097a0 tx=0x7fcbec02b790 comp rx=0 tx=0).stop 2024-04-20T10:22:36.063 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.061+0000 7fcc01200640 1 -- 172.21.15.121:0/2699885877 shutdown_connections 2024-04-20T10:22:36.063 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.061+0000 7fcc01200640 1 --2- 172.21.15.121:0/2699885877 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcbfc06b900 0x7fcbfc06bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:36.063 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.061+0000 7fcc01200640 1 --2- 172.21.15.121:0/2699885877 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbfc0b2f70 0x7fcbfc0b4320 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:36.063 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.061+0000 7fcc01200640 1 -- 172.21.15.121:0/2699885877 >> 172.21.15.121:0/2699885877 conn(0x7fcbfc0d0040 msgr2=0x7fcbfc0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:36.063 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.061+0000 7fcc01200640 1 -- 172.21.15.121:0/2699885877 shutdown_connections 2024-04-20T10:22:36.063 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.061+0000 7fcc01200640 1 -- 172.21.15.121:0/2699885877 wait complete. 2024-04-20T10:22:36.064 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.063+0000 7fcc01200640 1 Processor -- start 2024-04-20T10:22:36.064 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.063+0000 7fcc01200640 1 -- start start 2024-04-20T10:22:36.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.064+0000 7fcc01200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcbfc06b900 0x7fcbfc152a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:36.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.064+0000 7fcc01200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbfc0adf60 0x7fcbfc0cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:36.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.064+0000 7fcc01200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbfc0b2f70 0x7fcbfc0c9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:36.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.064+0000 7fcc01200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fcbfc0736c0 con 0x7fcbfc0b2f70 2024-04-20T10:22:36.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.064+0000 7fcc01200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcbfc0739c0 con 0x7fcbfc0adf60 2024-04-20T10:22:36.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.064+0000 7fcc01200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fcbfc15fcc0 con 0x7fcbfc06b900 2024-04-20T10:22:36.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.064+0000 7fcbfbe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbfc0b2f70 0x7fcbfc0c9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:36.066 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.064+0000 7fcbfbe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbfc0b2f70 0x7fcbfc0c9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:59344/0 (socket says 172.21.15.121:59344) 2024-04-20T10:22:36.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.064+0000 7fcbfbe00640 1 -- 172.21.15.121:0/1436121769 learned_addr learned my addr 172.21.15.121:0/1436121769 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:36.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.065+0000 7fcbfb400640 1 --2- 172.21.15.121:0/1436121769 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcbfc06b900 0x7fcbfc152a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:36.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.065+0000 7fcbfaa00640 1 --2- 172.21.15.121:0/1436121769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbfc0adf60 0x7fcbfc0cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:36.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.065+0000 7fcbfbe00640 1 -- 172.21.15.121:0/1436121769 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcbfc06b900 msgr2=0x7fcbfc152a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:36.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.065+0000 7fcbfbe00640 1 --2- 172.21.15.121:0/1436121769 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcbfc06b900 0x7fcbfc152a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:36.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.065+0000 7fcbfbe00640 1 -- 172.21.15.121:0/1436121769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbfc0adf60 msgr2=0x7fcbfc0cefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:36.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.065+0000 7fcbfbe00640 1 --2- 172.21.15.121:0/1436121769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbfc0adf60 0x7fcbfc0cefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:36.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.065+0000 7fcbfbe00640 1 -- 172.21.15.121:0/1436121769 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbfc0cdf00 con 0x7fcbfc0b2f70 2024-04-20T10:22:36.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.065+0000 7fcbfbe00640 1 --2- 172.21.15.121:0/1436121769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbfc0b2f70 0x7fcbfc0c9330 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fcbf400efc0 tx=0x7fcbf400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:36.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.065+0000 7fcbf3400640 1 -- 172.21.15.121:0/1436121769 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf4018020 con 0x7fcbfc0b2f70 2024-04-20T10:22:36.067 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.065+0000 7fcbf3400640 1 -- 172.21.15.121:0/1436121769 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbf4014480 con 0x7fcbfc0b2f70 2024-04-20T10:22:36.068 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.065+0000 7fcbf3400640 1 -- 172.21.15.121:0/1436121769 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf4014870 con 0x7fcbfc0b2f70 2024-04-20T10:22:36.068 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.065+0000 7fcc01200640 1 -- 172.21.15.121:0/1436121769 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbfc0ce1e0 con 0x7fcbfc0b2f70 2024-04-20T10:22:36.068 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.066+0000 7fcc01200640 1 -- 172.21.15.121:0/1436121769 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbfc0ce7a0 con 0x7fcbfc0b2f70 2024-04-20T10:22:36.068 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.067+0000 7fcc01200640 1 -- 172.21.15.121:0/1436121769 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbfc075b60 con 0x7fcbfc0b2f70 2024-04-20T10:22:36.068 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.068+0000 7fcbf3400640 1 -- 172.21.15.121:0/1436121769 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcbf4005d10 con 0x7fcbfc0b2f70 2024-04-20T10:22:36.069 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.069+0000 7fcbf3400640 1 --2- 172.21.15.121:0/1436121769 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fcbdc06c8f0 0x7fcbdc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:36.069 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.069+0000 7fcbf3400640 1 -- 172.21.15.121:0/1436121769 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fcbf408cb50 con 0x7fcbfc0b2f70 2024-04-20T10:22:36.070 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.069+0000 7fcbfb400640 1 --2- 172.21.15.121:0/1436121769 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fcbdc06c8f0 0x7fcbdc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:36.070 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.070+0000 7fcbfb400640 1 --2- 172.21.15.121:0/1436121769 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fcbdc06c8f0 0x7fcbdc06edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcbec034000 tx=0x7fcbec02f000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:36.073 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.073+0000 7fcbf3400640 1 -- 172.21.15.121:0/1436121769 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fcbf4022070 con 0x7fcbfc0b2f70 2024-04-20T10:22:36.296 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.296+0000 7fcc01200640 1 -- 172.21.15.121:0/1436121769 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcbfc06d220 con 0x7fcbfc0b2f70 2024-04-20T10:22:36.297 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.297+0000 7fcbf3400640 1 -- 172.21.15.121:0/1436121769 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5204 (secure 0 0 0) 0x7fcbf4010040 con 0x7fcbfc0b2f70 2024-04-20T10:22:36.297 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:22:36.297 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1376743826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1376743826},{"type":"v1","addr":"172.21.15.123:6833","nonce":1376743826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/999167058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":999167058},{"type":"v1","addr":"172.21.15.121:6835","nonce":999167058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2160736935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2160736935},{"type":"v1","addr":"172.21.15.121:6837","nonce":2160736935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:21:02.048590+0000","modified":"2024-04-20T10:21:29.775269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6835/3336775250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3336775250},{"type":"v1","addr":"172.21.15.123:6835","nonce":3336775250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1}]} 2024-04-20T10:22:36.298 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 8 2024-04-20T10:22:36.300 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.299+0000 7fcc01200640 1 -- 172.21.15.121:0/1436121769 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fcbdc06c8f0 msgr2=0x7fcbdc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:36.300 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.299+0000 7fcc01200640 1 --2- 172.21.15.121:0/1436121769 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fcbdc06c8f0 0x7fcbdc06edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcbec034000 tx=0x7fcbec02f000 comp rx=0 tx=0).stop 2024-04-20T10:22:36.300 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.299+0000 7fcc01200640 1 -- 172.21.15.121:0/1436121769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbfc0b2f70 msgr2=0x7fcbfc0c9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:36.300 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.299+0000 7fcc01200640 1 --2- 172.21.15.121:0/1436121769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbfc0b2f70 0x7fcbfc0c9330 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fcbf400efc0 tx=0x7fcbf400c490 comp rx=0 tx=0).stop 2024-04-20T10:22:36.300 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.300+0000 7fcc01200640 1 -- 172.21.15.121:0/1436121769 shutdown_connections 2024-04-20T10:22:36.301 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.300+0000 7fcc01200640 1 --2- 172.21.15.121:0/1436121769 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fcbfc0b2f70 0x7fcbfc0c9330 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:36.301 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.300+0000 7fcc01200640 1 --2- 172.21.15.121:0/1436121769 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fcbdc06c8f0 0x7fcbdc06edb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:36.301 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.300+0000 7fcc01200640 1 --2- 172.21.15.121:0/1436121769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcbfc0adf60 0x7fcbfc0cefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:36.301 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.300+0000 7fcc01200640 1 --2- 172.21.15.121:0/1436121769 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fcbfc06b900 0x7fcbfc152a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:36.301 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.300+0000 7fcc01200640 1 -- 172.21.15.121:0/1436121769 >> 172.21.15.121:0/1436121769 conn(0x7fcbfc0d0040 msgr2=0x7fcbfc06db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:36.301 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.300+0000 7fcc01200640 1 -- 172.21.15.121:0/1436121769 shutdown_connections 2024-04-20T10:22:36.301 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.300+0000 7fcc01200640 1 -- 172.21.15.121:0/1436121769 wait complete. 2024-04-20T10:22:36.317 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:22:36.510 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.510+0000 7fe963200640 1 Processor -- start 2024-04-20T10:22:36.511 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.510+0000 7fe963200640 1 -- start start 2024-04-20T10:22:36.513 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.511+0000 7fe963200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe96415a1f0 0x7fe964072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:36.513 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.512+0000 7fe963200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe964072190 0x7fe964073420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:36.513 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.512+0000 7fe963200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe96415cc90 con 0x7fe964159e10 2024-04-20T10:22:36.513 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.512+0000 7fe963200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fe964072900 con 0x7fe964072190 2024-04-20T10:22:36.513 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.512+0000 7fe963200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe96415ca40 con 0x7fe96415a1f0 2024-04-20T10:22:36.513 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.512+0000 7fe962800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe964072190 0x7fe964073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:36.513 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.512+0000 7fe962800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe964072190 0x7fe964073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34182/0 (socket says 172.21.15.121:34182) 2024-04-20T10:22:36.513 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.512+0000 7fe962800640 1 -- 172.21.15.121:0/544556280 learned_addr learned my addr 172.21.15.121:0/544556280 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:36.514 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.512+0000 7fe95be00640 1 --2- 172.21.15.121:0/544556280 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe96415a1f0 0x7fe964072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:36.514 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.512+0000 7fe962800640 1 -- 172.21.15.121:0/544556280 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe96415a1f0 msgr2=0x7fe964072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:36.514 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.512+0000 7fe962800640 1 --2- 172.21.15.121:0/544556280 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe96415a1f0 0x7fe964072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:36.514 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.512+0000 7fe962800640 1 -- 172.21.15.121:0/544556280 >> v1:172.21.15.121:6789/0 conn(0x7fe964159e10 legacy=0x7fe964071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:36.514 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.512+0000 7fe962800640 1 -- 172.21.15.121:0/544556280 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe96415cf70 con 0x7fe964072190 2024-04-20T10:22:36.515 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.514+0000 7fe962800640 1 --2- 172.21.15.121:0/544556280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe964072190 0x7fe964073420 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe95c009e50 tx=0x7fe95c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b6a066e2df0c537 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:36.516 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.514+0000 7fe961400640 1 -- 172.21.15.121:0/544556280 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95c02cb00 con 0x7fe964072190 2024-04-20T10:22:36.516 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.514+0000 7fe961400640 1 -- 172.21.15.121:0/544556280 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe95c02cca0 con 0x7fe964072190 2024-04-20T10:22:36.516 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.514+0000 7fe961400640 1 -- 172.21.15.121:0/544556280 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95c034610 con 0x7fe964072190 2024-04-20T10:22:36.516 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.514+0000 7fe963200640 1 -- 172.21.15.121:0/544556280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe964072190 msgr2=0x7fe964073420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:36.516 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.514+0000 7fe963200640 1 --2- 172.21.15.121:0/544556280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe964072190 0x7fe964073420 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe95c009e50 tx=0x7fe95c02bf60 comp rx=0 tx=0).stop 2024-04-20T10:22:36.516 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.515+0000 7fe963200640 1 -- 172.21.15.121:0/544556280 shutdown_connections 2024-04-20T10:22:36.516 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.515+0000 7fe963200640 1 --2- 172.21.15.121:0/544556280 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe964072190 0x7fe964073420 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:36.516 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.515+0000 7fe963200640 1 --2- 172.21.15.121:0/544556280 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe96415a1f0 0x7fe964072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:36.516 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.515+0000 7fe963200640 1 -- 172.21.15.121:0/544556280 >> 172.21.15.121:0/544556280 conn(0x7fe9640d0040 msgr2=0x7fe9640d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:36.516 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.515+0000 7fe963200640 1 -- 172.21.15.121:0/544556280 shutdown_connections 2024-04-20T10:22:36.516 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.515+0000 7fe963200640 1 -- 172.21.15.121:0/544556280 wait complete. 2024-04-20T10:22:36.517 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.516+0000 7fe963200640 1 Processor -- start 2024-04-20T10:22:36.517 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.517+0000 7fe963200640 1 -- start start 2024-04-20T10:22:36.519 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.517+0000 7fe963200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe964072190 0x7fe96414ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:36.519 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.517+0000 7fe963200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe964159e10 0x7fe964158610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:36.519 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.517+0000 7fe963200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe96415a1f0 0x7fe96406c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:36.520 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.517+0000 7fe963200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe96415d810 con 0x7fe964072190 2024-04-20T10:22:36.520 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.517+0000 7fe963200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe96415d530 con 0x7fe964159e10 2024-04-20T10:22:36.520 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.517+0000 7fe963200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe96406da90 con 0x7fe96415a1f0 2024-04-20T10:22:36.520 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.518+0000 7fe961e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe964072190 0x7fe96414ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:36.520 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.518+0000 7fe961e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe964072190 0x7fe96414ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34192/0 (socket says 172.21.15.121:34192) 2024-04-20T10:22:36.520 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.518+0000 7fe961e00640 1 -- 172.21.15.121:0/4262913133 learned_addr learned my addr 172.21.15.121:0/4262913133 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:36.520 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.518+0000 7fe961e00640 1 -- 172.21.15.121:0/4262913133 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe96415a1f0 msgr2=0x7fe96406c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:36.520 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.518+0000 7fe961e00640 1 --2- 172.21.15.121:0/4262913133 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe96415a1f0 0x7fe96406c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:36.520 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.518+0000 7fe961e00640 1 -- 172.21.15.121:0/4262913133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe964159e10 msgr2=0x7fe964158610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:36.520 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.518+0000 7fe961e00640 1 --2- 172.21.15.121:0/4262913133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe964159e10 0x7fe964158610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:36.520 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.518+0000 7fe961e00640 1 -- 172.21.15.121:0/4262913133 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe96414f0f0 con 0x7fe964072190 2024-04-20T10:22:36.521 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.518+0000 7fe961e00640 1 --2- 172.21.15.121:0/4262913133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe964072190 0x7fe96414ebb0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe95400c430 tx=0x7fe95400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:36.521 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.519+0000 7fe95a000640 1 -- 172.21.15.121:0/4262913133 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9540052d0 con 0x7fe964072190 2024-04-20T10:22:36.521 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.519+0000 7fe95a000640 1 -- 172.21.15.121:0/4262913133 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe954005490 con 0x7fe964072190 2024-04-20T10:22:36.521 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.519+0000 7fe95a000640 1 -- 172.21.15.121:0/4262913133 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe954007260 con 0x7fe964072190 2024-04-20T10:22:36.521 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.519+0000 7fe963200640 1 -- 172.21.15.121:0/4262913133 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe964153f50 con 0x7fe964072190 2024-04-20T10:22:36.521 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.519+0000 7fe963200640 1 -- 172.21.15.121:0/4262913133 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe964154510 con 0x7fe964072190 2024-04-20T10:22:36.521 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.520+0000 7fe963200640 1 -- 172.21.15.121:0/4262913133 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe964075b60 con 0x7fe964072190 2024-04-20T10:22:36.522 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.521+0000 7fe95a000640 1 -- 172.21.15.121:0/4262913133 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe954007400 con 0x7fe964072190 2024-04-20T10:22:36.522 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.522+0000 7fe95a000640 1 --2- 172.21.15.121:0/4262913133 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe94006c8f0 0x7fe94006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:36.522 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.522+0000 7fe95a000640 1 -- 172.21.15.121:0/4262913133 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fe95408b4b0 con 0x7fe964072190 2024-04-20T10:22:36.523 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.522+0000 7fe95be00640 1 --2- 172.21.15.121:0/4262913133 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe94006c8f0 0x7fe94006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:36.523 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.523+0000 7fe95be00640 1 --2- 172.21.15.121:0/4262913133 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe94006c8f0 0x7fe94006edb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe964157a20 tx=0x7fe94c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:36.527 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.526+0000 7fe95a000640 1 -- 172.21.15.121:0/4262913133 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe96414f0f0 con 0x7fe964072190 2024-04-20T10:22:36.729 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.728+0000 7fe963200640 1 -- 172.21.15.121:0/4262913133 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe96406c4b0 con 0x7fe964072190 2024-04-20T10:22:36.730 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.730+0000 7fe95a000640 1 -- 172.21.15.121:0/4262913133 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18940 (secure 0 0 0) 0x7fe9540594e0 con 0x7fe964072190 2024-04-20T10:22:36.731 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:22:36.732 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":27,"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","created":"2024-04-20T10:20:38.076076+0000","modified":"2024-04-20T10:21:44.715898+0000","last_up_change":"2024-04-20T10:20:50.162700+0000","last_in_change":"2024-04-20T10:20:41.510382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:20:51.141365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:21:00.436937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:21:01.026957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:21:04.862693+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"629a7b8c-a40a-4a2a-a3cc-d9f357774310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6809","nonce":1044588835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6811","nonce":1044588835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6815","nonce":1044588835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6813","nonce":1044588835}]},"public_addr":"172.21.15.121:6809/1044588835","cluster_addr":"172.21.15.121:6811/1044588835","heartbeat_back_addr":"172.21.15.121:6815/1044588835","heartbeat_front_addr":"172.21.15.121:6813/1044588835","state":["exists","up"]},{"osd":1,"uuid":"f2570399-aa50-4ff3-bf5d-3d09207290db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6801","nonce":3505199624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6803","nonce":3505199624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6807","nonce":3505199624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6805","nonce":3505199624}]},"public_addr":"172.21.15.121:6801/3505199624","cluster_addr":"172.21.15.121:6803/3505199624","heartbeat_back_addr":"172.21.15.121:6807/3505199624","heartbeat_front_addr":"172.21.15.121:6805/3505199624","state":["exists","up"]},{"osd":2,"uuid":"50893807-dc30-49f4-b41f-739366626658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6817","nonce":3656724207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6819","nonce":3656724207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6827","nonce":3656724207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6822","nonce":3656724207}]},"public_addr":"172.21.15.121:6817/3656724207","cluster_addr":"172.21.15.121:6819/3656724207","heartbeat_back_addr":"172.21.15.121:6827/3656724207","heartbeat_front_addr":"172.21.15.121:6822/3656724207","state":["exists","up"]},{"osd":3,"uuid":"dadb01bc-5491-4ffc-8824-d48a057240b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6823","nonce":2232342832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6826","nonce":2232342832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6831","nonce":2232342832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6829","nonce":2232342832}]},"public_addr":"172.21.15.121:6823/2232342832","cluster_addr":"172.21.15.121:6826/2232342832","heartbeat_back_addr":"172.21.15.121:6831/2232342832","heartbeat_front_addr":"172.21.15.121:6829/2232342832","state":["exists","up"]},{"osd":4,"uuid":"b3f3b4a1-b375-4526-b69e-344ab24c2da1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6801","nonce":99353111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6803","nonce":99353111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6807","nonce":99353111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6805","nonce":99353111}]},"public_addr":"172.21.15.123:6801/99353111","cluster_addr":"172.21.15.123:6803/99353111","heartbeat_back_addr":"172.21.15.123:6807/99353111","heartbeat_front_addr":"172.21.15.123:6805/99353111","state":["exists","up"]},{"osd":5,"uuid":"2ee24a08-1624-4961-9642-d7b510282484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6818","nonce":2504374688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6822","nonce":2504374688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6830","nonce":2504374688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6826","nonce":2504374688}]},"public_addr":"172.21.15.123:6818/2504374688","cluster_addr":"172.21.15.123:6822/2504374688","heartbeat_back_addr":"172.21.15.123:6830/2504374688","heartbeat_front_addr":"172.21.15.123:6826/2504374688","state":["exists","up"]},{"osd":6,"uuid":"186118cc-6128-49de-acca-55260945c1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6819","nonce":4139299599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6823","nonce":4139299599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6829","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6831","nonce":4139299599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6827","nonce":4139299599}]},"public_addr":"172.21.15.123:6819/4139299599","cluster_addr":"172.21.15.123:6823/4139299599","heartbeat_back_addr":"172.21.15.123:6831/4139299599","heartbeat_front_addr":"172.21.15.123:6827/4139299599","state":["exists","up"]},{"osd":7,"uuid":"a00aac6e-4bd9-415a-a93d-ae85210865ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6809","nonce":4043981069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6811","nonce":4043981069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6815","nonce":4043981069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6813","nonce":4043981069}]},"public_addr":"172.21.15.123:6809/4043981069","cluster_addr":"172.21.15.123:6811/4043981069","heartbeat_back_addr":"172.21.15.123:6815/4043981069","heartbeat_front_addr":"172.21.15.123:6813/4043981069","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.904540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.958585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.616367+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.885244+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.103127+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.167248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.884348+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.862356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.23","mappings":[{"from":3,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:22:36.734 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.734+0000 7fe963200640 1 -- 172.21.15.121:0/4262913133 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe94006c8f0 msgr2=0x7fe94006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:36.734 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.734+0000 7fe963200640 1 --2- 172.21.15.121:0/4262913133 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe94006c8f0 0x7fe94006edb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe964157a20 tx=0x7fe94c008040 comp rx=0 tx=0).stop 2024-04-20T10:22:36.735 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.734+0000 7fe963200640 1 -- 172.21.15.121:0/4262913133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe964072190 msgr2=0x7fe96414ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:36.735 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.734+0000 7fe963200640 1 --2- 172.21.15.121:0/4262913133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe964072190 0x7fe96414ebb0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe95400c430 tx=0x7fe95400c8f0 comp rx=0 tx=0).stop 2024-04-20T10:22:36.735 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.734+0000 7fe963200640 1 -- 172.21.15.121:0/4262913133 shutdown_connections 2024-04-20T10:22:36.735 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.734+0000 7fe963200640 1 --2- 172.21.15.121:0/4262913133 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe96415a1f0 0x7fe96406c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:36.735 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.734+0000 7fe963200640 1 --2- 172.21.15.121:0/4262913133 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe94006c8f0 0x7fe94006edb0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:36.735 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.734+0000 7fe963200640 1 --2- 172.21.15.121:0/4262913133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe964159e10 0x7fe964158610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:36.735 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.734+0000 7fe963200640 1 --2- 172.21.15.121:0/4262913133 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe964072190 0x7fe96414ebb0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:36.735 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.734+0000 7fe963200640 1 -- 172.21.15.121:0/4262913133 >> 172.21.15.121:0/4262913133 conn(0x7fe9640d0040 msgr2=0x7fe964160830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:36.735 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.734+0000 7fe963200640 1 -- 172.21.15.121:0/4262913133 shutdown_connections 2024-04-20T10:22:36.736 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.734+0000 7fe963200640 1 -- 172.21.15.121:0/4262913133 wait complete. 2024-04-20T10:22:36.753 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-20T10:22:36.753 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:22:36.945 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.944+0000 7f86e0600640 1 Processor -- start 2024-04-20T10:22:36.945 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.944+0000 7f86e0600640 1 -- start start 2024-04-20T10:22:36.946 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.946+0000 7f86e0600640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86d815b3b0 0x7f86d806eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:36.947 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.946+0000 7f86e0600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f86d806dcb0 0x7f86d8071b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:36.947 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.946+0000 7f86e0600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f86d8072090 0x7f86d80d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:36.947 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.946+0000 7f86e0600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f86d80ade40 con 0x7f86d8072090 2024-04-20T10:22:36.947 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.946+0000 7f86e0600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f86d80ae0e0 con 0x7f86d806dcb0 2024-04-20T10:22:36.947 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.946+0000 7f86e0600640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f86d815b790 con 0x7f86d815b3b0 2024-04-20T10:22:36.947 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.946+0000 7f86de600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f86d8072090 0x7f86d80d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:36.947 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.946+0000 7f86de600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f86d8072090 0x7f86d80d0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34204/0 (socket says 172.21.15.121:34204) 2024-04-20T10:22:36.947 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.946+0000 7f86de600640 1 -- 172.21.15.121:0/780584379 learned_addr learned my addr 172.21.15.121:0/780584379 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:36.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.946+0000 7f86ddc00640 1 --2- 172.21.15.121:0/780584379 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86d815b3b0 0x7f86d806eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:36.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.947+0000 7f86dd200640 1 --2- 172.21.15.121:0/780584379 >> v2:172.21.15.123:3300/0 conn(0x7f86d806dcb0 0x7f86d8071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:36.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.947+0000 7f86de600640 1 -- 172.21.15.121:0/780584379 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86d815b3b0 msgr2=0x7f86d806eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:36.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.947+0000 7f86de600640 1 --2- 172.21.15.121:0/780584379 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86d815b3b0 0x7f86d806eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:36.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.947+0000 7f86de600640 1 -- 172.21.15.121:0/780584379 >> v2:172.21.15.123:3300/0 conn(0x7f86d806dcb0 msgr2=0x7f86d8071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:36.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.947+0000 7f86de600640 1 --2- 172.21.15.121:0/780584379 >> v2:172.21.15.123:3300/0 conn(0x7f86d806dcb0 0x7f86d8071b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:36.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.947+0000 7f86de600640 1 -- 172.21.15.121:0/780584379 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d80ae380 con 0x7f86d8072090 2024-04-20T10:22:36.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.947+0000 7f86dd200640 1 --2- 172.21.15.121:0/780584379 >> v2:172.21.15.123:3300/0 conn(0x7f86d806dcb0 0x7f86d8071b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:36.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.947+0000 7f86ddc00640 1 --2- 172.21.15.121:0/780584379 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86d815b3b0 0x7f86d806eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:36.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.948+0000 7f86de600640 1 --2- 172.21.15.121:0/780584379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86d8072090 0x7f86d80d0360 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f86d806d920 tx=0x7f86d002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e283a84cd956738 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:36.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.948+0000 7f86d7e00640 1 -- 172.21.15.121:0/780584379 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d002ca80 con 0x7f86d8072090 2024-04-20T10:22:36.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.948+0000 7f86d7e00640 1 -- 172.21.15.121:0/780584379 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86d002cc20 con 0x7f86d8072090 2024-04-20T10:22:36.949 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.948+0000 7f86d7e00640 1 -- 172.21.15.121:0/780584379 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d00343f0 con 0x7f86d8072090 2024-04-20T10:22:36.950 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.949+0000 7f86e0600640 1 -- 172.21.15.121:0/780584379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86d8072090 msgr2=0x7f86d80d0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:36.950 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.949+0000 7f86e0600640 1 --2- 172.21.15.121:0/780584379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86d8072090 0x7f86d80d0360 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f86d806d920 tx=0x7f86d002bee0 comp rx=0 tx=0).stop 2024-04-20T10:22:36.951 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.949+0000 7f86e0600640 1 -- 172.21.15.121:0/780584379 shutdown_connections 2024-04-20T10:22:36.951 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.949+0000 7f86e0600640 1 --2- 172.21.15.121:0/780584379 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86d8072090 0x7f86d80d0360 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:36.951 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.949+0000 7f86e0600640 1 --2- 172.21.15.121:0/780584379 >> v2:172.21.15.123:3300/0 conn(0x7f86d806dcb0 0x7f86d8071b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:36.951 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.949+0000 7f86e0600640 1 --2- 172.21.15.121:0/780584379 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86d815b3b0 0x7f86d806eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:36.951 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.949+0000 7f86e0600640 1 -- 172.21.15.121:0/780584379 >> 172.21.15.121:0/780584379 conn(0x7f86d80b3350 msgr2=0x7f86d80b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:36.951 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.949+0000 7f86e0600640 1 -- 172.21.15.121:0/780584379 shutdown_connections 2024-04-20T10:22:36.951 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.949+0000 7f86e0600640 1 -- 172.21.15.121:0/780584379 wait complete. 2024-04-20T10:22:36.953 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.950+0000 7f86e0600640 1 Processor -- start 2024-04-20T10:22:36.953 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.951+0000 7f86e0600640 1 -- start start 2024-04-20T10:22:36.953 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.951+0000 7f86e0600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86d806dcb0 0x7f86d814e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:36.953 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.952+0000 7f86e0600640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86d815b3b0 0x7f86d8155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:36.953 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.952+0000 7f86e0600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86d809d910 0x7f86d814f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:36.953 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.952+0000 7f86e0600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f86d80aec20 con 0x7f86d809d910 2024-04-20T10:22:36.953 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.952+0000 7f86e0600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f86d80ae940 con 0x7f86d806dcb0 2024-04-20T10:22:36.953 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.952+0000 7f86e0600640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f86d806efe0 con 0x7f86d815b3b0 2024-04-20T10:22:36.953 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.952+0000 7f86de600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86d809d910 0x7f86d814f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:36.954 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.952+0000 7f86de600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86d809d910 0x7f86d814f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34206/0 (socket says 172.21.15.121:34206) 2024-04-20T10:22:36.954 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.952+0000 7f86de600640 1 -- 172.21.15.121:0/1323344748 learned_addr learned my addr 172.21.15.121:0/1323344748 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:36.954 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.952+0000 7f86de600640 1 -- 172.21.15.121:0/1323344748 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86d815b3b0 msgr2=0x7f86d8155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:36.954 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.952+0000 7f86dd200640 1 --2- 172.21.15.121:0/1323344748 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86d815b3b0 0x7f86d8155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:36.954 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.952+0000 7f86de600640 1 --2- 172.21.15.121:0/1323344748 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86d815b3b0 0x7f86d8155420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:36.954 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.952+0000 7f86de600640 1 -- 172.21.15.121:0/1323344748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86d806dcb0 msgr2=0x7f86d814e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:36.954 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.952+0000 7f86de600640 1 --2- 172.21.15.121:0/1323344748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86d806dcb0 0x7f86d814e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:36.954 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.952+0000 7f86de600640 1 -- 172.21.15.121:0/1323344748 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86d8155ac0 con 0x7f86d809d910 2024-04-20T10:22:36.954 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.953+0000 7f86de600640 1 --2- 172.21.15.121:0/1323344748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86d809d910 0x7f86d814f260 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f86d0009f40 tx=0x7f86d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:36.955 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.953+0000 7f86d5600640 1 -- 172.21.15.121:0/1323344748 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d0043020 con 0x7f86d809d910 2024-04-20T10:22:36.955 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.953+0000 7f86d5600640 1 -- 172.21.15.121:0/1323344748 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86d003e450 con 0x7f86d809d910 2024-04-20T10:22:36.955 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.953+0000 7f86e0600640 1 -- 172.21.15.121:0/1323344748 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86d8155da0 con 0x7f86d809d910 2024-04-20T10:22:36.955 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.953+0000 7f86d5600640 1 -- 172.21.15.121:0/1323344748 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86d0009510 con 0x7f86d809d910 2024-04-20T10:22:36.955 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.953+0000 7f86e0600640 1 -- 172.21.15.121:0/1323344748 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86d8156360 con 0x7f86d809d910 2024-04-20T10:22:36.955 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.954+0000 7f86e0600640 1 -- 172.21.15.121:0/1323344748 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86d8068980 con 0x7f86d809d910 2024-04-20T10:22:36.956 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.955+0000 7f86d5600640 1 -- 172.21.15.121:0/1323344748 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f86d0046020 con 0x7f86d809d910 2024-04-20T10:22:36.957 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.956+0000 7f86d5600640 1 --2- 172.21.15.121:0/1323344748 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f86b806c8f0 0x7f86b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:36.957 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.957+0000 7f86d5600640 1 -- 172.21.15.121:0/1323344748 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f86d00ae7e0 con 0x7f86d809d910 2024-04-20T10:22:36.957 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.957+0000 7f86ddc00640 1 --2- 172.21.15.121:0/1323344748 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f86b806c8f0 0x7f86b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:36.957 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.957+0000 7f86ddc00640 1 --2- 172.21.15.121:0/1323344748 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f86b806c8f0 0x7f86b806edb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f86cc006fd0 tx=0x7f86cc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:36.961 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:36.961+0000 7f86d5600640 1 -- 172.21.15.121:0/1323344748 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f86d004d030 con 0x7f86d809d910 2024-04-20T10:22:37.184 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.183+0000 7f86e0600640 1 -- 172.21.15.121:0/1323344748 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86d80738a0 con 0x7f86d809d910 2024-04-20T10:22:37.184 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.184+0000 7f86d5600640 1 -- 172.21.15.121:0/1323344748 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5204 (secure 0 0 0) 0x7f86d007c800 con 0x7f86d809d910 2024-04-20T10:22:37.185 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:22:37.185 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1376743826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1376743826},{"type":"v1","addr":"172.21.15.123:6833","nonce":1376743826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/999167058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":999167058},{"type":"v1","addr":"172.21.15.121:6835","nonce":999167058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2160736935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2160736935},{"type":"v1","addr":"172.21.15.121:6837","nonce":2160736935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:21:02.048590+0000","modified":"2024-04-20T10:21:29.775269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6835/3336775250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3336775250},{"type":"v1","addr":"172.21.15.123:6835","nonce":3336775250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1}]} 2024-04-20T10:22:37.185 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 8 2024-04-20T10:22:37.187 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.186+0000 7f86e0600640 1 -- 172.21.15.121:0/1323344748 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f86b806c8f0 msgr2=0x7f86b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:37.187 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.186+0000 7f86e0600640 1 --2- 172.21.15.121:0/1323344748 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f86b806c8f0 0x7f86b806edb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f86cc006fd0 tx=0x7f86cc008040 comp rx=0 tx=0).stop 2024-04-20T10:22:37.188 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.186+0000 7f86e0600640 1 -- 172.21.15.121:0/1323344748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86d809d910 msgr2=0x7f86d814f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:37.188 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.186+0000 7f86e0600640 1 --2- 172.21.15.121:0/1323344748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86d809d910 0x7f86d814f260 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f86d0009f40 tx=0x7f86d0004a40 comp rx=0 tx=0).stop 2024-04-20T10:22:37.188 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.187+0000 7f86e0600640 1 -- 172.21.15.121:0/1323344748 shutdown_connections 2024-04-20T10:22:37.188 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.187+0000 7f86e0600640 1 --2- 172.21.15.121:0/1323344748 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f86d809d910 0x7f86d814f260 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:37.188 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.187+0000 7f86e0600640 1 --2- 172.21.15.121:0/1323344748 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f86d815b3b0 0x7f86d8155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:37.188 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.187+0000 7f86e0600640 1 --2- 172.21.15.121:0/1323344748 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f86b806c8f0 0x7f86b806edb0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:37.188 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.187+0000 7f86e0600640 1 --2- 172.21.15.121:0/1323344748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86d806dcb0 0x7f86d814e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:37.188 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.187+0000 7f86e0600640 1 -- 172.21.15.121:0/1323344748 >> 172.21.15.121:0/1323344748 conn(0x7f86d80b3350 msgr2=0x7f86d814ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:37.188 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.187+0000 7f86e0600640 1 -- 172.21.15.121:0/1323344748 shutdown_connections 2024-04-20T10:22:37.188 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.187+0000 7f86e0600640 1 -- 172.21.15.121:0/1323344748 wait complete. 2024-04-20T10:22:37.209 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:22:37.405 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.404+0000 7fccd5000640 1 Processor -- start 2024-04-20T10:22:37.405 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.404+0000 7fccd5000640 1 -- start start 2024-04-20T10:22:37.406 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.406+0000 7fccd5000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fccd0159e10 0x7fccd0071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:37.407 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.406+0000 7fccd5000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fccd006da90 0x7fccd0071fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:37.407 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.406+0000 7fccd5000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccd015cc90 con 0x7fccd015a1f0 2024-04-20T10:22:37.407 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.406+0000 7fccd5000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fccd0072900 con 0x7fccd0159e10 2024-04-20T10:22:37.407 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.406+0000 7fccd5000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fccd015ca40 con 0x7fccd006da90 2024-04-20T10:22:37.407 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.406+0000 7fcccea00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7fccd015a1f0 0x7fccd0160030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:46166/0 (socket says 172.21.15.121:46166) 2024-04-20T10:22:37.407 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.406+0000 7fcccea00640 1 -- 172.21.15.121:0/332157487 learned_addr learned my addr 172.21.15.121:0/332157487 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:37.409 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.406+0000 7fcccfe00640 1 --2- 172.21.15.121:0/332157487 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fccd006da90 0x7fccd0071fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:37.410 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.407+0000 7fcccf400640 1 --2- 172.21.15.121:0/332157487 >> v2:172.21.15.123:3300/0 conn(0x7fccd0159e10 0x7fccd0071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:37.410 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.407+0000 7fccce000640 1 -- 172.21.15.121:0/332157487 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1761202741 0 0) 0x7fccd015cc90 con 0x7fccd015a1f0 2024-04-20T10:22:37.410 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.407+0000 7fccce000640 1 -- 172.21.15.121:0/332157487 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fccb4003520 con 0x7fccd015a1f0 2024-04-20T10:22:37.410 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.407+0000 7fccce000640 1 -- 172.21.15.121:0/332157487 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 95258432 0 0) 0x7fccb4003520 con 0x7fccd015a1f0 2024-04-20T10:22:37.410 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.407+0000 7fccce000640 1 -- 172.21.15.121:0/332157487 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fccd006da90 msgr2=0x7fccd0071fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:37.410 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.407+0000 7fccce000640 1 --2- 172.21.15.121:0/332157487 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fccd006da90 0x7fccd0071fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:37.410 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.407+0000 7fccce000640 1 -- 172.21.15.121:0/332157487 >> v2:172.21.15.123:3300/0 conn(0x7fccd0159e10 msgr2=0x7fccd0071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:37.410 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.407+0000 7fccce000640 1 --2- 172.21.15.121:0/332157487 >> v2:172.21.15.123:3300/0 conn(0x7fccd0159e10 0x7fccd0071a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:37.410 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.407+0000 7fccce000640 1 -- 172.21.15.121:0/332157487 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd015cf70 con 0x7fccd015a1f0 2024-04-20T10:22:37.410 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.407+0000 7fccce000640 1 -- 172.21.15.121:0/332157487 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7fccbc001690 con 0x7fccd015a1f0 2024-04-20T10:22:37.410 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.407+0000 7fccce000640 1 -- 172.21.15.121:0/332157487 >> v1:172.21.15.121:6789/0 conn(0x7fccd015a1f0 legacy=0x7fccd0160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:37.410 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.407+0000 7fccce000640 1 --2- 172.21.15.121:0/332157487 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fccd006da90 0x7fccb40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:37.411 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.407+0000 7fccce000640 1 --2- 172.21.15.121:0/332157487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccb4003f30 0x7fccb4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:37.411 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.407+0000 7fccce000640 1 --2- 172.21.15.121:0/332157487 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fccb4004d40 0x7fccb40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:37.411 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.407+0000 7fccce000640 1 -- 172.21.15.121:0/332157487 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fccd015ca40 con 0x7fccb4004d40 2024-04-20T10:22:37.411 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.407+0000 7fccce000640 1 -- 172.21.15.121:0/332157487 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fccb400b9c0 con 0x7fccb4003f30 2024-04-20T10:22:37.411 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.407+0000 7fccce000640 1 -- 172.21.15.121:0/332157487 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fccb400bb40 con 0x7fccd006da90 2024-04-20T10:22:37.411 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.408+0000 7fcccfe00640 1 --2- 172.21.15.121:0/332157487 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fccb4004d40 0x7fccb40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:37.411 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.408+0000 7fcccf400640 1 --2- 172.21.15.121:0/332157487 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fccd006da90 0x7fccb40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:37.411 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.408+0000 7fcccfe00640 1 -- 172.21.15.121:0/332157487 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fccd006da90 msgr2=0x7fccb40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:37.411 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.408+0000 7fcccfe00640 1 --2- 172.21.15.121:0/332157487 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fccd006da90 0x7fccb40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:37.411 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.408+0000 7fcccfe00640 1 -- 172.21.15.121:0/332157487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccb4003f30 msgr2=0x7fccb4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:37.411 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.408+0000 7fcccea00640 1 --2- 172.21.15.121:0/332157487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccb4003f30 0x7fccb4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:37.411 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.408+0000 7fcccfe00640 1 --2- 172.21.15.121:0/332157487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccb4003f30 0x7fccb4004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:37.411 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.408+0000 7fcccfe00640 1 -- 172.21.15.121:0/332157487 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb4003520 con 0x7fccb4004d40 2024-04-20T10:22:37.412 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.408+0000 7fcccf400640 1 --2- 172.21.15.121:0/332157487 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fccd006da90 0x7fccb40039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:37.412 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.409+0000 7fcccfe00640 1 --2- 172.21.15.121:0/332157487 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fccb4004d40 0x7fccb40051c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fccc8009fc0 tx=0x7fccb4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e857b7c20c8d5a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:37.412 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.409+0000 7fccce000640 1 -- 172.21.15.121:0/332157487 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc8039070 con 0x7fccb4004d40 2024-04-20T10:22:37.412 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.409+0000 7fccce000640 1 -- 172.21.15.121:0/332157487 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccc802cc00 con 0x7fccb4004d40 2024-04-20T10:22:37.412 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.409+0000 7fccce000640 1 -- 172.21.15.121:0/332157487 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc8009c10 con 0x7fccb4004d40 2024-04-20T10:22:37.413 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.410+0000 7fccd5000640 1 -- 172.21.15.121:0/332157487 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fccb4004d40 msgr2=0x7fccb40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:37.413 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.410+0000 7fccd5000640 1 --2- 172.21.15.121:0/332157487 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fccb4004d40 0x7fccb40051c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fccc8009fc0 tx=0x7fccb4005830 comp rx=0 tx=0).stop 2024-04-20T10:22:37.413 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.410+0000 7fcccfe00640 1 -- 172.21.15.121:0/332157487 reap_dead start 2024-04-20T10:22:37.413 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.410+0000 7fccd5000640 1 -- 172.21.15.121:0/332157487 shutdown_connections 2024-04-20T10:22:37.413 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.410+0000 7fccd5000640 1 -- 172.21.15.121:0/332157487 >> 172.21.15.121:0/332157487 conn(0x7fccd00d0040 msgr2=0x7fccd00d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:37.413 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.410+0000 7fccd5000640 1 -- 172.21.15.121:0/332157487 shutdown_connections 2024-04-20T10:22:37.413 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.410+0000 7fccd5000640 1 -- 172.21.15.121:0/332157487 wait complete. 2024-04-20T10:22:37.413 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.412+0000 7fccd5000640 1 Processor -- start 2024-04-20T10:22:37.413 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.412+0000 7fccd5000640 1 -- start start 2024-04-20T10:22:37.415 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.413+0000 7fccd5000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fccb4004d40 0x7fccd015e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:37.415 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.413+0000 7fccd5000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fccd00cf6a0 0x7fccd01f1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:37.415 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.413+0000 7fccd5000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccd01f2110 0x7fccd01f2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:37.415 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.413+0000 7fccd5000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fccd015d810 con 0x7fccd00cf6a0 2024-04-20T10:22:37.415 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.413+0000 7fccd5000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fccd015d530 con 0x7fccd01f2110 2024-04-20T10:22:37.416 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.413+0000 7fccd5000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fccd006b900 con 0x7fccb4004d40 2024-04-20T10:22:37.416 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.413+0000 7fcccea00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fccd00cf6a0 0x7fccd01f1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:37.416 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.413+0000 7fcccea00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fccd00cf6a0 0x7fccd01f1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34214/0 (socket says 172.21.15.121:34214) 2024-04-20T10:22:37.416 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.413+0000 7fcccea00640 1 -- 172.21.15.121:0/3114915643 learned_addr learned my addr 172.21.15.121:0/3114915643 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:37.416 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.413+0000 7fcccea00640 1 -- 172.21.15.121:0/3114915643 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fccb4004d40 msgr2=0x7fccd015e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:37.416 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.413+0000 7fcccea00640 1 --2- 172.21.15.121:0/3114915643 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fccb4004d40 0x7fccd015e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:37.416 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.413+0000 7fcccea00640 1 -- 172.21.15.121:0/3114915643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccd01f2110 msgr2=0x7fccd01f2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:37.416 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.413+0000 7fcccea00640 1 --2- 172.21.15.121:0/3114915643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccd01f2110 0x7fccd01f2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:37.416 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.414+0000 7fcccea00640 1 -- 172.21.15.121:0/3114915643 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd00cfb20 con 0x7fccd00cf6a0 2024-04-20T10:22:37.416 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.414+0000 7fcccea00640 1 --2- 172.21.15.121:0/3114915643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fccd00cf6a0 0x7fccd01f1bd0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fccbc00d830 tx=0x7fccbc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:37.416 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.414+0000 7fccc7400640 1 -- 172.21.15.121:0/3114915643 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccbc019070 con 0x7fccd00cf6a0 2024-04-20T10:22:37.416 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.414+0000 7fccc7400640 1 -- 172.21.15.121:0/3114915643 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccbc0074e0 con 0x7fccd00cf6a0 2024-04-20T10:22:37.416 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.414+0000 7fccc7400640 1 -- 172.21.15.121:0/3114915643 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccbc004a00 con 0x7fccd00cf6a0 2024-04-20T10:22:37.416 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.414+0000 7fccd5000640 1 -- 172.21.15.121:0/3114915643 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccd015ec60 con 0x7fccd00cf6a0 2024-04-20T10:22:37.417 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.414+0000 7fccd5000640 1 -- 172.21.15.121:0/3114915643 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccd015f220 con 0x7fccd00cf6a0 2024-04-20T10:22:37.417 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.416+0000 7fccd5000640 1 -- 172.21.15.121:0/3114915643 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccd015f460 con 0x7fccd00cf6a0 2024-04-20T10:22:37.417 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.416+0000 7fccc7400640 1 -- 172.21.15.121:0/3114915643 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fccbc004ba0 con 0x7fccd00cf6a0 2024-04-20T10:22:37.418 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.417+0000 7fccc7400640 1 --2- 172.21.15.121:0/3114915643 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fccb006c8f0 0x7fccb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:37.418 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.418+0000 7fccc7400640 1 -- 172.21.15.121:0/3114915643 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fccbc08b930 con 0x7fccd00cf6a0 2024-04-20T10:22:37.418 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.418+0000 7fcccf400640 1 --2- 172.21.15.121:0/3114915643 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fccb006c8f0 0x7fccb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:37.418 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.418+0000 7fcccf400640 1 --2- 172.21.15.121:0/3114915643 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fccb006c8f0 0x7fccb006edb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fccc0009a60 tx=0x7fccc0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:37.422 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.422+0000 7fccc7400640 1 -- 172.21.15.121:0/3114915643 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fccd015f460 con 0x7fccd00cf6a0 2024-04-20T10:22:37.667 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.666+0000 7fccd5000640 1 -- 172.21.15.121:0/3114915643 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fccd0160620 con 0x7fccd00cf6a0 2024-04-20T10:22:37.668 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.667+0000 7fccc7400640 1 -- 172.21.15.121:0/3114915643 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5204 (secure 0 0 0) 0x7fccbc059a20 con 0x7fccd00cf6a0 2024-04-20T10:22:37.668 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:22:37.668 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4238,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1376743826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1376743826},{"type":"v1","addr":"172.21.15.123:6833","nonce":1376743826}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/999167058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":999167058},{"type":"v1","addr":"172.21.15.121:6835","nonce":999167058}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6837/2160736935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":2160736935},{"type":"v1","addr":"172.21.15.121:6837","nonce":2160736935}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:21:02.048590+0000","modified":"2024-04-20T10:21:29.775269+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6835/3336775250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3336775250},{"type":"v1","addr":"172.21.15.123:6835","nonce":3336775250}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4281,"qdb_cluster":[4281]},"id":1}]} 2024-04-20T10:22:37.669 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 8 2024-04-20T10:22:37.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.670+0000 7fccd5000640 1 -- 172.21.15.121:0/3114915643 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fccb006c8f0 msgr2=0x7fccb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:37.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.670+0000 7fccd5000640 1 --2- 172.21.15.121:0/3114915643 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fccb006c8f0 0x7fccb006edb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fccc0009a60 tx=0x7fccc0008040 comp rx=0 tx=0).stop 2024-04-20T10:22:37.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.670+0000 7fccd5000640 1 -- 172.21.15.121:0/3114915643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fccd00cf6a0 msgr2=0x7fccd01f1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:37.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.670+0000 7fccd5000640 1 --2- 172.21.15.121:0/3114915643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fccd00cf6a0 0x7fccd01f1bd0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fccbc00d830 tx=0x7fccbc00dcf0 comp rx=0 tx=0).stop 2024-04-20T10:22:37.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.670+0000 7fccd5000640 1 -- 172.21.15.121:0/3114915643 shutdown_connections 2024-04-20T10:22:37.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.670+0000 7fccd5000640 1 --2- 172.21.15.121:0/3114915643 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fccb006c8f0 0x7fccb006edb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:37.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.670+0000 7fccd5000640 1 --2- 172.21.15.121:0/3114915643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fccd01f2110 0x7fccd01f2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:37.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.670+0000 7fccd5000640 1 --2- 172.21.15.121:0/3114915643 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fccd00cf6a0 0x7fccd01f1bd0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:37.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.670+0000 7fccd5000640 1 --2- 172.21.15.121:0/3114915643 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fccb4004d40 0x7fccd015e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:37.672 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.670+0000 7fccd5000640 1 -- 172.21.15.121:0/3114915643 >> 172.21.15.121:0/3114915643 conn(0x7fccd00d0040 msgr2=0x7fccd0071a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:37.672 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.670+0000 7fccd5000640 1 -- 172.21.15.121:0/3114915643 shutdown_connections 2024-04-20T10:22:37.672 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.670+0000 7fccd5000640 1 -- 172.21.15.121:0/3114915643 wait complete. 2024-04-20T10:22:37.690 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:22:37.890 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.889+0000 7f036a800640 1 Processor -- start 2024-04-20T10:22:37.890 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.889+0000 7f036a800640 1 -- start start 2024-04-20T10:22:37.891 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.891+0000 7f036a800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0364161d50 0x7f03640d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:37.892 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.891+0000 7f036a800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f036406cde0 0x7f03640b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:37.892 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.891+0000 7f036a800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f036406c190 con 0x7f03640b3330 2024-04-20T10:22:37.892 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.891+0000 7f036a800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f0364162130 con 0x7f036406cde0 2024-04-20T10:22:37.892 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.891+0000 7f036a800640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f036406c470 con 0x7f0364161d50 2024-04-20T10:22:37.892 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.891+0000 7f0369e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f036406cde0 0x7f03640b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:37.892 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.891+0000 7f0368a00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f03640b3330 0x7f036406c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:46170/0 (socket says 172.21.15.121:46170) 2024-04-20T10:22:37.892 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.891+0000 7f0368a00640 1 -- 172.21.15.121:0/527880322 learned_addr learned my addr 172.21.15.121:0/527880322 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:37.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.892+0000 7f0369400640 1 --2- 172.21.15.121:0/527880322 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0364161d50 0x7f03640d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:37.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.892+0000 7f0369e00640 1 -- 172.21.15.121:0/527880322 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0364161d50 msgr2=0x7f03640d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:37.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.892+0000 7f0369e00640 1 --2- 172.21.15.121:0/527880322 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0364161d50 0x7f03640d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:37.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.892+0000 7f0369e00640 1 -- 172.21.15.121:0/527880322 >> v1:172.21.15.121:6789/0 conn(0x7f03640b3330 legacy=0x7f036406c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:37.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.892+0000 7f0369e00640 1 -- 172.21.15.121:0/527880322 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03640d2e00 con 0x7f036406cde0 2024-04-20T10:22:37.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.892+0000 7f0363e00640 1 -- 172.21.15.121:0/527880322 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1687108800 0 0) 0x7f036406c190 con 0x7f03640b3330 2024-04-20T10:22:37.894 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.893+0000 7f0369e00640 1 --2- 172.21.15.121:0/527880322 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f036406cde0 0x7f03640b3880 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f035c009d90 tx=0x7f035c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=525818a162131e3f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:37.894 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.893+0000 7f0363e00640 1 -- 172.21.15.121:0/527880322 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f035c039070 con 0x7f036406cde0 2024-04-20T10:22:37.895 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.893+0000 7f0363e00640 1 -- 172.21.15.121:0/527880322 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f035c02cc60 con 0x7f036406cde0 2024-04-20T10:22:37.895 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.893+0000 7f0363e00640 1 -- 172.21.15.121:0/527880322 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f035c034660 con 0x7f036406cde0 2024-04-20T10:22:37.895 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.894+0000 7f036a800640 1 -- 172.21.15.121:0/527880322 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f036406cde0 msgr2=0x7f03640b3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:37.895 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.894+0000 7f036a800640 1 --2- 172.21.15.121:0/527880322 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f036406cde0 0x7f03640b3880 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f035c009d90 tx=0x7f035c02beb0 comp rx=0 tx=0).stop 2024-04-20T10:22:37.895 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.894+0000 7f036a800640 1 -- 172.21.15.121:0/527880322 shutdown_connections 2024-04-20T10:22:37.895 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.894+0000 7f036a800640 1 --2- 172.21.15.121:0/527880322 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f036406cde0 0x7f03640b3880 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:37.895 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.894+0000 7f036a800640 1 --2- 172.21.15.121:0/527880322 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0364161d50 0x7f03640d38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:37.895 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.894+0000 7f036a800640 1 -- 172.21.15.121:0/527880322 >> 172.21.15.121:0/527880322 conn(0x7f03640d4450 msgr2=0x7f03640d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:37.895 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.894+0000 7f036a800640 1 -- 172.21.15.121:0/527880322 shutdown_connections 2024-04-20T10:22:37.895 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.894+0000 7f036a800640 1 -- 172.21.15.121:0/527880322 wait complete. 2024-04-20T10:22:37.896 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.896+0000 7f036a800640 1 Processor -- start 2024-04-20T10:22:37.896 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.896+0000 7f036a800640 1 -- start start 2024-04-20T10:22:37.898 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.897+0000 7f036a800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f036406cde0 0x7f036414f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:37.898 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.897+0000 7f036a800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03640b3330 0x7f0364159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:37.898 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.897+0000 7f036a800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0364161d50 0x7f03640cd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:37.898 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.897+0000 7f036a800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f03640b7660 con 0x7f036406cde0 2024-04-20T10:22:37.898 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.897+0000 7f036a800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f03640d33c0 con 0x7f03640b3330 2024-04-20T10:22:37.898 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.897+0000 7f036a800640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f03640b77e0 con 0x7f0364161d50 2024-04-20T10:22:37.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.897+0000 7f0369400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f036406cde0 0x7f036414f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:37.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.897+0000 7f0369400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f036406cde0 0x7f036414f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34230/0 (socket says 172.21.15.121:34230) 2024-04-20T10:22:37.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.897+0000 7f0369400640 1 -- 172.21.15.121:0/3116316725 learned_addr learned my addr 172.21.15.121:0/3116316725 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:37.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.897+0000 7f0369e00640 1 --2- 172.21.15.121:0/3116316725 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0364161d50 0x7f03640cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:37.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.898+0000 7f0369400640 1 -- 172.21.15.121:0/3116316725 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0364161d50 msgr2=0x7f03640cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:37.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.898+0000 7f0369400640 1 --2- 172.21.15.121:0/3116316725 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0364161d50 0x7f03640cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:37.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.898+0000 7f0369400640 1 -- 172.21.15.121:0/3116316725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03640b3330 msgr2=0x7f0364159370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:37.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.898+0000 7f0369400640 1 --2- 172.21.15.121:0/3116316725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03640b3330 0x7f0364159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:37.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.898+0000 7f0369400640 1 -- 172.21.15.121:0/3116316725 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f036415a6d0 con 0x7f036406cde0 2024-04-20T10:22:37.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.898+0000 7f0369e00640 1 --2- 172.21.15.121:0/3116316725 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0364161d50 0x7f03640cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:37.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.898+0000 7f0369400640 1 --2- 172.21.15.121:0/3116316725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f036406cde0 0x7f036414f6e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f035800d620 tx=0x7f035800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:37.900 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.898+0000 7f0361600640 1 -- 172.21.15.121:0/3116316725 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03580125f0 con 0x7f036406cde0 2024-04-20T10:22:37.900 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.898+0000 7f0361600640 1 -- 172.21.15.121:0/3116316725 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0358019070 con 0x7f036406cde0 2024-04-20T10:22:37.900 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.898+0000 7f0361600640 1 -- 172.21.15.121:0/3116316725 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03580129d0 con 0x7f036406cde0 2024-04-20T10:22:37.900 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.898+0000 7f036a800640 1 -- 172.21.15.121:0/3116316725 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0364159e30 con 0x7f036406cde0 2024-04-20T10:22:37.900 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.898+0000 7f036a800640 1 -- 172.21.15.121:0/3116316725 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f036415a3f0 con 0x7f036406cde0 2024-04-20T10:22:37.900 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.899+0000 7f036a800640 1 -- 172.21.15.121:0/3116316725 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0364075b60 con 0x7f036406cde0 2024-04-20T10:22:37.901 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.900+0000 7f0361600640 1 -- 172.21.15.121:0/3116316725 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0358012b70 con 0x7f036406cde0 2024-04-20T10:22:37.902 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.901+0000 7f0361600640 1 --2- 172.21.15.121:0/3116316725 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f034406c8f0 0x7f034406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:37.902 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.901+0000 7f0361600640 1 -- 172.21.15.121:0/3116316725 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f035808c310 con 0x7f036406cde0 2024-04-20T10:22:37.902 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.902+0000 7f0368a00640 1 --2- 172.21.15.121:0/3116316725 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f034406c8f0 0x7f034406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:37.903 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.902+0000 7f0368a00640 1 --2- 172.21.15.121:0/3116316725 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f034406c8f0 0x7f034406edb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0350005880 tx=0x7f0350007950 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:37.906 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:37.906+0000 7f0361600640 1 -- 172.21.15.121:0/3116316725 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f035805a330 con 0x7f036406cde0 2024-04-20T10:22:38.106 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.105+0000 7f036a800640 1 -- 172.21.15.121:0/3116316725 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f036406cba0 con 0x7f036406cde0 2024-04-20T10:22:38.108 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.108+0000 7f0361600640 1 -- 172.21.15.121:0/3116316725 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18940 (secure 0 0 0) 0x7f0358059c30 con 0x7f036406cde0 2024-04-20T10:22:38.109 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:22:38.109 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":27,"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","created":"2024-04-20T10:20:38.076076+0000","modified":"2024-04-20T10:21:44.715898+0000","last_up_change":"2024-04-20T10:20:50.162700+0000","last_in_change":"2024-04-20T10:20:41.510382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:20:51.141365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:21:00.436937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:21:01.026957+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:21:04.862693+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"629a7b8c-a40a-4a2a-a3cc-d9f357774310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6809","nonce":1044588835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6811","nonce":1044588835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6815","nonce":1044588835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6813","nonce":1044588835}]},"public_addr":"172.21.15.121:6809/1044588835","cluster_addr":"172.21.15.121:6811/1044588835","heartbeat_back_addr":"172.21.15.121:6815/1044588835","heartbeat_front_addr":"172.21.15.121:6813/1044588835","state":["exists","up"]},{"osd":1,"uuid":"f2570399-aa50-4ff3-bf5d-3d09207290db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6801","nonce":3505199624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6803","nonce":3505199624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6807","nonce":3505199624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6805","nonce":3505199624}]},"public_addr":"172.21.15.121:6801/3505199624","cluster_addr":"172.21.15.121:6803/3505199624","heartbeat_back_addr":"172.21.15.121:6807/3505199624","heartbeat_front_addr":"172.21.15.121:6805/3505199624","state":["exists","up"]},{"osd":2,"uuid":"50893807-dc30-49f4-b41f-739366626658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6817","nonce":3656724207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6819","nonce":3656724207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6827","nonce":3656724207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6822","nonce":3656724207}]},"public_addr":"172.21.15.121:6817/3656724207","cluster_addr":"172.21.15.121:6819/3656724207","heartbeat_back_addr":"172.21.15.121:6827/3656724207","heartbeat_front_addr":"172.21.15.121:6822/3656724207","state":["exists","up"]},{"osd":3,"uuid":"dadb01bc-5491-4ffc-8824-d48a057240b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6823","nonce":2232342832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6826","nonce":2232342832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6831","nonce":2232342832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6829","nonce":2232342832}]},"public_addr":"172.21.15.121:6823/2232342832","cluster_addr":"172.21.15.121:6826/2232342832","heartbeat_back_addr":"172.21.15.121:6831/2232342832","heartbeat_front_addr":"172.21.15.121:6829/2232342832","state":["exists","up"]},{"osd":4,"uuid":"b3f3b4a1-b375-4526-b69e-344ab24c2da1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6801","nonce":99353111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6803","nonce":99353111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6807","nonce":99353111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6805","nonce":99353111}]},"public_addr":"172.21.15.123:6801/99353111","cluster_addr":"172.21.15.123:6803/99353111","heartbeat_back_addr":"172.21.15.123:6807/99353111","heartbeat_front_addr":"172.21.15.123:6805/99353111","state":["exists","up"]},{"osd":5,"uuid":"2ee24a08-1624-4961-9642-d7b510282484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6818","nonce":2504374688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6822","nonce":2504374688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6830","nonce":2504374688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6826","nonce":2504374688}]},"public_addr":"172.21.15.123:6818/2504374688","cluster_addr":"172.21.15.123:6822/2504374688","heartbeat_back_addr":"172.21.15.123:6830/2504374688","heartbeat_front_addr":"172.21.15.123:6826/2504374688","state":["exists","up"]},{"osd":6,"uuid":"186118cc-6128-49de-acca-55260945c1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6819","nonce":4139299599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6823","nonce":4139299599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6829","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6831","nonce":4139299599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6827","nonce":4139299599}]},"public_addr":"172.21.15.123:6819/4139299599","cluster_addr":"172.21.15.123:6823/4139299599","heartbeat_back_addr":"172.21.15.123:6831/4139299599","heartbeat_front_addr":"172.21.15.123:6827/4139299599","state":["exists","up"]},{"osd":7,"uuid":"a00aac6e-4bd9-415a-a93d-ae85210865ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6809","nonce":4043981069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6811","nonce":4043981069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6815","nonce":4043981069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6813","nonce":4043981069}]},"public_addr":"172.21.15.123:6809/4043981069","cluster_addr":"172.21.15.123:6811/4043981069","heartbeat_back_addr":"172.21.15.123:6815/4043981069","heartbeat_front_addr":"172.21.15.123:6813/4043981069","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.904540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.958585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.616367+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.885244+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.103127+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.167248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.884348+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.862356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.23","mappings":[{"from":3,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"3.37","mappings":[{"from":7,"to":1}]},{"pgid":"3.3c","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.17","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:22:38.111 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.110+0000 7f036a800640 1 -- 172.21.15.121:0/3116316725 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f034406c8f0 msgr2=0x7f034406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:38.111 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.110+0000 7f036a800640 1 --2- 172.21.15.121:0/3116316725 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f034406c8f0 0x7f034406edb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0350005880 tx=0x7f0350007950 comp rx=0 tx=0).stop 2024-04-20T10:22:38.111 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.110+0000 7f036a800640 1 -- 172.21.15.121:0/3116316725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f036406cde0 msgr2=0x7f036414f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:38.111 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.110+0000 7f036a800640 1 --2- 172.21.15.121:0/3116316725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f036406cde0 0x7f036414f6e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f035800d620 tx=0x7f035800dae0 comp rx=0 tx=0).stop 2024-04-20T10:22:38.111 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.110+0000 7f036a800640 1 -- 172.21.15.121:0/3116316725 shutdown_connections 2024-04-20T10:22:38.112 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.110+0000 7f036a800640 1 --2- 172.21.15.121:0/3116316725 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0364161d50 0x7f03640cd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:38.112 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.110+0000 7f036a800640 1 --2- 172.21.15.121:0/3116316725 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f034406c8f0 0x7f034406edb0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:38.112 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.110+0000 7f036a800640 1 --2- 172.21.15.121:0/3116316725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03640b3330 0x7f0364159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:38.112 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.110+0000 7f036a800640 1 --2- 172.21.15.121:0/3116316725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f036406cde0 0x7f036414f6e0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:38.112 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.111+0000 7f036a800640 1 -- 172.21.15.121:0/3116316725 >> 172.21.15.121:0/3116316725 conn(0x7f03640d4450 msgr2=0x7f03640cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:38.112 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.111+0000 7f036a800640 1 -- 172.21.15.121:0/3116316725 shutdown_connections 2024-04-20T10:22:38.112 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.111+0000 7f036a800640 1 -- 172.21.15.121:0/3116316725 wait complete. 2024-04-20T10:22:38.130 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-20T10:22:38.327 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.326+0000 7f4bbfe00640 1 Processor -- start 2024-04-20T10:22:38.327 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.327+0000 7f4bbfe00640 1 -- start start 2024-04-20T10:22:38.329 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.328+0000 7f4bbfe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4bc015b400 0x7f4bc0072d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:38.329 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.328+0000 7f4bbfe00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4bc015eec0 0x7f4bc015f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:38.329 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.328+0000 7f4bbfe00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bc015fcd0 con 0x7f4bc015f870 2024-04-20T10:22:38.329 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.328+0000 7f4bbfe00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f4bc006d280 con 0x7f4bc015eec0 2024-04-20T10:22:38.329 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.328+0000 7f4bbfe00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f4bc006ccc0 con 0x7f4bc015b400 2024-04-20T10:22:38.329 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.328+0000 7f4bb7e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4bc015eec0 0x7f4bc015f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:38.330 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.328+0000 7f4bb7e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4bc015eec0 0x7f4bc015f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34236/0 (socket says 172.21.15.121:34236) 2024-04-20T10:22:38.330 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.328+0000 7f4bb7e00640 1 -- 172.21.15.121:0/3278579406 learned_addr learned my addr 172.21.15.121:0/3278579406 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:38.330 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.329+0000 7f4bb7e00640 1 -- 172.21.15.121:0/3278579406 >> v2:172.21.15.123:3300/0 conn(0x7f4bc015b400 msgr2=0x7f4bc0072d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:38.330 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.329+0000 7f4bb7e00640 1 --2- 172.21.15.121:0/3278579406 >> v2:172.21.15.123:3300/0 conn(0x7f4bc015b400 0x7f4bc0072d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:38.330 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.329+0000 7f4bb7e00640 1 -- 172.21.15.121:0/3278579406 >> v1:172.21.15.123:6789/0 conn(0x7f4bc015f870 legacy=0x7f4bc015b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:38.330 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.329+0000 7f4bb7e00640 1 -- 172.21.15.121:0/3278579406 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bc006cfa0 con 0x7f4bc015eec0 2024-04-20T10:22:38.331 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.330+0000 7f4bb7e00640 1 --2- 172.21.15.121:0/3278579406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bc015eec0 0x7f4bc015f330 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0074060 tx=0x7f4ba802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d941d94b7f5e731 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:38.331 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.330+0000 7f4bbe000640 1 -- 172.21.15.121:0/3278579406 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba8035020 con 0x7f4bc015eec0 2024-04-20T10:22:38.331 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.330+0000 7f4bbe000640 1 -- 172.21.15.121:0/3278579406 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ba8031870 con 0x7f4bc015eec0 2024-04-20T10:22:38.331 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.330+0000 7f4bbe000640 1 -- 172.21.15.121:0/3278579406 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba8031c40 con 0x7f4bc015eec0 2024-04-20T10:22:38.331 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.330+0000 7f4bbfe00640 1 -- 172.21.15.121:0/3278579406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bc015eec0 msgr2=0x7f4bc015f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:38.331 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.330+0000 7f4bbfe00640 1 --2- 172.21.15.121:0/3278579406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bc015eec0 0x7f4bc015f330 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0074060 tx=0x7f4ba802b910 comp rx=0 tx=0).stop 2024-04-20T10:22:38.332 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.331+0000 7f4bbfe00640 1 -- 172.21.15.121:0/3278579406 shutdown_connections 2024-04-20T10:22:38.332 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.331+0000 7f4bbfe00640 1 --2- 172.21.15.121:0/3278579406 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bc015eec0 0x7f4bc015f330 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:38.332 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.331+0000 7f4bbfe00640 1 --2- 172.21.15.121:0/3278579406 >> v2:172.21.15.123:3300/0 conn(0x7f4bc015b400 0x7f4bc0072d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:38.332 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.331+0000 7f4bbfe00640 1 -- 172.21.15.121:0/3278579406 >> 172.21.15.121:0/3278579406 conn(0x7f4bc00d0040 msgr2=0x7f4bc00d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:38.332 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.331+0000 7f4bbfe00640 1 -- 172.21.15.121:0/3278579406 shutdown_connections 2024-04-20T10:22:38.332 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.331+0000 7f4bbfe00640 1 -- 172.21.15.121:0/3278579406 wait complete. 2024-04-20T10:22:38.332 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.332+0000 7f4bbfe00640 1 Processor -- start 2024-04-20T10:22:38.333 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.332+0000 7f4bbfe00640 1 -- start start 2024-04-20T10:22:38.334 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.333+0000 7f4bbfe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4bc015b400 0x7f4bc0155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:38.335 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.333+0000 7f4bbfe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bc015eec0 0x7f4bc01567b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:38.335 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.333+0000 7f4bbfe00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4bc015f870 0x7f4bc009d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:38.335 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.333+0000 7f4bbfe00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc0073240 con 0x7f4bc015eec0 2024-04-20T10:22:38.335 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.333+0000 7f4bbfe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc006e720 con 0x7f4bc015b400 2024-04-20T10:22:38.335 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.333+0000 7f4bbfe00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f4bc006fc50 con 0x7f4bc015f870 2024-04-20T10:22:38.335 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.334+0000 7f4bb7e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bc015eec0 0x7f4bc01567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:38.335 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.334+0000 7f4bb7e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bc015eec0 0x7f4bc01567b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34244/0 (socket says 172.21.15.121:34244) 2024-04-20T10:22:38.335 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.334+0000 7f4bb7e00640 1 -- 172.21.15.121:0/3938820327 learned_addr learned my addr 172.21.15.121:0/3938820327 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:38.335 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.334+0000 7f4bbea00640 1 --2- 172.21.15.121:0/3938820327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4bc015b400 0x7f4bc0155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:38.335 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.334+0000 7f4bb7e00640 1 -- 172.21.15.121:0/3938820327 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4bc015f870 msgr2=0x7f4bc009d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:38.335 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.334+0000 7f4bb7e00640 1 --2- 172.21.15.121:0/3938820327 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4bc015f870 0x7f4bc009d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:38.335 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.334+0000 7f4bb7e00640 1 -- 172.21.15.121:0/3938820327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4bc015b400 msgr2=0x7f4bc0155f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:38.335 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.334+0000 7f4bb7e00640 1 --2- 172.21.15.121:0/3938820327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4bc015b400 0x7f4bc0155f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:38.335 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.334+0000 7f4bb7e00640 1 -- 172.21.15.121:0/3938820327 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bc009d980 con 0x7f4bc015eec0 2024-04-20T10:22:38.336 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.334+0000 7f4bb7e00640 1 --2- 172.21.15.121:0/3938820327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bc015eec0 0x7f4bc01567b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8000c00 tx=0x7f4ba80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:38.336 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.335+0000 7f4bb6a00640 1 -- 172.21.15.121:0/3938820327 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba803ad60 con 0x7f4bc015eec0 2024-04-20T10:22:38.337 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.335+0000 7f4bb6a00640 1 -- 172.21.15.121:0/3938820327 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ba80443f0 con 0x7f4bc015eec0 2024-04-20T10:22:38.337 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.335+0000 7f4bb6a00640 1 -- 172.21.15.121:0/3938820327 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba80447e0 con 0x7f4bc015eec0 2024-04-20T10:22:38.337 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.335+0000 7f4bbfe00640 1 -- 172.21.15.121:0/3938820327 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bc0156ee0 con 0x7f4bc015eec0 2024-04-20T10:22:38.337 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.335+0000 7f4bbfe00640 1 -- 172.21.15.121:0/3938820327 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bc01574a0 con 0x7f4bc015eec0 2024-04-20T10:22:38.337 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.336+0000 7f4bbfe00640 1 -- 172.21.15.121:0/3938820327 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bc0075b60 con 0x7f4bc015eec0 2024-04-20T10:22:38.337 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.337+0000 7f4bb6a00640 1 -- 172.21.15.121:0/3938820327 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4ba804b060 con 0x7f4bc015eec0 2024-04-20T10:22:38.338 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.338+0000 7f4bb6a00640 1 --2- 172.21.15.121:0/3938820327 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f4b9c06c8f0 0x7f4b9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:38.338 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.338+0000 7f4bb6a00640 1 -- 172.21.15.121:0/3938820327 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f4ba80ae2c0 con 0x7f4bc015eec0 2024-04-20T10:22:38.339 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.338+0000 7f4bbea00640 1 --2- 172.21.15.121:0/3938820327 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f4b9c06c8f0 0x7f4b9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:38.339 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.339+0000 7f4bbea00640 1 --2- 172.21.15.121:0/3938820327 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f4b9c06c8f0 0x7f4b9c06edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4bc015a710 tx=0x7f4bb0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:38.343 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.343+0000 7f4bb6a00640 1 -- 172.21.15.121:0/3938820327 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4ba803f070 con 0x7f4bc015eec0 2024-04-20T10:22:38.563 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.563+0000 7f4bbfe00640 1 -- 172.21.15.121:0/3938820327 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f4bc00cfdb0 con 0x7f4bc015eec0 2024-04-20T10:22:38.565 INFO:tasks.ceph.mon.a.smithi121.stderr:2024-04-20T10:22:38.564+0000 7f4470000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T10:22:38.569 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.568+0000 7f4bb6a00640 1 -- 172.21.15.121:0/3938820327 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7f4ba802f320 con 0x7f4bc015eec0 2024-04-20T10:22:38.569 INFO:teuthology.orchestra.run.smithi121.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-20T10:22:38.571 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.571+0000 7f4bbfe00640 1 -- 172.21.15.121:0/3938820327 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f4b9c06c8f0 msgr2=0x7f4b9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:38.571 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.571+0000 7f4bbfe00640 1 --2- 172.21.15.121:0/3938820327 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f4b9c06c8f0 0x7f4b9c06edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4bc015a710 tx=0x7f4bb0008040 comp rx=0 tx=0).stop 2024-04-20T10:22:38.572 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.571+0000 7f4bbfe00640 1 -- 172.21.15.121:0/3938820327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bc015eec0 msgr2=0x7f4bc01567b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:38.573 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.571+0000 7f4bbfe00640 1 --2- 172.21.15.121:0/3938820327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bc015eec0 0x7f4bc01567b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f4ba8000c00 tx=0x7f4ba80046c0 comp rx=0 tx=0).stop 2024-04-20T10:22:38.573 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.571+0000 7f4bbfe00640 1 -- 172.21.15.121:0/3938820327 shutdown_connections 2024-04-20T10:22:38.573 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.571+0000 7f4bbfe00640 1 --2- 172.21.15.121:0/3938820327 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4bc015f870 0x7f4bc009d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:38.573 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.571+0000 7f4bbfe00640 1 --2- 172.21.15.121:0/3938820327 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4bc015eec0 0x7f4bc01567b0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:38.573 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.571+0000 7f4bbfe00640 1 --2- 172.21.15.121:0/3938820327 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f4b9c06c8f0 0x7f4b9c06edb0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:38.573 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.571+0000 7f4bbfe00640 1 --2- 172.21.15.121:0/3938820327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4bc015b400 0x7f4bc0155f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:38.573 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.571+0000 7f4bbfe00640 1 -- 172.21.15.121:0/3938820327 >> 172.21.15.121:0/3938820327 conn(0x7f4bc00d0040 msgr2=0x7f4bc006b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:38.573 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.571+0000 7f4bbfe00640 1 -- 172.21.15.121:0/3938820327 shutdown_connections 2024-04-20T10:22:38.573 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.571+0000 7f4bbfe00640 1 -- 172.21.15.121:0/3938820327 wait complete. 2024-04-20T10:22:38.592 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-20T10:22:38.605 INFO:tasks.ceph.mds.d.smithi123.stderr: -6080> 2024-04-20T10:21:03.555+0000 7f52c3800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T10:22:38.698 INFO:tasks.ceph.mds.d.smithi123.stdout:starting mds.d at 2024-04-20T10:22:38.785 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.785+0000 7f4a37200640 1 Processor -- start 2024-04-20T10:22:38.785 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.785+0000 7f4a37200640 1 -- start start 2024-04-20T10:22:38.787 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.786+0000 7f4a37200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4a3815d4f0 0x7f4a380ad0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:38.787 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.786+0000 7f4a37200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a3806b900 con 0x7f4a3806e060 2024-04-20T10:22:38.787 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.786+0000 7f4a37200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a3806bbe0 con 0x7f4a380aee60 2024-04-20T10:22:38.787 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.786+0000 7f4a37200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f4a3806c510 con 0x7f4a3815d4f0 2024-04-20T10:22:38.788 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.787+0000 7f4a36800640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f4a3806e060 0x7f4a3806e4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:46182/0 (socket says 172.21.15.121:46182) 2024-04-20T10:22:38.788 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.787+0000 7f4a36800640 1 -- 172.21.15.121:0/4093818185 learned_addr learned my addr 172.21.15.121:0/4093818185 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:38.788 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.787+0000 7f4a35400640 1 --2- 172.21.15.121:0/4093818185 >> v2:172.21.15.123:3300/0 conn(0x7f4a3815d4f0 0x7f4a380ad0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:38.790 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.787+0000 7f4a34a00640 1 -- 172.21.15.121:0/4093818185 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1373143300 0 0) 0x7f4a3806b900 con 0x7f4a3806e060 2024-04-20T10:22:38.790 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.787+0000 7f4a34a00640 1 -- 172.21.15.121:0/4093818185 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a1c003520 con 0x7f4a3806e060 2024-04-20T10:22:38.790 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.788+0000 7f4a34a00640 1 -- 172.21.15.121:0/4093818185 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3703928677 0 0) 0x7f4a1c003520 con 0x7f4a3806e060 2024-04-20T10:22:38.790 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.788+0000 7f4a34a00640 1 -- 172.21.15.121:0/4093818185 >> v2:172.21.15.123:3300/0 conn(0x7f4a3815d4f0 msgr2=0x7f4a380ad0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:38.790 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.788+0000 7f4a34a00640 1 --2- 172.21.15.121:0/4093818185 >> v2:172.21.15.123:3300/0 conn(0x7f4a3815d4f0 0x7f4a380ad0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:38.790 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.788+0000 7f4a34a00640 1 -- 172.21.15.121:0/4093818185 >> v1:172.21.15.123:6789/0 conn(0x7f4a380aee60 legacy=0x7f4a380af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:38.790 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.788+0000 7f4a34a00640 1 -- 172.21.15.121:0/4093818185 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a3806bec0 con 0x7f4a3806e060 2024-04-20T10:22:38.791 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.788+0000 7f4a34a00640 1 -- 172.21.15.121:0/4093818185 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f4a300019f0 con 0x7f4a3806e060 2024-04-20T10:22:38.791 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.788+0000 7f4a34a00640 1 -- 172.21.15.121:0/4093818185 >> v1:172.21.15.121:6789/0 conn(0x7f4a3806e060 legacy=0x7f4a3806e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:38.791 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.788+0000 7f4a34a00640 1 --2- 172.21.15.121:0/4093818185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a1c0039b0 0x7f4a1c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:38.791 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.788+0000 7f4a34a00640 1 --2- 172.21.15.121:0/4093818185 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a1c004800 0x7f4a1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:38.791 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.788+0000 7f4a34a00640 1 --2- 172.21.15.121:0/4093818185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a1c0051c0 0x7f4a1c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:38.791 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.788+0000 7f4a34a00640 1 -- 172.21.15.121:0/4093818185 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3806c510 con 0x7f4a1c0051c0 2024-04-20T10:22:38.791 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.788+0000 7f4a34a00640 1 -- 172.21.15.121:0/4093818185 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1c00c060 con 0x7f4a1c0039b0 2024-04-20T10:22:38.791 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.788+0000 7f4a34a00640 1 -- 172.21.15.121:0/4093818185 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f4a1c00c260 con 0x7f4a1c004800 2024-04-20T10:22:38.791 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.788+0000 7f4a36800640 1 --2- 172.21.15.121:0/4093818185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a1c0051c0 0x7f4a1c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:38.791 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.789+0000 7f4a35400640 1 --2- 172.21.15.121:0/4093818185 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a1c004800 0x7f4a1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:38.791 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.789+0000 7f4a36800640 1 -- 172.21.15.121:0/4093818185 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a1c004800 msgr2=0x7f4a1c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:38.791 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.789+0000 7f4a36800640 1 --2- 172.21.15.121:0/4093818185 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a1c004800 0x7f4a1c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:38.791 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.789+0000 7f4a36800640 1 -- 172.21.15.121:0/4093818185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a1c0039b0 msgr2=0x7f4a1c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:38.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.789+0000 7f4a35e00640 1 --2- 172.21.15.121:0/4093818185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a1c0039b0 0x7f4a1c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:38.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.789+0000 7f4a36800640 1 --2- 172.21.15.121:0/4093818185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a1c0039b0 0x7f4a1c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:38.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.789+0000 7f4a36800640 1 -- 172.21.15.121:0/4093818185 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a1c003520 con 0x7f4a1c0051c0 2024-04-20T10:22:38.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.789+0000 7f4a35e00640 1 --2- 172.21.15.121:0/4093818185 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a1c0039b0 0x7f4a1c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:38.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.790+0000 7f4a36800640 1 -- 172.21.15.121:0/4093818185 reap_dead start 2024-04-20T10:22:38.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.790+0000 7f4a36800640 1 --2- 172.21.15.121:0/4093818185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a1c0051c0 0x7f4a1c00b9c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4a3000a840 tx=0x7f4a1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e573d2bc9879045 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:38.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.790+0000 7f4a34a00640 1 -- 172.21.15.121:0/4093818185 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3000a460 con 0x7f4a1c0051c0 2024-04-20T10:22:38.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.790+0000 7f4a34a00640 1 -- 172.21.15.121:0/4093818185 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a30034650 con 0x7f4a1c0051c0 2024-04-20T10:22:38.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.790+0000 7f4a34a00640 1 -- 172.21.15.121:0/4093818185 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a30034a40 con 0x7f4a1c0051c0 2024-04-20T10:22:38.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.791+0000 7f4a37200640 1 -- 172.21.15.121:0/4093818185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a1c0051c0 msgr2=0x7f4a1c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:38.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.791+0000 7f4a37200640 1 --2- 172.21.15.121:0/4093818185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a1c0051c0 0x7f4a1c00b9c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4a3000a840 tx=0x7f4a1c005830 comp rx=0 tx=0).stop 2024-04-20T10:22:38.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.791+0000 7f4a37200640 1 -- 172.21.15.121:0/4093818185 shutdown_connections 2024-04-20T10:22:38.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.791+0000 7f4a37200640 1 --2- 172.21.15.121:0/4093818185 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a1c0051c0 0x7f4a1c00b9c0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:38.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.791+0000 7f4a37200640 1 -- 172.21.15.121:0/4093818185 >> 172.21.15.121:0/4093818185 conn(0x7f4a380b5270 msgr2=0x7f4a380b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:38.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.791+0000 7f4a37200640 1 -- 172.21.15.121:0/4093818185 shutdown_connections 2024-04-20T10:22:38.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.791+0000 7f4a37200640 1 -- 172.21.15.121:0/4093818185 wait complete. 2024-04-20T10:22:38.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.792+0000 7f4a37200640 1 Processor -- start 2024-04-20T10:22:38.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.793+0000 7f4a37200640 1 -- start start 2024-04-20T10:22:38.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.793+0000 7f4a37200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a1c0051c0 0x7f4a3809aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:38.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.794+0000 7f4a37200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a3814e190 0x7f4a381f4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:38.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.794+0000 7f4a37200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a380afba0 0x7f4a380b0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:38.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.794+0000 7f4a37200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4a380af240 con 0x7f4a1c0051c0 2024-04-20T10:22:38.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.794+0000 7f4a37200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3806c9a0 con 0x7f4a3814e190 2024-04-20T10:22:38.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.794+0000 7f4a37200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f4a380af3c0 con 0x7f4a380afba0 2024-04-20T10:22:38.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.794+0000 7f4a35e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a1c0051c0 0x7f4a3809aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:38.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.794+0000 7f4a35e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a1c0051c0 0x7f4a3809aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34258/0 (socket says 172.21.15.121:34258) 2024-04-20T10:22:38.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.794+0000 7f4a35e00640 1 -- 172.21.15.121:0/801339115 learned_addr learned my addr 172.21.15.121:0/801339115 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:38.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.794+0000 7f4a35400640 1 --2- 172.21.15.121:0/801339115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a3814e190 0x7f4a381f4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:38.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.794+0000 7f4a36800640 1 --2- 172.21.15.121:0/801339115 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a380afba0 0x7f4a380b0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:38.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.794+0000 7f4a35e00640 1 -- 172.21.15.121:0/801339115 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a380afba0 msgr2=0x7f4a380b0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:38.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.794+0000 7f4a35e00640 1 --2- 172.21.15.121:0/801339115 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a380afba0 0x7f4a380b0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:38.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.794+0000 7f4a35e00640 1 -- 172.21.15.121:0/801339115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a3814e190 msgr2=0x7f4a381f4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:38.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.794+0000 7f4a35e00640 1 --2- 172.21.15.121:0/801339115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a3814e190 0x7f4a381f4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:38.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.794+0000 7f4a35e00640 1 -- 172.21.15.121:0/801339115 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a381f53e0 con 0x7f4a1c0051c0 2024-04-20T10:22:38.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.795+0000 7f4a35e00640 1 --2- 172.21.15.121:0/801339115 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a1c0051c0 0x7f4a3809aed0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f4a2800d560 tx=0x7f4a2800da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:38.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.795+0000 7f4a2e000640 1 -- 172.21.15.121:0/801339115 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a28005480 con 0x7f4a1c0051c0 2024-04-20T10:22:38.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.795+0000 7f4a2e000640 1 -- 172.21.15.121:0/801339115 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a28017070 con 0x7f4a1c0051c0 2024-04-20T10:22:38.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.795+0000 7f4a2e000640 1 -- 172.21.15.121:0/801339115 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a28003810 con 0x7f4a1c0051c0 2024-04-20T10:22:38.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.795+0000 7f4a37200640 1 -- 172.21.15.121:0/801339115 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a381f56c0 con 0x7f4a1c0051c0 2024-04-20T10:22:38.798 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.795+0000 7f4a37200640 1 -- 172.21.15.121:0/801339115 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a381f5c80 con 0x7f4a1c0051c0 2024-04-20T10:22:38.798 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.796+0000 7f4a37200640 1 -- 172.21.15.121:0/801339115 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a38075b30 con 0x7f4a1c0051c0 2024-04-20T10:22:38.798 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.797+0000 7f4a2e000640 1 -- 172.21.15.121:0/801339115 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4a280039b0 con 0x7f4a1c0051c0 2024-04-20T10:22:38.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.798+0000 7f4a2e000640 1 --2- 172.21.15.121:0/801339115 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f4a1806c8f0 0x7f4a1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:38.802 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.798+0000 7f4a2e000640 1 -- 172.21.15.121:0/801339115 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6821+0+0 (secure 0 0 0) 0x7f4a2808bdf0 con 0x7f4a1c0051c0 2024-04-20T10:22:38.802 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.798+0000 7f4a35400640 1 --2- 172.21.15.121:0/801339115 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f4a1806c8f0 0x7f4a1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:38.803 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.799+0000 7f4a35400640 1 --2- 172.21.15.121:0/801339115 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f4a1806c8f0 0x7f4a1806edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4a380add60 tx=0x7f4a24008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:38.803 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:38.802+0000 7f4a2e000640 1 -- 172.21.15.121:0/801339115 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4a2800f2f0 con 0x7f4a1c0051c0 2024-04-20T10:22:39.031 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.030+0000 7f4a37200640 1 -- 172.21.15.121:0/801339115 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f4a3806de40 con 0x7f4a1c0051c0 2024-04-20T10:22:39.580 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.579+0000 7f4a2e000640 1 -- 172.21.15.121:0/801339115 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f4a28059d20 con 0x7f4a1c0051c0 2024-04-20T10:22:39.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.582+0000 7f4a37200640 1 -- 172.21.15.121:0/801339115 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f4a1806c8f0 msgr2=0x7f4a1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:39.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.582+0000 7f4a37200640 1 --2- 172.21.15.121:0/801339115 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f4a1806c8f0 0x7f4a1806edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4a380add60 tx=0x7f4a24008040 comp rx=0 tx=0).stop 2024-04-20T10:22:39.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.582+0000 7f4a37200640 1 -- 172.21.15.121:0/801339115 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a1c0051c0 msgr2=0x7f4a3809aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:39.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.582+0000 7f4a37200640 1 --2- 172.21.15.121:0/801339115 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a1c0051c0 0x7f4a3809aed0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f4a2800d560 tx=0x7f4a2800da20 comp rx=0 tx=0).stop 2024-04-20T10:22:39.584 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.582+0000 7f4a37200640 1 -- 172.21.15.121:0/801339115 shutdown_connections 2024-04-20T10:22:39.584 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.582+0000 7f4a37200640 1 --2- 172.21.15.121:0/801339115 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4a380afba0 0x7f4a380b0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:39.584 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.582+0000 7f4a37200640 1 --2- 172.21.15.121:0/801339115 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f4a1806c8f0 0x7f4a1806edb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:39.584 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.582+0000 7f4a37200640 1 --2- 172.21.15.121:0/801339115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a3814e190 0x7f4a381f4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:39.584 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.582+0000 7f4a37200640 1 --2- 172.21.15.121:0/801339115 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4a1c0051c0 0x7f4a3809aed0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:39.584 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.582+0000 7f4a37200640 1 -- 172.21.15.121:0/801339115 >> 172.21.15.121:0/801339115 conn(0x7f4a380b5270 msgr2=0x7f4a3815b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:39.584 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.583+0000 7f4a37200640 1 -- 172.21.15.121:0/801339115 shutdown_connections 2024-04-20T10:22:39.584 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.583+0000 7f4a37200640 1 -- 172.21.15.121:0/801339115 wait complete. 2024-04-20T10:22:39.607 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-20T10:22:39.810 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.809+0000 7fa04c000640 1 Processor -- start 2024-04-20T10:22:39.810 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.810+0000 7fa04c000640 1 -- start start 2024-04-20T10:22:39.812 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.811+0000 7fa04c000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa04415e8a0 0x7fa04406e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:39.812 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.811+0000 7fa04c000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa04406dbb0 0x7fa0440d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:39.812 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.811+0000 7fa04c000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa0440d08a0 0x7fa04406b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:39.812 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.811+0000 7fa04c000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fa04406f060 con 0x7fa04415e8a0 2024-04-20T10:22:39.812 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.811+0000 7fa04c000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fa04406e7c0 con 0x7fa0440d08a0 2024-04-20T10:22:39.812 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.811+0000 7fa04c000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fa04406eaa0 con 0x7fa04406dbb0 2024-04-20T10:22:39.813 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.811+0000 7fa049600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa04415e8a0 0x7fa04406e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:39.813 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.812+0000 7fa049600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fa04415e8a0 0x7fa04406e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34260/0 (socket says 172.21.15.121:34260) 2024-04-20T10:22:39.813 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.812+0000 7fa049600640 1 -- 172.21.15.121:0/2638570847 learned_addr learned my addr 172.21.15.121:0/2638570847 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:39.813 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.812+0000 7fa04a000640 1 --2- 172.21.15.121:0/2638570847 >> v2:172.21.15.123:3300/0 conn(0x7fa0440d08a0 0x7fa04406b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:39.813 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.812+0000 7fa048c00640 1 --2- 172.21.15.121:0/2638570847 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa04406dbb0 0x7fa0440d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:39.813 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.812+0000 7fa049600640 1 -- 172.21.15.121:0/2638570847 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa04406dbb0 msgr2=0x7fa0440d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:39.813 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.812+0000 7fa049600640 1 --2- 172.21.15.121:0/2638570847 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa04406dbb0 0x7fa0440d0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:39.814 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.812+0000 7fa049600640 1 -- 172.21.15.121:0/2638570847 >> v2:172.21.15.123:3300/0 conn(0x7fa0440d08a0 msgr2=0x7fa04406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:39.814 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.812+0000 7fa049600640 1 --2- 172.21.15.121:0/2638570847 >> v2:172.21.15.123:3300/0 conn(0x7fa0440d08a0 0x7fa04406b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:39.814 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.812+0000 7fa049600640 1 -- 172.21.15.121:0/2638570847 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa04406ed80 con 0x7fa04415e8a0 2024-04-20T10:22:39.814 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.812+0000 7fa04a000640 1 --2- 172.21.15.121:0/2638570847 >> v2:172.21.15.123:3300/0 conn(0x7fa0440d08a0 0x7fa04406b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:22:39.815 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.814+0000 7fa049600640 1 --2- 172.21.15.121:0/2638570847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa04415e8a0 0x7fa04406e1e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa04406d740 tx=0x7fa03802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fb62703c0a0f3d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:39.815 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.814+0000 7fa043e00640 1 -- 172.21.15.121:0/2638570847 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa038035020 con 0x7fa04415e8a0 2024-04-20T10:22:39.815 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.814+0000 7fa043e00640 1 -- 172.21.15.121:0/2638570847 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa038031870 con 0x7fa04415e8a0 2024-04-20T10:22:39.815 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.814+0000 7fa043e00640 1 -- 172.21.15.121:0/2638570847 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa038031c40 con 0x7fa04415e8a0 2024-04-20T10:22:39.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.815+0000 7fa04c000640 1 -- 172.21.15.121:0/2638570847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa04415e8a0 msgr2=0x7fa04406e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:39.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.815+0000 7fa04c000640 1 --2- 172.21.15.121:0/2638570847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa04415e8a0 0x7fa04406e1e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa04406d740 tx=0x7fa03802b910 comp rx=0 tx=0).stop 2024-04-20T10:22:39.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.815+0000 7fa04c000640 1 -- 172.21.15.121:0/2638570847 shutdown_connections 2024-04-20T10:22:39.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.815+0000 7fa04c000640 1 --2- 172.21.15.121:0/2638570847 >> v2:172.21.15.123:3300/0 conn(0x7fa0440d08a0 0x7fa04406b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:39.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.815+0000 7fa04c000640 1 --2- 172.21.15.121:0/2638570847 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa04406dbb0 0x7fa0440d0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:39.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.815+0000 7fa04c000640 1 --2- 172.21.15.121:0/2638570847 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa04415e8a0 0x7fa04406e1e0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:39.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.815+0000 7fa04c000640 1 -- 172.21.15.121:0/2638570847 >> 172.21.15.121:0/2638570847 conn(0x7fa0440b3350 msgr2=0x7fa0440b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:39.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.815+0000 7fa04c000640 1 -- 172.21.15.121:0/2638570847 shutdown_connections 2024-04-20T10:22:39.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.815+0000 7fa04c000640 1 -- 172.21.15.121:0/2638570847 wait complete. 2024-04-20T10:22:39.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.817+0000 7fa04c000640 1 Processor -- start 2024-04-20T10:22:39.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.817+0000 7fa04c000640 1 -- start start 2024-04-20T10:22:39.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.818+0000 7fa04c000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa04406dbb0 0x7fa0440ae090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:39.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.818+0000 7fa04c000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0440d08a0 0x7fa04409db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:39.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.818+0000 7fa04c000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa04415e8a0 0x7fa044148310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:39.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.818+0000 7fa04c000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fa04406fef0 con 0x7fa0440d08a0 2024-04-20T10:22:39.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.818+0000 7fa04c000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa04406fd70 con 0x7fa04406dbb0 2024-04-20T10:22:39.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.818+0000 7fa04c000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fa04406be40 con 0x7fa04415e8a0 2024-04-20T10:22:39.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.818+0000 7fa048c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0440d08a0 0x7fa04409db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:39.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.818+0000 7fa048c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0440d08a0 0x7fa04409db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34276/0 (socket says 172.21.15.121:34276) 2024-04-20T10:22:39.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.818+0000 7fa048c00640 1 -- 172.21.15.121:0/962555410 learned_addr learned my addr 172.21.15.121:0/962555410 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:39.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.818+0000 7fa049600640 1 --2- 172.21.15.121:0/962555410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa04406dbb0 0x7fa0440ae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:39.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.818+0000 7fa04a000640 1 --2- 172.21.15.121:0/962555410 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa04415e8a0 0x7fa044148310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:39.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.818+0000 7fa048c00640 1 -- 172.21.15.121:0/962555410 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa04415e8a0 msgr2=0x7fa044148310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:39.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.818+0000 7fa048c00640 1 --2- 172.21.15.121:0/962555410 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa04415e8a0 0x7fa044148310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:39.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.818+0000 7fa048c00640 1 -- 172.21.15.121:0/962555410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa04406dbb0 msgr2=0x7fa0440ae090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:39.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.818+0000 7fa048c00640 1 --2- 172.21.15.121:0/962555410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa04406dbb0 0x7fa0440ae090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:39.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.819+0000 7fa048c00640 1 -- 172.21.15.121:0/962555410 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0440ae750 con 0x7fa0440d08a0 2024-04-20T10:22:39.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.819+0000 7fa048c00640 1 --2- 172.21.15.121:0/962555410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0440d08a0 0x7fa04409db30 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa03000d800 tx=0x7fa03000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:39.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.819+0000 7fa041600640 1 -- 172.21.15.121:0/962555410 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa030018020 con 0x7fa0440d08a0 2024-04-20T10:22:39.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.819+0000 7fa041600640 1 -- 172.21.15.121:0/962555410 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0300096a0 con 0x7fa0440d08a0 2024-04-20T10:22:39.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.819+0000 7fa041600640 1 -- 172.21.15.121:0/962555410 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa030009a70 con 0x7fa0440d08a0 2024-04-20T10:22:39.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.819+0000 7fa04c000640 1 -- 172.21.15.121:0/962555410 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0440aea30 con 0x7fa0440d08a0 2024-04-20T10:22:39.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.819+0000 7fa04c000640 1 -- 172.21.15.121:0/962555410 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa044148850 con 0x7fa0440d08a0 2024-04-20T10:22:39.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.820+0000 7fa04c000640 1 -- 172.21.15.121:0/962555410 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa044068980 con 0x7fa0440d08a0 2024-04-20T10:22:39.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.821+0000 7fa041600640 1 -- 172.21.15.121:0/962555410 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa03001b020 con 0x7fa0440d08a0 2024-04-20T10:22:39.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.822+0000 7fa041600640 1 --2- 172.21.15.121:0/962555410 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fa02406c8f0 0x7fa02406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:39.823 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.822+0000 7fa041600640 1 -- 172.21.15.121:0/962555410 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6821+0+0 (secure 0 0 0) 0x7fa03008c2e0 con 0x7fa0440d08a0 2024-04-20T10:22:39.823 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.823+0000 7fa049600640 1 --2- 172.21.15.121:0/962555410 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fa02406c8f0 0x7fa02406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:39.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.823+0000 7fa049600640 1 --2- 172.21.15.121:0/962555410 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fa02406c8f0 0x7fa02406edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa038036000 tx=0x7fa03803c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:39.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:39.827+0000 7fa041600640 1 -- 172.21.15.121:0/962555410 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa030022030 con 0x7fa0440d08a0 2024-04-20T10:22:40.032 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.031+0000 7fa04c000640 1 -- 172.21.15.121:0/962555410 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa04406d160 con 0x7fa0440d08a0 2024-04-20T10:22:40.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.583+0000 7fa041600640 1 -- 172.21.15.121:0/962555410 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) ==== 181+0+0 (secure 0 0 0) 0x7fa03005a210 con 0x7fa0440d08a0 2024-04-20T10:22:40.584 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'cephfs_metadata' removed 2024-04-20T10:22:40.587 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.586+0000 7fa04c000640 1 -- 172.21.15.121:0/962555410 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fa02406c8f0 msgr2=0x7fa02406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:40.587 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.586+0000 7fa04c000640 1 --2- 172.21.15.121:0/962555410 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fa02406c8f0 0x7fa02406edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa038036000 tx=0x7fa03803c040 comp rx=0 tx=0).stop 2024-04-20T10:22:40.587 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.587+0000 7fa04c000640 1 -- 172.21.15.121:0/962555410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0440d08a0 msgr2=0x7fa04409db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:40.588 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.587+0000 7fa04c000640 1 --2- 172.21.15.121:0/962555410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0440d08a0 0x7fa04409db30 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa03000d800 tx=0x7fa03000dcc0 comp rx=0 tx=0).stop 2024-04-20T10:22:40.588 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.587+0000 7fa04c000640 1 -- 172.21.15.121:0/962555410 shutdown_connections 2024-04-20T10:22:40.588 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.587+0000 7fa04c000640 1 --2- 172.21.15.121:0/962555410 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fa04415e8a0 0x7fa044148310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:40.588 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.587+0000 7fa04c000640 1 --2- 172.21.15.121:0/962555410 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fa0440d08a0 0x7fa04409db30 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:40.588 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.587+0000 7fa04c000640 1 --2- 172.21.15.121:0/962555410 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fa02406c8f0 0x7fa02406edb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:40.588 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.587+0000 7fa04c000640 1 --2- 172.21.15.121:0/962555410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa04406dbb0 0x7fa0440ae090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:40.588 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.587+0000 7fa04c000640 1 -- 172.21.15.121:0/962555410 >> 172.21.15.121:0/962555410 conn(0x7fa0440b3350 msgr2=0x7fa04415d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:40.588 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.587+0000 7fa04c000640 1 -- 172.21.15.121:0/962555410 shutdown_connections 2024-04-20T10:22:40.589 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.587+0000 7fa04c000640 1 -- 172.21.15.121:0/962555410 wait complete. 2024-04-20T10:22:40.612 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-20T10:22:40.815 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.815+0000 7f4b93200640 1 Processor -- start 2024-04-20T10:22:40.815 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.815+0000 7f4b93200640 1 -- start start 2024-04-20T10:22:40.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.816+0000 7f4b93200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4b9415de30 0x7f4b940cdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:40.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.816+0000 7f4b93200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4b94071a10 0x7f4b94071ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:40.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.816+0000 7f4b93200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b94072400 0x7f4b94072840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:40.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.816+0000 7f4b93200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f4b94072d80 con 0x7f4b9415de30 2024-04-20T10:22:40.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.816+0000 7f4b93200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f4b9415e210 con 0x7f4b94071a10 2024-04-20T10:22:40.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.816+0000 7f4b93200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f4b940cb5f0 con 0x7f4b94072400 2024-04-20T10:22:40.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.816+0000 7f4b91e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4b9415de30 0x7f4b940cdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:40.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.816+0000 7f4b91e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f4b9415de30 0x7f4b940cdb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34290/0 (socket says 172.21.15.121:34290) 2024-04-20T10:22:40.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.816+0000 7f4b91e00640 1 -- 172.21.15.121:0/2346472495 learned_addr learned my addr 172.21.15.121:0/2346472495 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:40.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.817+0000 7f4b8be00640 1 --2- 172.21.15.121:0/2346472495 >> v2:172.21.15.123:3300/0 conn(0x7f4b94071a10 0x7f4b94071ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:40.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.817+0000 7f4b92800640 1 --2- 172.21.15.121:0/2346472495 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b94072400 0x7f4b94072840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:40.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.817+0000 7f4b91e00640 1 -- 172.21.15.121:0/2346472495 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b94072400 msgr2=0x7f4b94072840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:40.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.817+0000 7f4b91e00640 1 --2- 172.21.15.121:0/2346472495 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b94072400 0x7f4b94072840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:40.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.817+0000 7f4b91e00640 1 -- 172.21.15.121:0/2346472495 >> v2:172.21.15.123:3300/0 conn(0x7f4b94071a10 msgr2=0x7f4b94071ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:40.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.817+0000 7f4b91e00640 1 --2- 172.21.15.121:0/2346472495 >> v2:172.21.15.123:3300/0 conn(0x7f4b94071a10 0x7f4b94071ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:40.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.817+0000 7f4b91e00640 1 -- 172.21.15.121:0/2346472495 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b940cb890 con 0x7f4b9415de30 2024-04-20T10:22:40.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.817+0000 7f4b8be00640 1 --2- 172.21.15.121:0/2346472495 >> v2:172.21.15.123:3300/0 conn(0x7f4b94071a10 0x7f4b94071ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:40.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.818+0000 7f4b91e00640 1 --2- 172.21.15.121:0/2346472495 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b9415de30 0x7f4b940cdb00 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4b940ca830 tx=0x7f4b8402b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac50b3719b29b140 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:40.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.818+0000 7f4b91400640 1 -- 172.21.15.121:0/2346472495 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b84038070 con 0x7f4b9415de30 2024-04-20T10:22:40.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.818+0000 7f4b91400640 1 -- 172.21.15.121:0/2346472495 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b84031740 con 0x7f4b9415de30 2024-04-20T10:22:40.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.818+0000 7f4b91400640 1 -- 172.21.15.121:0/2346472495 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8402f050 con 0x7f4b9415de30 2024-04-20T10:22:40.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.819+0000 7f4b93200640 1 -- 172.21.15.121:0/2346472495 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b9415de30 msgr2=0x7f4b940cdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:40.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.819+0000 7f4b93200640 1 --2- 172.21.15.121:0/2346472495 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b9415de30 0x7f4b940cdb00 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4b940ca830 tx=0x7f4b8402b7e0 comp rx=0 tx=0).stop 2024-04-20T10:22:40.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.819+0000 7f4b93200640 1 -- 172.21.15.121:0/2346472495 shutdown_connections 2024-04-20T10:22:40.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.819+0000 7f4b93200640 1 --2- 172.21.15.121:0/2346472495 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b94072400 0x7f4b94072840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:40.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.819+0000 7f4b93200640 1 --2- 172.21.15.121:0/2346472495 >> v2:172.21.15.123:3300/0 conn(0x7f4b94071a10 0x7f4b94071ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:40.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.819+0000 7f4b93200640 1 --2- 172.21.15.121:0/2346472495 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b9415de30 0x7f4b940cdb00 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:40.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.819+0000 7f4b93200640 1 -- 172.21.15.121:0/2346472495 >> 172.21.15.121:0/2346472495 conn(0x7f4b940cc9b0 msgr2=0x7f4b940ccdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:40.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.819+0000 7f4b93200640 1 -- 172.21.15.121:0/2346472495 shutdown_connections 2024-04-20T10:22:40.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.819+0000 7f4b93200640 1 -- 172.21.15.121:0/2346472495 wait complete. 2024-04-20T10:22:40.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.821+0000 7f4b93200640 1 Processor -- start 2024-04-20T10:22:40.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.821+0000 7f4b93200640 1 -- start start 2024-04-20T10:22:40.823 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.822+0000 7f4b93200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b94071a10 0x7f4b940c4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:40.823 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.822+0000 7f4b93200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b94072400 0x7f4b940c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:40.823 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.822+0000 7f4b93200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b9415de30 0x7f4b941679c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:40.823 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.822+0000 7f4b93200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9406fbd0 con 0x7f4b94071a10 2024-04-20T10:22:40.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.822+0000 7f4b93200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9406e040 con 0x7f4b9415de30 2024-04-20T10:22:40.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.822+0000 7f4b93200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f4b9406fd50 con 0x7f4b94072400 2024-04-20T10:22:40.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.822+0000 7f4b91e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b94071a10 0x7f4b940c4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:40.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.822+0000 7f4b91e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b94071a10 0x7f4b940c4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34302/0 (socket says 172.21.15.121:34302) 2024-04-20T10:22:40.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.822+0000 7f4b91e00640 1 -- 172.21.15.121:0/107815013 learned_addr learned my addr 172.21.15.121:0/107815013 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:40.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.823+0000 7f4b92800640 1 --2- 172.21.15.121:0/107815013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b9415de30 0x7f4b941679c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:40.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.823+0000 7f4b91e00640 1 -- 172.21.15.121:0/107815013 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b94072400 msgr2=0x7f4b940c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:40.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.823+0000 7f4b91e00640 1 --2- 172.21.15.121:0/107815013 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b94072400 0x7f4b940c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:40.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.823+0000 7f4b91e00640 1 -- 172.21.15.121:0/107815013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b9415de30 msgr2=0x7f4b941679c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:40.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.823+0000 7f4b91e00640 1 --2- 172.21.15.121:0/107815013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b9415de30 0x7f4b941679c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:40.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.823+0000 7f4b91e00640 1 -- 172.21.15.121:0/107815013 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b940c3ae0 con 0x7f4b94071a10 2024-04-20T10:22:40.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.823+0000 7f4b92800640 1 --2- 172.21.15.121:0/107815013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b9415de30 0x7f4b941679c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:40.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.823+0000 7f4b91e00640 1 --2- 172.21.15.121:0/107815013 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b94071a10 0x7f4b940c4580 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4b84000c00 tx=0x7f4b840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:40.825 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.823+0000 7f4b8a000640 1 -- 172.21.15.121:0/107815013 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b84042020 con 0x7f4b94071a10 2024-04-20T10:22:40.825 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.823+0000 7f4b8a000640 1 -- 172.21.15.121:0/107815013 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b84033c90 con 0x7f4b94071a10 2024-04-20T10:22:40.825 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.823+0000 7f4b8a000640 1 -- 172.21.15.121:0/107815013 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b84038040 con 0x7f4b94071a10 2024-04-20T10:22:40.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.823+0000 7f4b93200640 1 -- 172.21.15.121:0/107815013 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b941683a0 con 0x7f4b94071a10 2024-04-20T10:22:40.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.824+0000 7f4b93200640 1 -- 172.21.15.121:0/107815013 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b94168680 con 0x7f4b94071a10 2024-04-20T10:22:40.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.825+0000 7f4b93200640 1 -- 172.21.15.121:0/107815013 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b94075b30 con 0x7f4b94071a10 2024-04-20T10:22:40.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.826+0000 7f4b8a000640 1 -- 172.21.15.121:0/107815013 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4b84045020 con 0x7f4b94071a10 2024-04-20T10:22:40.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.827+0000 7f4b8a000640 1 --2- 172.21.15.121:0/107815013 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f4b7006c8f0 0x7f4b7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:40.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.827+0000 7f4b8a000640 1 -- 172.21.15.121:0/107815013 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6383+0+0 (secure 0 0 0) 0x7f4b840adb60 con 0x7f4b94071a10 2024-04-20T10:22:40.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.827+0000 7f4b8be00640 1 --2- 172.21.15.121:0/107815013 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f4b7006c8f0 0x7f4b7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:40.828 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.828+0000 7f4b8be00640 1 --2- 172.21.15.121:0/107815013 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f4b7006c8f0 0x7f4b7006edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c006fd0 tx=0x7f4b7c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:40.831 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:40.831+0000 7f4b8a000640 1 -- 172.21.15.121:0/107815013 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4b8407bce0 con 0x7f4b94071a10 2024-04-20T10:22:41.041 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.040+0000 7f4b93200640 1 -- 172.21.15.121:0/107815013 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4b940cbfc0 con 0x7f4b94071a10 2024-04-20T10:22:41.595 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.594+0000 7f4b8a000640 1 -- 172.21.15.121:0/107815013 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) ==== 169+0+0 (secure 0 0 0) 0x7f4b8407b5e0 con 0x7f4b94071a10 2024-04-20T10:22:41.595 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'cephfs_data' removed 2024-04-20T10:22:41.598 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.597+0000 7f4b93200640 1 -- 172.21.15.121:0/107815013 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f4b7006c8f0 msgr2=0x7f4b7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:41.598 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.597+0000 7f4b93200640 1 --2- 172.21.15.121:0/107815013 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f4b7006c8f0 0x7f4b7006edb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c006fd0 tx=0x7f4b7c008040 comp rx=0 tx=0).stop 2024-04-20T10:22:41.598 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.597+0000 7f4b93200640 1 -- 172.21.15.121:0/107815013 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b94071a10 msgr2=0x7f4b940c4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:41.598 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.597+0000 7f4b93200640 1 --2- 172.21.15.121:0/107815013 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b94071a10 0x7f4b940c4580 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4b84000c00 tx=0x7f4b840046c0 comp rx=0 tx=0).stop 2024-04-20T10:22:41.599 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.597+0000 7f4b93200640 1 -- 172.21.15.121:0/107815013 shutdown_connections 2024-04-20T10:22:41.599 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.597+0000 7f4b93200640 1 --2- 172.21.15.121:0/107815013 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f4b7006c8f0 0x7f4b7006edb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:41.599 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.597+0000 7f4b93200640 1 --2- 172.21.15.121:0/107815013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b9415de30 0x7f4b941679c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:41.599 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.597+0000 7f4b93200640 1 --2- 172.21.15.121:0/107815013 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f4b94072400 0x7f4b940c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:41.599 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.597+0000 7f4b93200640 1 --2- 172.21.15.121:0/107815013 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f4b94071a10 0x7f4b940c4580 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:41.599 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.597+0000 7f4b93200640 1 -- 172.21.15.121:0/107815013 >> 172.21.15.121:0/107815013 conn(0x7f4b940cc9b0 msgr2=0x7f4b940aee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:41.599 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.598+0000 7f4b93200640 1 -- 172.21.15.121:0/107815013 shutdown_connections 2024-04-20T10:22:41.599 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.598+0000 7f4b93200640 1 -- 172.21.15.121:0/107815013 wait complete. 2024-04-20T10:22:41.625 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-20T10:22:41.815 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.815+0000 7f368e400640 1 Processor -- start 2024-04-20T10:22:41.815 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.815+0000 7f368e400640 1 -- start start 2024-04-20T10:22:41.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.816+0000 7f368e400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3688156c20 0x7f368806b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:41.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.816+0000 7f368e400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3688141c60 con 0x7f368815ca40 2024-04-20T10:22:41.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.816+0000 7f368e400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36880adab0 con 0x7f36880d0360 2024-04-20T10:22:41.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.816+0000 7f368e400640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f368815cf90 con 0x7f3688156c20 2024-04-20T10:22:41.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.816+0000 7f3686a00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f368815ca40 0x7f36880ad3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:46186/0 (socket says 172.21.15.121:46186) 2024-04-20T10:22:41.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.816+0000 7f3686a00640 1 -- 172.21.15.121:0/4250889822 learned_addr learned my addr 172.21.15.121:0/4250889822 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:41.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.817+0000 7f3687400640 1 --2- 172.21.15.121:0/4250889822 >> v2:172.21.15.123:3300/0 conn(0x7f3688156c20 0x7f368806b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:41.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.817+0000 7f3686000640 1 -- 172.21.15.121:0/4250889822 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3131712779 0 0) 0x7f3688141c60 con 0x7f368815ca40 2024-04-20T10:22:41.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.817+0000 7f3686000640 1 -- 172.21.15.121:0/4250889822 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f366c003520 con 0x7f368815ca40 2024-04-20T10:22:41.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.817+0000 7f3686000640 1 -- 172.21.15.121:0/4250889822 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1805056263 0 0) 0x7f366c003520 con 0x7f368815ca40 2024-04-20T10:22:41.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.817+0000 7f3686000640 1 -- 172.21.15.121:0/4250889822 >> v2:172.21.15.123:3300/0 conn(0x7f3688156c20 msgr2=0x7f368806b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:41.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.817+0000 7f3686000640 1 --2- 172.21.15.121:0/4250889822 >> v2:172.21.15.123:3300/0 conn(0x7f3688156c20 0x7f368806b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:41.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.817+0000 7f3686000640 1 -- 172.21.15.121:0/4250889822 >> v1:172.21.15.123:6789/0 conn(0x7f36880d0360 legacy=0x7f3688142140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:41.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.817+0000 7f3686000640 1 -- 172.21.15.121:0/4250889822 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f368815d270 con 0x7f368815ca40 2024-04-20T10:22:41.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.817+0000 7f3686000640 1 -- 172.21.15.121:0/4250889822 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f3674001670 con 0x7f368815ca40 2024-04-20T10:22:41.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.817+0000 7f3686000640 1 -- 172.21.15.121:0/4250889822 >> v1:172.21.15.121:6789/0 conn(0x7f368815ca40 legacy=0x7f36880ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:41.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.818+0000 7f3686000640 1 --2- 172.21.15.121:0/4250889822 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f366c0039b0 0x7f366c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:41.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.818+0000 7f3686000640 1 --2- 172.21.15.121:0/4250889822 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f366c004800 0x7f366c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:41.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.818+0000 7f3686000640 1 --2- 172.21.15.121:0/4250889822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f366c0051c0 0x7f366c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:41.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.818+0000 7f3686000640 1 -- 172.21.15.121:0/4250889822 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f368815cf90 con 0x7f366c004800 2024-04-20T10:22:41.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.818+0000 7f3686000640 1 -- 172.21.15.121:0/4250889822 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f366c00c060 con 0x7f366c0051c0 2024-04-20T10:22:41.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.818+0000 7f3686000640 1 -- 172.21.15.121:0/4250889822 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f366c00c260 con 0x7f366c0039b0 2024-04-20T10:22:41.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.818+0000 7f3686a00640 1 --2- 172.21.15.121:0/4250889822 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f366c004800 0x7f366c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:41.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.818+0000 7f3687400640 1 --2- 172.21.15.121:0/4250889822 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f366c0039b0 0x7f366c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:41.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.818+0000 7f3687e00640 1 --2- 172.21.15.121:0/4250889822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f366c0051c0 0x7f366c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:41.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.818+0000 7f3686a00640 1 -- 172.21.15.121:0/4250889822 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f366c0039b0 msgr2=0x7f366c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:41.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.818+0000 7f3686a00640 1 --2- 172.21.15.121:0/4250889822 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f366c0039b0 0x7f366c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:41.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.818+0000 7f3686a00640 1 -- 172.21.15.121:0/4250889822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f366c0051c0 msgr2=0x7f366c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:41.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.818+0000 7f3686a00640 1 --2- 172.21.15.121:0/4250889822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f366c0051c0 0x7f366c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:41.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.818+0000 7f3686a00640 1 -- 172.21.15.121:0/4250889822 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f366c003520 con 0x7f366c004800 2024-04-20T10:22:41.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.818+0000 7f3687e00640 1 --2- 172.21.15.121:0/4250889822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f366c0051c0 0x7f366c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:41.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.818+0000 7f3687400640 1 --2- 172.21.15.121:0/4250889822 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f366c0039b0 0x7f366c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:41.823 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.818+0000 7f3687e00640 1 -- 172.21.15.121:0/4250889822 reap_dead start 2024-04-20T10:22:41.823 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.820+0000 7f3686a00640 1 --2- 172.21.15.121:0/4250889822 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f366c004800 0x7f366c004c80 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f367400c210 tx=0x7f366c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da91ce1bcaa6fd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:41.823 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.820+0000 7f3686000640 1 -- 172.21.15.121:0/4250889822 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3674007d30 con 0x7f366c004800 2024-04-20T10:22:41.823 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.820+0000 7f3686000640 1 -- 172.21.15.121:0/4250889822 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3674030070 con 0x7f366c004800 2024-04-20T10:22:41.823 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.820+0000 7f3686000640 1 -- 172.21.15.121:0/4250889822 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367402cda0 con 0x7f366c004800 2024-04-20T10:22:41.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.820+0000 7f368e400640 1 -- 172.21.15.121:0/4250889822 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f366c004800 msgr2=0x7f366c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:41.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.820+0000 7f368e400640 1 --2- 172.21.15.121:0/4250889822 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f366c004800 0x7f366c004c80 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f367400c210 tx=0x7f366c005830 comp rx=0 tx=0).stop 2024-04-20T10:22:41.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.821+0000 7f368e400640 1 -- 172.21.15.121:0/4250889822 shutdown_connections 2024-04-20T10:22:41.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.821+0000 7f368e400640 1 --2- 172.21.15.121:0/4250889822 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f366c004800 0x7f366c004c80 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:41.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.821+0000 7f368e400640 1 -- 172.21.15.121:0/4250889822 >> 172.21.15.121:0/4250889822 conn(0x7f36880b3350 msgr2=0x7f36880b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:41.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.821+0000 7f368e400640 1 -- 172.21.15.121:0/4250889822 shutdown_connections 2024-04-20T10:22:41.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.821+0000 7f368e400640 1 -- 172.21.15.121:0/4250889822 wait complete. 2024-04-20T10:22:41.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.822+0000 7f368e400640 1 Processor -- start 2024-04-20T10:22:41.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.823+0000 7f368e400640 1 -- start start 2024-04-20T10:22:41.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.823+0000 7f368e400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f366c004800 0x7f36880994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:41.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.823+0000 7f368e400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36880ccb30 0x7f36880ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:41.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.823+0000 7f368e400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36881e75e0 0x7f36881e7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:41.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.823+0000 7f368e400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f36880d07d0 con 0x7f36880ccb30 2024-04-20T10:22:41.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.823+0000 7f368e400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f368815d830 con 0x7f36881e75e0 2024-04-20T10:22:41.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.823+0000 7f368e400640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f36880d0950 con 0x7f366c004800 2024-04-20T10:22:41.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.824+0000 7f3686a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36880ccb30 0x7f36880ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:41.828 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.824+0000 7f3686a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36880ccb30 0x7f36880ccf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34312/0 (socket says 172.21.15.121:34312) 2024-04-20T10:22:41.828 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.824+0000 7f3686a00640 1 -- 172.21.15.121:0/2794051207 learned_addr learned my addr 172.21.15.121:0/2794051207 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:41.828 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.824+0000 7f3687400640 1 --2- 172.21.15.121:0/2794051207 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f366c004800 0x7f36880994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:41.828 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.824+0000 7f3687e00640 1 --2- 172.21.15.121:0/2794051207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36881e75e0 0x7f36881e7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:41.828 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.824+0000 7f3686a00640 1 -- 172.21.15.121:0/2794051207 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f366c004800 msgr2=0x7f36880994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:41.828 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.824+0000 7f3686a00640 1 --2- 172.21.15.121:0/2794051207 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f366c004800 0x7f36880994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:41.828 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.824+0000 7f3686a00640 1 -- 172.21.15.121:0/2794051207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36881e75e0 msgr2=0x7f36881e7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:41.828 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.824+0000 7f3686a00640 1 --2- 172.21.15.121:0/2794051207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36881e75e0 0x7f36881e7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:41.828 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.824+0000 7f3686a00640 1 -- 172.21.15.121:0/2794051207 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36880cedb0 con 0x7f36880ccb30 2024-04-20T10:22:41.828 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.825+0000 7f3686a00640 1 --2- 172.21.15.121:0/2794051207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36880ccb30 0x7f36880ccf90 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f367402c570 tx=0x7f367402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:41.828 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.825+0000 7f367f400640 1 -- 172.21.15.121:0/2794051207 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3674046070 con 0x7f36880ccb30 2024-04-20T10:22:41.829 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.825+0000 7f367f400640 1 -- 172.21.15.121:0/2794051207 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3674004bf0 con 0x7f36880ccb30 2024-04-20T10:22:41.829 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.825+0000 7f367f400640 1 -- 172.21.15.121:0/2794051207 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3674007d10 con 0x7f36880ccb30 2024-04-20T10:22:41.829 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.825+0000 7f368e400640 1 -- 172.21.15.121:0/2794051207 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36880cd6a0 con 0x7f36880ccb30 2024-04-20T10:22:41.829 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.825+0000 7f368e400640 1 -- 172.21.15.121:0/2794051207 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36880cdc60 con 0x7f36880ccb30 2024-04-20T10:22:41.830 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.826+0000 7f368e400640 1 -- 172.21.15.121:0/2794051207 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3688068980 con 0x7f36880ccb30 2024-04-20T10:22:41.830 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.827+0000 7f367f400640 1 -- 172.21.15.121:0/2794051207 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3674007070 con 0x7f36880ccb30 2024-04-20T10:22:41.830 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.828+0000 7f367f400640 1 --2- 172.21.15.121:0/2794051207 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f366806c8f0 0x7f366806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:41.830 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.828+0000 7f367f400640 1 -- 172.21.15.121:0/2794051207 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5853+0+0 (secure 0 0 0) 0x7f36740ad2f0 con 0x7f36880ccb30 2024-04-20T10:22:41.830 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.828+0000 7f3687400640 1 --2- 172.21.15.121:0/2794051207 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f366806c8f0 0x7f366806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:41.830 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.829+0000 7f3687400640 1 --2- 172.21.15.121:0/2794051207 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f366806c8f0 0x7f366806edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3688069030 tx=0x7f3678008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:41.833 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:41.833+0000 7f367f400640 1 -- 172.21.15.121:0/2794051207 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f367407ae90 con 0x7f36880ccb30 2024-04-20T10:22:42.032 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.031+0000 7f368e400640 1 -- 172.21.15.121:0/2794051207 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f36880cfd90 con 0x7f36880ccb30 2024-04-20T10:22:42.598 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.598+0000 7f367f400640 1 -- 172.21.15.121:0/2794051207 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) ==== 178+0+0 (secure 0 0 0) 0x7f367407f190 con 0x7f36880ccb30 2024-04-20T10:22:42.598 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'cephfs_data_ec' removed 2024-04-20T10:22:42.601 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.602+0000 7f368e400640 1 -- 172.21.15.121:0/2794051207 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f366806c8f0 msgr2=0x7f366806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:42.601 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.602+0000 7f368e400640 1 --2- 172.21.15.121:0/2794051207 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f366806c8f0 0x7f366806edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3688069030 tx=0x7f3678008040 comp rx=0 tx=0).stop 2024-04-20T10:22:42.602 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.602+0000 7f368e400640 1 -- 172.21.15.121:0/2794051207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36880ccb30 msgr2=0x7f36880ccf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:42.602 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.602+0000 7f368e400640 1 --2- 172.21.15.121:0/2794051207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36880ccb30 0x7f36880ccf90 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f367402c570 tx=0x7f367402e070 comp rx=0 tx=0).stop 2024-04-20T10:22:42.602 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.602+0000 7f368e400640 1 -- 172.21.15.121:0/2794051207 shutdown_connections 2024-04-20T10:22:42.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.602+0000 7f368e400640 1 --2- 172.21.15.121:0/2794051207 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f366806c8f0 0x7f366806edb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:42.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.602+0000 7f368e400640 1 --2- 172.21.15.121:0/2794051207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36881e75e0 0x7f36881e7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:42.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.602+0000 7f368e400640 1 --2- 172.21.15.121:0/2794051207 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36880ccb30 0x7f36880ccf90 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:42.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.602+0000 7f368e400640 1 --2- 172.21.15.121:0/2794051207 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f366c004800 0x7f36880994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:42.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.602+0000 7f368e400640 1 -- 172.21.15.121:0/2794051207 >> 172.21.15.121:0/2794051207 conn(0x7f36880b3350 msgr2=0x7f368815db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:42.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.603+0000 7f368e400640 1 -- 172.21.15.121:0/2794051207 shutdown_connections 2024-04-20T10:22:42.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.603+0000 7f368e400640 1 -- 172.21.15.121:0/2794051207 wait complete. 2024-04-20T10:22:42.627 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-20T10:22:42.627 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-20T10:22:42.628 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-20T10:22:42.628 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:22:42.628 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-20T10:22:42.628 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-20T10:22:42.628 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-20T10:22:42.628 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:22:42.629 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-20T10:22:42.629 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-20T10:22:42.629 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-20T10:22:42.629 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:22:42.629 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-20T10:22:42.629 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-20T10:22:42.630 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-20T10:22:42.630 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:22:42.647 INFO:tasks.ceph.mds.a:Stopped 2024-04-20T10:22:42.647 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-20T10:22:42.650 INFO:tasks.ceph.mds.c:Stopped 2024-04-20T10:22:42.650 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-20T10:22:42.651 INFO:tasks.ceph.mds.b:Stopped 2024-04-20T10:22:42.651 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-20T10:22:42.659 INFO:tasks.ceph.mds.d:Stopped 2024-04-20T10:22:42.659 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-20T10:22:42.697 INFO:tasks.ceph.mds.a:Started 2024-04-20T10:22:42.697 DEBUG:teuthology.parallel:result is None 2024-04-20T10:22:42.703 INFO:tasks.ceph.mds.c:Started 2024-04-20T10:22:42.703 DEBUG:teuthology.parallel:result is None 2024-04-20T10:22:42.705 INFO:tasks.ceph.mds.b:Started 2024-04-20T10:22:42.705 DEBUG:teuthology.parallel:result is None 2024-04-20T10:22:42.711 INFO:tasks.ceph.mds.d:Started 2024-04-20T10:22:42.711 DEBUG:teuthology.parallel:result is None 2024-04-20T10:22:42.712 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-20T10:22:42.845 INFO:tasks.ceph.mds.a.smithi121.stdout:starting mds.a at 2024-04-20T10:22:42.850 INFO:tasks.ceph.mds.c.smithi121.stdout:starting mds.c at 2024-04-20T10:22:42.859 INFO:tasks.ceph.mds.d.smithi123.stdout:starting mds.d at 2024-04-20T10:22:42.862 INFO:tasks.ceph.mds.b.smithi123.stdout:starting mds.b at 2024-04-20T10:22:42.890 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.891+0000 7f2c41200640 1 Processor -- start 2024-04-20T10:22:42.891 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.891+0000 7f2c41200640 1 -- start start 2024-04-20T10:22:42.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.893+0000 7f2c41200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2c3c06d510 0x7f2c3c0b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:42.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.893+0000 7f2c41200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f2c3c163060 0x7f2c3c0ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:42.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.893+0000 7f2c41200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c3c06d8f0 0x7f2c3c0cef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:42.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.893+0000 7f2c41200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f2c3c0b2d00 con 0x7f2c3c06d510 2024-04-20T10:22:42.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.893+0000 7f2c41200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f2c3c0b1c60 con 0x7f2c3c163060 2024-04-20T10:22:42.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.893+0000 7f2c41200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f2c3c0b1f40 con 0x7f2c3c06d8f0 2024-04-20T10:22:42.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.893+0000 7f2c39e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2c3c06d510 0x7f2c3c0b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:42.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.893+0000 7f2c39e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f2c3c06d510 0x7f2c3c0b3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34346/0 (socket says 172.21.15.121:34346) 2024-04-20T10:22:42.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.893+0000 7f2c39e00640 1 -- 172.21.15.121:0/103174065 learned_addr learned my addr 172.21.15.121:0/103174065 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:42.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.893+0000 7f2c3a800640 1 --2- 172.21.15.121:0/103174065 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c3c06d8f0 0x7f2c3c0cef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:42.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.893+0000 7f2c39400640 1 --2- 172.21.15.121:0/103174065 >> v2:172.21.15.123:3300/0 conn(0x7f2c3c163060 0x7f2c3c0ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:42.894 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.893+0000 7f2c39e00640 1 -- 172.21.15.121:0/103174065 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c3c06d8f0 msgr2=0x7f2c3c0cef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:42.894 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.893+0000 7f2c39e00640 1 --2- 172.21.15.121:0/103174065 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c3c06d8f0 0x7f2c3c0cef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:42.894 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.893+0000 7f2c39e00640 1 -- 172.21.15.121:0/103174065 >> v2:172.21.15.123:3300/0 conn(0x7f2c3c163060 msgr2=0x7f2c3c0ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:42.894 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.893+0000 7f2c39e00640 1 --2- 172.21.15.121:0/103174065 >> v2:172.21.15.123:3300/0 conn(0x7f2c3c163060 0x7f2c3c0ce9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:42.894 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.894+0000 7f2c39e00640 1 -- 172.21.15.121:0/103174065 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c3c0b2220 con 0x7f2c3c06d510 2024-04-20T10:22:42.894 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.894+0000 7f2c3a800640 1 --2- 172.21.15.121:0/103174065 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c3c06d8f0 0x7f2c3c0cef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:22:42.895 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.895+0000 7f2c39e00640 1 --2- 172.21.15.121:0/103174065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c3c06d510 0x7f2c3c0b3810 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c0b1e70 tx=0x7f2c2c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa0f86c31a7c23e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:42.895 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.895+0000 7f2c38a00640 1 -- 172.21.15.121:0/103174065 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c035020 con 0x7f2c3c06d510 2024-04-20T10:22:42.895 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.895+0000 7f2c38a00640 1 -- 172.21.15.121:0/103174065 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c2c031640 con 0x7f2c3c06d510 2024-04-20T10:22:42.895 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.895+0000 7f2c38a00640 1 -- 172.21.15.121:0/103174065 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c02d040 con 0x7f2c3c06d510 2024-04-20T10:22:42.895 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.896+0000 7f2c41200640 1 -- 172.21.15.121:0/103174065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c3c06d510 msgr2=0x7f2c3c0b3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:42.896 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.896+0000 7f2c41200640 1 --2- 172.21.15.121:0/103174065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c3c06d510 0x7f2c3c0b3810 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c0b1e70 tx=0x7f2c2c02b8f0 comp rx=0 tx=0).stop 2024-04-20T10:22:42.896 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.896+0000 7f2c41200640 1 -- 172.21.15.121:0/103174065 shutdown_connections 2024-04-20T10:22:42.896 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.896+0000 7f2c41200640 1 --2- 172.21.15.121:0/103174065 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c3c06d8f0 0x7f2c3c0cef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:42.896 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.896+0000 7f2c41200640 1 --2- 172.21.15.121:0/103174065 >> v2:172.21.15.123:3300/0 conn(0x7f2c3c163060 0x7f2c3c0ce9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:42.896 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.896+0000 7f2c41200640 1 --2- 172.21.15.121:0/103174065 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c3c06d510 0x7f2c3c0b3810 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:42.896 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.896+0000 7f2c41200640 1 -- 172.21.15.121:0/103174065 >> 172.21.15.121:0/103174065 conn(0x7f2c3c071800 msgr2=0x7f2c3c071c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:42.896 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.896+0000 7f2c41200640 1 -- 172.21.15.121:0/103174065 shutdown_connections 2024-04-20T10:22:42.896 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.896+0000 7f2c41200640 1 -- 172.21.15.121:0/103174065 wait complete. 2024-04-20T10:22:42.897 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.897+0000 7f2c41200640 1 Processor -- start 2024-04-20T10:22:42.897 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.898+0000 7f2c41200640 1 -- start start 2024-04-20T10:22:42.898 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.899+0000 7f2c41200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c3c06d510 0x7f2c3c0c20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:42.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.899+0000 7f2c41200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c3c06d8f0 0x7f2c3c1e8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:42.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.899+0000 7f2c41200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c3c163060 0x7f2c3c0cb770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:42.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.899+0000 7f2c41200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c0cf460 con 0x7f2c3c06d8f0 2024-04-20T10:22:42.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.899+0000 7f2c41200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2c3c0b27e0 con 0x7f2c3c06d510 2024-04-20T10:22:42.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.899+0000 7f2c41200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f2c3c0cf5e0 con 0x7f2c3c163060 2024-04-20T10:22:42.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.899+0000 7f2c39e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c3c06d510 0x7f2c3c0c20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:42.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.899+0000 7f2c39400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c3c06d8f0 0x7f2c3c1e8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:42.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.899+0000 7f2c39400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c3c06d8f0 0x7f2c3c1e8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34358/0 (socket says 172.21.15.121:34358) 2024-04-20T10:22:42.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.899+0000 7f2c39400640 1 -- 172.21.15.121:0/199424720 learned_addr learned my addr 172.21.15.121:0/199424720 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:42.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.899+0000 7f2c3a800640 1 --2- 172.21.15.121:0/199424720 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c3c163060 0x7f2c3c0cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:42.900 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.900+0000 7f2c39400640 1 -- 172.21.15.121:0/199424720 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c3c163060 msgr2=0x7f2c3c0cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:42.900 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.900+0000 7f2c39400640 1 --2- 172.21.15.121:0/199424720 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c3c163060 0x7f2c3c0cb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:42.900 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.900+0000 7f2c39400640 1 -- 172.21.15.121:0/199424720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c3c06d510 msgr2=0x7f2c3c0c20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:42.901 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.900+0000 7f2c39400640 1 --2- 172.21.15.121:0/199424720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c3c06d510 0x7f2c3c0c20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:42.901 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.900+0000 7f2c39400640 1 -- 172.21.15.121:0/199424720 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c3c1574b0 con 0x7f2c3c06d8f0 2024-04-20T10:22:42.901 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.900+0000 7f2c39e00640 1 --2- 172.21.15.121:0/199424720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c3c06d510 0x7f2c3c0c20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:22:42.901 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.900+0000 7f2c39400640 1 --2- 172.21.15.121:0/199424720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c3c06d8f0 0x7f2c3c1e8950 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2c2800d830 tx=0x7f2c2800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:42.901 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.900+0000 7f2c32000640 1 -- 172.21.15.121:0/199424720 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2800b6c0 con 0x7f2c3c06d8f0 2024-04-20T10:22:42.901 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.901+0000 7f2c32000640 1 -- 172.21.15.121:0/199424720 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c28019070 con 0x7f2c3c06d8f0 2024-04-20T10:22:42.901 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.901+0000 7f2c41200640 1 -- 172.21.15.121:0/199424720 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c3c0c9860 con 0x7f2c3c06d8f0 2024-04-20T10:22:42.901 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.901+0000 7f2c32000640 1 -- 172.21.15.121:0/199424720 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c2800ba80 con 0x7f2c3c06d8f0 2024-04-20T10:22:42.901 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.901+0000 7f2c41200640 1 -- 172.21.15.121:0/199424720 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c3c0c9e20 con 0x7f2c3c06d8f0 2024-04-20T10:22:42.901 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.902+0000 7f2c41200640 1 -- 172.21.15.121:0/199424720 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c3c068980 con 0x7f2c3c06d8f0 2024-04-20T10:22:42.902 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.903+0000 7f2c32000640 1 -- 172.21.15.121:0/199424720 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2c2801e460 con 0x7f2c3c06d8f0 2024-04-20T10:22:42.903 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.904+0000 7f2c32000640 1 --2- 172.21.15.121:0/199424720 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f2c1c06c8f0 0x7f2c1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:42.904 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.904+0000 7f2c32000640 1 -- 172.21.15.121:0/199424720 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5335+0+0 (secure 0 0 0) 0x7f2c2808bed0 con 0x7f2c3c06d8f0 2024-04-20T10:22:42.904 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.904+0000 7f2c39e00640 1 --2- 172.21.15.121:0/199424720 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f2c1c06c8f0 0x7f2c1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:42.904 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.905+0000 7f2c39e00640 1 --2- 172.21.15.121:0/199424720 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f2c1c06c8f0 0x7f2c1c06edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c036000 tx=0x7f2c2c03c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:42.908 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:42.909+0000 7f2c32000640 1 -- 172.21.15.121:0/199424720 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2c28059e20 con 0x7f2c3c06d8f0 2024-04-20T10:22:43.126 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.126+0000 7f2c41200640 1 -- 172.21.15.121:0/199424720 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f2c3c0b35f0 con 0x7f2c3c06d8f0 2024-04-20T10:22:43.632 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.632+0000 7f2c32000640 1 -- 172.21.15.121:0/199424720 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v33) ==== 97+0+0 (secure 0 0 0) 0x7f2c2805e120 con 0x7f2c3c06d8f0 2024-04-20T10:22:43.632 INFO:teuthology.orchestra.run.smithi121.stderr: removed all blocklist entries 2024-04-20T10:22:43.637 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.637+0000 7f2c41200640 1 -- 172.21.15.121:0/199424720 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f2c1c06c8f0 msgr2=0x7f2c1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:43.637 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.637+0000 7f2c41200640 1 --2- 172.21.15.121:0/199424720 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f2c1c06c8f0 0x7f2c1c06edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c036000 tx=0x7f2c2c03c040 comp rx=0 tx=0).stop 2024-04-20T10:22:43.637 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.637+0000 7f2c41200640 1 -- 172.21.15.121:0/199424720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c3c06d8f0 msgr2=0x7f2c3c1e8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:43.638 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.637+0000 7f2c41200640 1 --2- 172.21.15.121:0/199424720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c3c06d8f0 0x7f2c3c1e8950 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2c2800d830 tx=0x7f2c2800dcf0 comp rx=0 tx=0).stop 2024-04-20T10:22:43.638 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.637+0000 7f2c41200640 1 -- 172.21.15.121:0/199424720 shutdown_connections 2024-04-20T10:22:43.638 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.637+0000 7f2c41200640 1 --2- 172.21.15.121:0/199424720 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f2c3c163060 0x7f2c3c0cb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:43.638 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.637+0000 7f2c41200640 1 --2- 172.21.15.121:0/199424720 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f2c3c06d8f0 0x7f2c3c1e8950 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:43.638 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.637+0000 7f2c41200640 1 --2- 172.21.15.121:0/199424720 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f2c1c06c8f0 0x7f2c1c06edb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:43.638 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.637+0000 7f2c41200640 1 --2- 172.21.15.121:0/199424720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c3c06d510 0x7f2c3c0c20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:43.638 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.637+0000 7f2c41200640 1 -- 172.21.15.121:0/199424720 >> 172.21.15.121:0/199424720 conn(0x7f2c3c071800 msgr2=0x7f2c3c161330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:43.638 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.637+0000 7f2c41200640 1 -- 172.21.15.121:0/199424720 shutdown_connections 2024-04-20T10:22:43.638 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.638+0000 7f2c41200640 1 -- 172.21.15.121:0/199424720 wait complete. 2024-04-20T10:22:43.660 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-20T10:22:43.854 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.855+0000 7fc223000640 1 Processor -- start 2024-04-20T10:22:43.855 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.855+0000 7fc223000640 1 -- start start 2024-04-20T10:22:43.856 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.856+0000 7fc223000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc21c15b380 0x7fc21c15f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:43.857 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.856+0000 7fc223000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc21c0cc360 0x7fc21c06dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:43.857 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.856+0000 7fc223000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc21c072d70 0x7fc21c0731d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:43.857 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.856+0000 7fc223000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fc21c073710 con 0x7fc21c0cc360 2024-04-20T10:22:43.857 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.856+0000 7fc223000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fc21c0aec60 con 0x7fc21c072d70 2024-04-20T10:22:43.857 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.856+0000 7fc223000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fc21c15b760 con 0x7fc21c15b380 2024-04-20T10:22:43.857 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.857+0000 7fc213400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc21c0cc360 0x7fc21c06dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:43.857 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.857+0000 7fc213400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fc21c0cc360 0x7fc21c06dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34374/0 (socket says 172.21.15.121:34374) 2024-04-20T10:22:43.857 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.857+0000 7fc213400640 1 -- 172.21.15.121:0/2380228489 learned_addr learned my addr 172.21.15.121:0/2380228489 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:43.857 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.857+0000 7fc21be00640 1 --2- 172.21.15.121:0/2380228489 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc21c15b380 0x7fc21c15f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:43.857 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.857+0000 7fc213400640 1 -- 172.21.15.121:0/2380228489 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc21c15b380 msgr2=0x7fc21c15f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:43.857 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.857+0000 7fc213400640 1 --2- 172.21.15.121:0/2380228489 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc21c15b380 0x7fc21c15f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:43.858 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.857+0000 7fc213400640 1 -- 172.21.15.121:0/2380228489 >> v2:172.21.15.123:3300/0 conn(0x7fc21c072d70 msgr2=0x7fc21c0731d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:43.858 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.857+0000 7fc213400640 1 --2- 172.21.15.121:0/2380228489 >> v2:172.21.15.123:3300/0 conn(0x7fc21c072d70 0x7fc21c0731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:43.858 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.857+0000 7fc213400640 1 -- 172.21.15.121:0/2380228489 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc21c0ae090 con 0x7fc21c0cc360 2024-04-20T10:22:43.859 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.858+0000 7fc213400640 1 --2- 172.21.15.121:0/2380228489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc21c0cc360 0x7fc21c06dbb0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc21c0b1d20 tx=0x7fc20402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=104bbffcf9244880 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:43.859 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.858+0000 7fc21b400640 1 -- 172.21.15.121:0/2380228489 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc20402e070 con 0x7fc21c0cc360 2024-04-20T10:22:43.859 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.858+0000 7fc21b400640 1 -- 172.21.15.121:0/2380228489 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc204037070 con 0x7fc21c0cc360 2024-04-20T10:22:43.859 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.859+0000 7fc21b400640 1 -- 172.21.15.121:0/2380228489 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc204031a00 con 0x7fc21c0cc360 2024-04-20T10:22:43.860 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.859+0000 7fc223000640 1 -- 172.21.15.121:0/2380228489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc21c0cc360 msgr2=0x7fc21c06dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:43.860 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.859+0000 7fc223000640 1 --2- 172.21.15.121:0/2380228489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc21c0cc360 0x7fc21c06dbb0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc21c0b1d20 tx=0x7fc20402b880 comp rx=0 tx=0).stop 2024-04-20T10:22:43.860 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.859+0000 7fc223000640 1 -- 172.21.15.121:0/2380228489 shutdown_connections 2024-04-20T10:22:43.860 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.859+0000 7fc223000640 1 --2- 172.21.15.121:0/2380228489 >> v2:172.21.15.123:3300/0 conn(0x7fc21c072d70 0x7fc21c0731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:43.860 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.859+0000 7fc223000640 1 --2- 172.21.15.121:0/2380228489 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc21c0cc360 0x7fc21c06dbb0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:43.860 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.859+0000 7fc223000640 1 --2- 172.21.15.121:0/2380228489 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc21c15b380 0x7fc21c15f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:43.860 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.859+0000 7fc223000640 1 -- 172.21.15.121:0/2380228489 >> 172.21.15.121:0/2380228489 conn(0x7fc21c0b1d70 msgr2=0x7fc21c0b2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:43.860 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.859+0000 7fc223000640 1 -- 172.21.15.121:0/2380228489 shutdown_connections 2024-04-20T10:22:43.860 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.859+0000 7fc223000640 1 -- 172.21.15.121:0/2380228489 wait complete. 2024-04-20T10:22:43.861 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.861+0000 7fc223000640 1 Processor -- start 2024-04-20T10:22:43.861 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.861+0000 7fc223000640 1 -- start start 2024-04-20T10:22:43.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.862+0000 7fc223000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc21c072d70 0x7fc21c15e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:43.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.862+0000 7fc223000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc21c0cc360 0x7fc21c14e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:43.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.862+0000 7fc223000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc21c15b380 0x7fc21c1f82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:43.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.862+0000 7fc223000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fc21c0ae610 con 0x7fc21c0cc360 2024-04-20T10:22:43.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.862+0000 7fc223000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc21c15fe00 con 0x7fc21c15b380 2024-04-20T10:22:43.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.862+0000 7fc223000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fc21c0ae8f0 con 0x7fc21c072d70 2024-04-20T10:22:43.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.863+0000 7fc213400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc21c0cc360 0x7fc21c14e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:43.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.863+0000 7fc213400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc21c0cc360 0x7fc21c14e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34384/0 (socket says 172.21.15.121:34384) 2024-04-20T10:22:43.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.863+0000 7fc213400640 1 -- 172.21.15.121:0/132770616 learned_addr learned my addr 172.21.15.121:0/132770616 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:43.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.863+0000 7fc21be00640 1 --2- 172.21.15.121:0/132770616 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc21c072d70 0x7fc21c15e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:43.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.863+0000 7fc213400640 1 -- 172.21.15.121:0/132770616 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc21c072d70 msgr2=0x7fc21c15e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:43.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.863+0000 7fc213400640 1 --2- 172.21.15.121:0/132770616 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc21c072d70 0x7fc21c15e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:43.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.863+0000 7fc213400640 1 -- 172.21.15.121:0/132770616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc21c15b380 msgr2=0x7fc21c1f82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:43.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.863+0000 7fc213400640 1 --2- 172.21.15.121:0/132770616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc21c15b380 0x7fc21c1f82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:43.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.863+0000 7fc213400640 1 -- 172.21.15.121:0/132770616 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc21c15e970 con 0x7fc21c0cc360 2024-04-20T10:22:43.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.864+0000 7fc213400640 1 --2- 172.21.15.121:0/132770616 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc21c0cc360 0x7fc21c14e700 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc204000c00 tx=0x7fc2040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:43.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.864+0000 7fc218c00640 1 -- 172.21.15.121:0/132770616 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc204032880 con 0x7fc21c0cc360 2024-04-20T10:22:43.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.864+0000 7fc218c00640 1 -- 172.21.15.121:0/132770616 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc204032a20 con 0x7fc21c0cc360 2024-04-20T10:22:43.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.864+0000 7fc218c00640 1 -- 172.21.15.121:0/132770616 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc204032e10 con 0x7fc21c0cc360 2024-04-20T10:22:43.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.864+0000 7fc223000640 1 -- 172.21.15.121:0/132770616 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc21c1f8df0 con 0x7fc21c0cc360 2024-04-20T10:22:43.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.864+0000 7fc223000640 1 -- 172.21.15.121:0/132770616 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc21c15dc30 con 0x7fc21c0cc360 2024-04-20T10:22:43.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.865+0000 7fc223000640 1 -- 172.21.15.121:0/132770616 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc21c15da20 con 0x7fc21c0cc360 2024-04-20T10:22:43.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.866+0000 7fc218c00640 1 -- 172.21.15.121:0/132770616 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc20404c060 con 0x7fc21c0cc360 2024-04-20T10:22:43.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.867+0000 7fc218c00640 1 --2- 172.21.15.121:0/132770616 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fc1f806c8f0 0x7fc1f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:43.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.867+0000 7fc218c00640 1 -- 172.21.15.121:0/132770616 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7fc2040ad820 con 0x7fc21c0cc360 2024-04-20T10:22:43.867 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.867+0000 7fc21be00640 1 --2- 172.21.15.121:0/132770616 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fc1f806c8f0 0x7fc1f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:43.867 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.868+0000 7fc21be00640 1 --2- 172.21.15.121:0/132770616 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fc1f806c8f0 0x7fc1f806edb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc20c00b7c0 tx=0x7fc20c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:43.871 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:43.872+0000 7fc218c00640 1 -- 172.21.15.121:0/132770616 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc204040070 con 0x7fc21c0cc360 2024-04-20T10:22:44.150 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.150+0000 7fc223000640 1 -- 172.21.15.121:0/132770616 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fc21c068980 con 0x7fc21c0cc360 2024-04-20T10:22:44.150 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.151+0000 7fc218c00640 1 -- 172.21.15.121:0/132770616 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) ==== 80+0+6645 (secure 0 0 0) 0x7fc204039310 con 0x7fc21c0cc360 2024-04-20T10:22:44.151 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:22:44.151 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T10:22:44.151 INFO:teuthology.orchestra.run.smithi121.stdout: "auth_dump": [ 2024-04-20T10:22:44.151 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T10:22:44.151 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mds.a", 2024-04-20T10:22:44.151 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQDOliNmpcYbJxAAt+TxajxrS1sHNHD7ciN9Sg==", 2024-04-20T10:22:44.151 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2024-04-20T10:22:44.151 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2024-04-20T10:22:44.152 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow *", 2024-04-20T10:22:44.152 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow *", 2024-04-20T10:22:44.152 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2024-04-20T10:22:44.152 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T10:22:44.152 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T10:22:44.152 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T10:22:44.152 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mds.b", 2024-04-20T10:22:44.152 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQDPliNmcsE6BRAA2kx+YMZFBMJGQ8ulIdFW8Q==", 2024-04-20T10:22:44.152 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2024-04-20T10:22:44.152 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2024-04-20T10:22:44.152 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow *", 2024-04-20T10:22:44.152 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow *", 2024-04-20T10:22:44.152 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2024-04-20T10:22:44.153 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T10:22:44.153 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T10:22:44.153 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T10:22:44.153 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mds.c", 2024-04-20T10:22:44.153 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQDOliNmnDJBMxAA0cFAA1boV1mn/BUzaMxU7g==", 2024-04-20T10:22:44.153 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2024-04-20T10:22:44.153 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2024-04-20T10:22:44.153 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow *", 2024-04-20T10:22:44.153 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow *", 2024-04-20T10:22:44.153 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2024-04-20T10:22:44.153 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T10:22:44.153 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T10:22:44.153 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T10:22:44.153 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mds.d", 2024-04-20T10:22:44.154 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQDPliNmjLI0ExAA8AMkCBO3j9STy8+ylXRJew==", 2024-04-20T10:22:44.154 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2024-04-20T10:22:44.154 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2024-04-20T10:22:44.154 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow *", 2024-04-20T10:22:44.154 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow *", 2024-04-20T10:22:44.154 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2024-04-20T10:22:44.154 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T10:22:44.154 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T10:22:44.154 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T10:22:44.154 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.0", 2024-04-20T10:22:44.155 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQDTliNmJWIXExAAK2RnVtCRRoJ+B1T9D+wOLg==", 2024-04-20T10:22:44.155 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2024-04-20T10:22:44.155 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2024-04-20T10:22:44.155 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2024-04-20T10:22:44.155 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2024-04-20T10:22:44.155 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T10:22:44.155 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T10:22:44.155 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T10:22:44.155 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.1", 2024-04-20T10:22:44.155 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQDWliNmaNFtLRAArWQ2gjzzaKKM0EH93+fL0w==", 2024-04-20T10:22:44.155 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2024-04-20T10:22:44.155 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2024-04-20T10:22:44.155 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2024-04-20T10:22:44.155 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2024-04-20T10:22:44.156 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T10:22:44.156 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T10:22:44.156 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T10:22:44.156 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.2", 2024-04-20T10:22:44.156 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQDZliNmd3VvOBAADO8HDWYGf6eGBT3/9tNOBg==", 2024-04-20T10:22:44.156 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2024-04-20T10:22:44.156 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2024-04-20T10:22:44.156 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2024-04-20T10:22:44.156 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2024-04-20T10:22:44.156 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T10:22:44.156 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T10:22:44.156 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T10:22:44.157 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.3", 2024-04-20T10:22:44.157 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQDdliNmfMFZFxAADSklm9tSKlQcw93sDLWJpw==", 2024-04-20T10:22:44.157 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2024-04-20T10:22:44.157 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2024-04-20T10:22:44.157 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2024-04-20T10:22:44.157 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2024-04-20T10:22:44.157 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T10:22:44.157 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T10:22:44.157 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T10:22:44.157 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.4", 2024-04-20T10:22:44.157 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQDjliNmru+LKxAAtIKi2DpRkATWzvsGUdxlkQ==", 2024-04-20T10:22:44.157 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2024-04-20T10:22:44.158 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2024-04-20T10:22:44.158 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2024-04-20T10:22:44.158 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2024-04-20T10:22:44.158 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T10:22:44.158 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T10:22:44.158 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T10:22:44.158 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.5", 2024-04-20T10:22:44.158 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQDmliNmglWkOBAAONYjMcLSddLPq32d5xa5eA==", 2024-04-20T10:22:44.158 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2024-04-20T10:22:44.158 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2024-04-20T10:22:44.158 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2024-04-20T10:22:44.158 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2024-04-20T10:22:44.158 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T10:22:44.159 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T10:22:44.159 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T10:22:44.159 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.6", 2024-04-20T10:22:44.159 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQDqliNmhz3BBxAAOokJT1XFqXHTnDRJKuobDg==", 2024-04-20T10:22:44.159 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2024-04-20T10:22:44.159 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2024-04-20T10:22:44.159 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2024-04-20T10:22:44.159 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2024-04-20T10:22:44.159 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T10:22:44.159 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T10:22:44.159 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T10:22:44.159 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "osd.7", 2024-04-20T10:22:44.159 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQDtliNmovn8IhAAFubL/COYWIO1a4mGZ1VIwA==", 2024-04-20T10:22:44.159 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2024-04-20T10:22:44.160 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow profile osd", 2024-04-20T10:22:44.160 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile osd", 2024-04-20T10:22:44.160 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2024-04-20T10:22:44.160 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T10:22:44.160 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T10:22:44.160 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T10:22:44.160 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.0", 2024-04-20T10:22:44.160 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQDPliNmP7t4HxAACvwRBNvbdSP0BmDmaoo32g==", 2024-04-20T10:22:44.160 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2024-04-20T10:22:44.160 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2024-04-20T10:22:44.160 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow r", 2024-04-20T10:22:44.160 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow rw", 2024-04-20T10:22:44.160 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow rwx" 2024-04-20T10:22:44.160 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T10:22:44.161 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T10:22:44.161 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T10:22:44.161 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.1", 2024-04-20T10:22:44.161 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQDPliNmVjLRMRAAE2gUeLSlsGol8FjxI3UzMg==", 2024-04-20T10:22:44.161 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2024-04-20T10:22:44.161 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2024-04-20T10:22:44.161 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow r", 2024-04-20T10:22:44.161 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow rw", 2024-04-20T10:22:44.161 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow rwx" 2024-04-20T10:22:44.161 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T10:22:44.161 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T10:22:44.161 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T10:22:44.161 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.2", 2024-04-20T10:22:44.161 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQDPliNm3TMTKBAAXwbwz8I3RwfoFqLMMvqFhw==", 2024-04-20T10:22:44.161 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2024-04-20T10:22:44.161 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2024-04-20T10:22:44.162 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow r", 2024-04-20T10:22:44.162 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow rw", 2024-04-20T10:22:44.162 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow rwx" 2024-04-20T10:22:44.162 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T10:22:44.162 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T10:22:44.162 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T10:22:44.162 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.3", 2024-04-20T10:22:44.162 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQDPliNm3WkGOxAAPGD1K7WlKziDTqKU/xfqDQ==", 2024-04-20T10:22:44.162 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2024-04-20T10:22:44.162 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow", 2024-04-20T10:22:44.162 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow r", 2024-04-20T10:22:44.162 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow rw", 2024-04-20T10:22:44.162 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow rwx" 2024-04-20T10:22:44.163 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T10:22:44.163 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T10:22:44.163 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T10:22:44.163 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.admin", 2024-04-20T10:22:44.163 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQDNliNmYSb0JxAAuAj2WcluQMgpSBhyiCaS4Q==", 2024-04-20T10:22:44.163 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2024-04-20T10:22:44.163 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow *", 2024-04-20T10:22:44.163 INFO:teuthology.orchestra.run.smithi121.stdout: "mgr": "allow *", 2024-04-20T10:22:44.163 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow *", 2024-04-20T10:22:44.163 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2024-04-20T10:22:44.163 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T10:22:44.163 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T10:22:44.163 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T10:22:44.163 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.bootstrap-mds", 2024-04-20T10:22:44.164 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQD2liNmJdKdBBAA5mLI3UvZ6ELufATp732LSQ==", 2024-04-20T10:22:44.164 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2024-04-20T10:22:44.164 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile bootstrap-mds" 2024-04-20T10:22:44.164 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T10:22:44.164 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T10:22:44.164 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T10:22:44.164 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.bootstrap-mgr", 2024-04-20T10:22:44.164 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQD2liNmRf2dBBAA/lRz2vMKSClJR9DB4dsf7A==", 2024-04-20T10:22:44.164 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2024-04-20T10:22:44.164 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-20T10:22:44.164 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T10:22:44.165 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T10:22:44.165 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T10:22:44.165 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.bootstrap-osd", 2024-04-20T10:22:44.165 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQD2liNmKxeeBBAAe4QiDtftrx/PtREazVkyEg==", 2024-04-20T10:22:44.165 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2024-04-20T10:22:44.165 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile bootstrap-osd" 2024-04-20T10:22:44.165 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T10:22:44.165 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T10:22:44.165 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T10:22:44.165 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.bootstrap-rbd", 2024-04-20T10:22:44.165 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQD2liNmji+eBBAAYL0O59/XsTCwLfkk10gxtw==", 2024-04-20T10:22:44.165 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2024-04-20T10:22:44.165 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-20T10:22:44.165 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T10:22:44.166 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T10:22:44.166 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T10:22:44.166 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-20T10:22:44.166 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQD2liNmeUieBBAAfY76djcOTdPEribJg0s66Q==", 2024-04-20T10:22:44.166 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2024-04-20T10:22:44.166 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-20T10:22:44.166 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T10:22:44.166 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T10:22:44.166 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T10:22:44.166 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "client.bootstrap-rgw", 2024-04-20T10:22:44.166 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQD2liNmpWGeBBAAl6H6uW6xixNnfk3tfiI7cQ==", 2024-04-20T10:22:44.167 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2024-04-20T10:22:44.167 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-20T10:22:44.167 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T10:22:44.167 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T10:22:44.167 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T10:22:44.167 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mgr.x", 2024-04-20T10:22:44.167 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQDOliNmTemVHhAAV0A7vk9k1brOe/Pb/TwWWg==", 2024-04-20T10:22:44.167 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2024-04-20T10:22:44.167 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow *", 2024-04-20T10:22:44.167 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile mgr", 2024-04-20T10:22:44.167 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2024-04-20T10:22:44.168 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T10:22:44.168 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T10:22:44.168 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T10:22:44.168 INFO:teuthology.orchestra.run.smithi121.stdout: "entity": "mgr.y", 2024-04-20T10:22:44.168 INFO:teuthology.orchestra.run.smithi121.stdout: "key": "AQDOliNmokq5ExAAZUOG88huC1qC8Jq0L377ew==", 2024-04-20T10:22:44.168 INFO:teuthology.orchestra.run.smithi121.stdout: "caps": { 2024-04-20T10:22:44.168 INFO:teuthology.orchestra.run.smithi121.stdout: "mds": "allow *", 2024-04-20T10:22:44.168 INFO:teuthology.orchestra.run.smithi121.stdout: "mon": "allow profile mgr", 2024-04-20T10:22:44.168 INFO:teuthology.orchestra.run.smithi121.stdout: "osd": "allow *" 2024-04-20T10:22:44.168 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T10:22:44.168 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T10:22:44.168 INFO:teuthology.orchestra.run.smithi121.stdout: ] 2024-04-20T10:22:44.168 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T10:22:44.169 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.153+0000 7fc223000640 1 -- 172.21.15.121:0/132770616 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fc1f806c8f0 msgr2=0x7fc1f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:44.169 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.153+0000 7fc223000640 1 --2- 172.21.15.121:0/132770616 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fc1f806c8f0 0x7fc1f806edb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc20c00b7c0 tx=0x7fc20c008040 comp rx=0 tx=0).stop 2024-04-20T10:22:44.169 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.153+0000 7fc223000640 1 -- 172.21.15.121:0/132770616 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc21c0cc360 msgr2=0x7fc21c14e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:44.170 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.153+0000 7fc223000640 1 --2- 172.21.15.121:0/132770616 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc21c0cc360 0x7fc21c14e700 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc204000c00 tx=0x7fc2040046c0 comp rx=0 tx=0).stop 2024-04-20T10:22:44.170 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.154+0000 7fc223000640 1 -- 172.21.15.121:0/132770616 shutdown_connections 2024-04-20T10:22:44.170 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.154+0000 7fc223000640 1 --2- 172.21.15.121:0/132770616 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fc1f806c8f0 0x7fc1f806edb0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:44.170 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.154+0000 7fc223000640 1 --2- 172.21.15.121:0/132770616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc21c15b380 0x7fc21c1f82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:44.170 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.154+0000 7fc223000640 1 --2- 172.21.15.121:0/132770616 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fc21c0cc360 0x7fc21c14e700 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:44.170 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.154+0000 7fc223000640 1 --2- 172.21.15.121:0/132770616 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fc21c072d70 0x7fc21c15e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:44.170 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.154+0000 7fc223000640 1 -- 172.21.15.121:0/132770616 >> 172.21.15.121:0/132770616 conn(0x7fc21c0b1d70 msgr2=0x7fc21c0afa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:44.170 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.154+0000 7fc223000640 1 -- 172.21.15.121:0/132770616 shutdown_connections 2024-04-20T10:22:44.170 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.154+0000 7fc223000640 1 -- 172.21.15.121:0/132770616 wait complete. 2024-04-20T10:22:44.171 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-20T10:22:44.362 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.363+0000 7f8b9b000640 1 Processor -- start 2024-04-20T10:22:44.363 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.363+0000 7f8b9b000640 1 -- start start 2024-04-20T10:22:44.363 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.364+0000 7f8b9b000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8b94162c00 0x7f8b940d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:44.363 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.364+0000 7f8b9b000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8b9406f090 0x7f8b940d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:44.363 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.364+0000 7f8b9b000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b9406df70 con 0x7f8b9406fc30 2024-04-20T10:22:44.364 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.364+0000 7f8b9b000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f8b94162fe0 con 0x7f8b9406f090 2024-04-20T10:22:44.364 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.364+0000 7f8b9b000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f8b9406ec00 con 0x7f8b94162c00 2024-04-20T10:22:44.366 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.364+0000 7f8b99000640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f8b9406fc30 0x7f8b9406b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:46208/0 (socket says 172.21.15.121:46208) 2024-04-20T10:22:44.366 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.364+0000 7f8b99000640 1 -- 172.21.15.121:0/2188953560 learned_addr learned my addr 172.21.15.121:0/2188953560 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:44.366 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.365+0000 7f8b93400640 1 --2- 172.21.15.121:0/2188953560 >> v2:172.21.15.123:3300/0 conn(0x7f8b9406f090 0x7f8b940d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:44.366 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.365+0000 7f8b93e00640 1 --2- 172.21.15.121:0/2188953560 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8b94162c00 0x7f8b940d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:44.366 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.365+0000 7f8b92a00640 1 -- 172.21.15.121:0/2188953560 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1049641520 0 0) 0x7f8b9406df70 con 0x7f8b9406fc30 2024-04-20T10:22:44.366 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.365+0000 7f8b92a00640 1 -- 172.21.15.121:0/2188953560 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b78003520 con 0x7f8b9406fc30 2024-04-20T10:22:44.366 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.365+0000 7f8b92a00640 1 -- 172.21.15.121:0/2188953560 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3111540658 0 0) 0x7f8b78003520 con 0x7f8b9406fc30 2024-04-20T10:22:44.366 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.365+0000 7f8b92a00640 1 -- 172.21.15.121:0/2188953560 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8b94162c00 msgr2=0x7f8b940d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:44.366 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.365+0000 7f8b92a00640 1 --2- 172.21.15.121:0/2188953560 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8b94162c00 0x7f8b940d0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:44.366 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.365+0000 7f8b92a00640 1 -- 172.21.15.121:0/2188953560 >> v2:172.21.15.123:3300/0 conn(0x7f8b9406f090 msgr2=0x7f8b940d08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:44.366 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.365+0000 7f8b92a00640 1 --2- 172.21.15.121:0/2188953560 >> v2:172.21.15.123:3300/0 conn(0x7f8b9406f090 0x7f8b940d08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:44.367 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.365+0000 7f8b92a00640 1 -- 172.21.15.121:0/2188953560 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b9406e310 con 0x7f8b9406fc30 2024-04-20T10:22:44.367 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.365+0000 7f8b92a00640 1 -- 172.21.15.121:0/2188953560 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f8b8c0019f0 con 0x7f8b9406fc30 2024-04-20T10:22:44.367 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.365+0000 7f8b92a00640 1 -- 172.21.15.121:0/2188953560 >> v1:172.21.15.121:6789/0 conn(0x7f8b9406fc30 legacy=0x7f8b9406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:44.367 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.365+0000 7f8b92a00640 1 --2- 172.21.15.121:0/2188953560 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8b94162c00 0x7f8b780039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:44.367 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.365+0000 7f8b92a00640 1 --2- 172.21.15.121:0/2188953560 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b78003fe0 0x7f8b78004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:44.367 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.365+0000 7f8b92a00640 1 --2- 172.21.15.121:0/2188953560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b78004d40 0x7f8b780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:44.367 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.365+0000 7f8b92a00640 1 -- 172.21.15.121:0/2188953560 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9406ec00 con 0x7f8b78003fe0 2024-04-20T10:22:44.367 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.365+0000 7f8b92a00640 1 -- 172.21.15.121:0/2188953560 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8b7800bb00 con 0x7f8b78004d40 2024-04-20T10:22:44.367 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.365+0000 7f8b92a00640 1 -- 172.21.15.121:0/2188953560 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f8b7800bd00 con 0x7f8b94162c00 2024-04-20T10:22:44.367 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.366+0000 7f8b93400640 1 --2- 172.21.15.121:0/2188953560 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b78003fe0 0x7f8b78004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:44.367 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.366+0000 7f8b93e00640 1 --2- 172.21.15.121:0/2188953560 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8b94162c00 0x7f8b780039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:44.367 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.366+0000 7f8b99000640 1 --2- 172.21.15.121:0/2188953560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b78004d40 0x7f8b780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:44.368 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.366+0000 7f8b93400640 1 -- 172.21.15.121:0/2188953560 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8b94162c00 msgr2=0x7f8b780039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:44.368 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.366+0000 7f8b93400640 1 --2- 172.21.15.121:0/2188953560 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8b94162c00 0x7f8b780039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:44.368 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.366+0000 7f8b93400640 1 -- 172.21.15.121:0/2188953560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b78004d40 msgr2=0x7f8b780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:44.368 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.366+0000 7f8b93400640 1 --2- 172.21.15.121:0/2188953560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b78004d40 0x7f8b780051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:44.368 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.366+0000 7f8b93400640 1 -- 172.21.15.121:0/2188953560 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b78003520 con 0x7f8b78003fe0 2024-04-20T10:22:44.368 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.366+0000 7f8b93e00640 1 --2- 172.21.15.121:0/2188953560 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8b94162c00 0x7f8b780039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:44.369 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.367+0000 7f8b93400640 1 --2- 172.21.15.121:0/2188953560 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b78003fe0 0x7f8b78004800 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8b8000ba00 tx=0x7f8b78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5182bdf57701256 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:44.369 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.367+0000 7f8b92a00640 1 -- 172.21.15.121:0/2188953560 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b80036020 con 0x7f8b78003fe0 2024-04-20T10:22:44.369 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.367+0000 7f8b92a00640 1 -- 172.21.15.121:0/2188953560 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b80032660 con 0x7f8b78003fe0 2024-04-20T10:22:44.369 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.368+0000 7f8b92a00640 1 -- 172.21.15.121:0/2188953560 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8000b600 con 0x7f8b78003fe0 2024-04-20T10:22:44.369 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.368+0000 7f8b9b000640 1 -- 172.21.15.121:0/2188953560 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b78003fe0 msgr2=0x7f8b78004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:44.369 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.368+0000 7f8b9b000640 1 --2- 172.21.15.121:0/2188953560 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b78003fe0 0x7f8b78004800 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8b8000ba00 tx=0x7f8b78005830 comp rx=0 tx=0).stop 2024-04-20T10:22:44.369 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.368+0000 7f8b99000640 1 -- 172.21.15.121:0/2188953560 reap_dead start 2024-04-20T10:22:44.370 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.368+0000 7f8b9b000640 1 -- 172.21.15.121:0/2188953560 shutdown_connections 2024-04-20T10:22:44.370 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.368+0000 7f8b9b000640 1 -- 172.21.15.121:0/2188953560 >> 172.21.15.121:0/2188953560 conn(0x7f8b940b3350 msgr2=0x7f8b940b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:44.370 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.368+0000 7f8b9b000640 1 -- 172.21.15.121:0/2188953560 shutdown_connections 2024-04-20T10:22:44.370 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.368+0000 7f8b9b000640 1 -- 172.21.15.121:0/2188953560 wait complete. 2024-04-20T10:22:44.371 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.370+0000 7f8b9b000640 1 Processor -- start 2024-04-20T10:22:44.372 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.370+0000 7f8b9b000640 1 -- start start 2024-04-20T10:22:44.372 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.371+0000 7f8b9b000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b78003fe0 0x7f8b9409dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:44.372 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.371+0000 7f8b9b000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b94164e30 0x7f8b940ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:44.372 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.371+0000 7f8b9b000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8b941f99f0 0x7f8b940cd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:44.372 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.371+0000 7f8b9b000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9406dbb0 con 0x7f8b94164e30 2024-04-20T10:22:44.372 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.371+0000 7f8b9b000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9406e890 con 0x7f8b78003fe0 2024-04-20T10:22:44.372 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.371+0000 7f8b9b000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f8b9406dd30 con 0x7f8b941f99f0 2024-04-20T10:22:44.372 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.371+0000 7f8b93400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b94164e30 0x7f8b940ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:44.372 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.372+0000 7f8b93400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b94164e30 0x7f8b940ccb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34400/0 (socket says 172.21.15.121:34400) 2024-04-20T10:22:44.372 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.372+0000 7f8b93400640 1 -- 172.21.15.121:0/4280416901 learned_addr learned my addr 172.21.15.121:0/4280416901 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:44.372 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.372+0000 7f8b93400640 1 -- 172.21.15.121:0/4280416901 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8b941f99f0 msgr2=0x7f8b940cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:44.372 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.372+0000 7f8b93400640 1 --2- 172.21.15.121:0/4280416901 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8b941f99f0 0x7f8b940cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:44.372 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.372+0000 7f8b93400640 1 -- 172.21.15.121:0/4280416901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b78003fe0 msgr2=0x7f8b9409dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:44.372 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.372+0000 7f8b93400640 1 --2- 172.21.15.121:0/4280416901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b78003fe0 0x7f8b9409dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:44.373 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.372+0000 7f8b93400640 1 -- 172.21.15.121:0/4280416901 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b9409e0e0 con 0x7f8b94164e30 2024-04-20T10:22:44.373 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.372+0000 7f8b93400640 1 --2- 172.21.15.121:0/4280416901 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b94164e30 0x7f8b940ccb60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8b8000bb20 tx=0x7f8b80004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:44.373 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.373+0000 7f8b8be00640 1 -- 172.21.15.121:0/4280416901 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b80041070 con 0x7f8b94164e30 2024-04-20T10:22:44.373 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.373+0000 7f8b8be00640 1 -- 172.21.15.121:0/4280416901 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b80030a70 con 0x7f8b94164e30 2024-04-20T10:22:44.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.373+0000 7f8b8be00640 1 -- 172.21.15.121:0/4280416901 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b80036040 con 0x7f8b94164e30 2024-04-20T10:22:44.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.373+0000 7f8b9b000640 1 -- 172.21.15.121:0/4280416901 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b940cd3b0 con 0x7f8b94164e30 2024-04-20T10:22:44.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.373+0000 7f8b9b000640 1 -- 172.21.15.121:0/4280416901 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b94163440 con 0x7f8b94164e30 2024-04-20T10:22:44.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.374+0000 7f8b9b000640 1 -- 172.21.15.121:0/4280416901 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b94068980 con 0x7f8b94164e30 2024-04-20T10:22:44.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.375+0000 7f8b8be00640 1 -- 172.21.15.121:0/4280416901 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8b80039020 con 0x7f8b94164e30 2024-04-20T10:22:44.375 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.376+0000 7f8b8be00640 1 --2- 172.21.15.121:0/4280416901 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f8b7406c8f0 0x7f8b7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:44.375 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.376+0000 7f8b8be00640 1 -- 172.21.15.121:0/4280416901 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f8b800add40 con 0x7f8b94164e30 2024-04-20T10:22:44.375 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.376+0000 7f8b93e00640 1 --2- 172.21.15.121:0/4280416901 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f8b7406c8f0 0x7f8b7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:44.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.377+0000 7f8b93e00640 1 --2- 172.21.15.121:0/4280416901 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f8b7406c8f0 0x7f8b7406edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8b84006f00 tx=0x7f8b84006e30 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:44.380 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.380+0000 7f8b8be00640 1 -- 172.21.15.121:0/4280416901 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8b9409e0e0 con 0x7f8b94164e30 2024-04-20T10:22:44.643 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.643+0000 7f8b9b000640 1 -- 172.21.15.121:0/4280416901 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f8b940af0b0 con 0x7f8b94164e30 2024-04-20T10:22:44.648 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.649+0000 7f8b8be00640 1 -- 172.21.15.121:0/4280416901 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f8b8007c3c0 con 0x7f8b94164e30 2024-04-20T10:22:44.652 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.652+0000 7f8b9b000640 1 -- 172.21.15.121:0/4280416901 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f8b7406c8f0 msgr2=0x7f8b7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:44.652 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.652+0000 7f8b9b000640 1 --2- 172.21.15.121:0/4280416901 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f8b7406c8f0 0x7f8b7406edb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8b84006f00 tx=0x7f8b84006e30 comp rx=0 tx=0).stop 2024-04-20T10:22:44.652 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.652+0000 7f8b9b000640 1 -- 172.21.15.121:0/4280416901 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b94164e30 msgr2=0x7f8b940ccb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:44.652 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.652+0000 7f8b9b000640 1 --2- 172.21.15.121:0/4280416901 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b94164e30 0x7f8b940ccb60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8b8000bb20 tx=0x7f8b80004730 comp rx=0 tx=0).stop 2024-04-20T10:22:44.652 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.652+0000 7f8b9b000640 1 -- 172.21.15.121:0/4280416901 shutdown_connections 2024-04-20T10:22:44.652 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.652+0000 7f8b9b000640 1 --2- 172.21.15.121:0/4280416901 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8b941f99f0 0x7f8b940cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:44.652 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.652+0000 7f8b9b000640 1 --2- 172.21.15.121:0/4280416901 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f8b94164e30 0x7f8b940ccb60 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:44.652 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.652+0000 7f8b9b000640 1 --2- 172.21.15.121:0/4280416901 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f8b7406c8f0 0x7f8b7406edb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:44.653 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.652+0000 7f8b9b000640 1 --2- 172.21.15.121:0/4280416901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b78003fe0 0x7f8b9409dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:44.653 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.652+0000 7f8b9b000640 1 -- 172.21.15.121:0/4280416901 >> 172.21.15.121:0/4280416901 conn(0x7f8b940b3350 msgr2=0x7f8b940b0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:44.653 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.652+0000 7f8b9b000640 1 -- 172.21.15.121:0/4280416901 shutdown_connections 2024-04-20T10:22:44.653 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.653+0000 7f8b9b000640 1 -- 172.21.15.121:0/4280416901 wait complete. 2024-04-20T10:22:44.669 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-20T10:22:44.856 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.856+0000 7f3641800640 1 Processor -- start 2024-04-20T10:22:44.856 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.856+0000 7f3641800640 1 -- start start 2024-04-20T10:22:44.858 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.858+0000 7f3641800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f363c15a130 0x7f363c073470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:44.858 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.858+0000 7f3641800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f363c071a60 0x7f363c0739b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:44.859 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.858+0000 7f3641800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f363c15cc20 con 0x7f363c06f160 2024-04-20T10:22:44.859 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.858+0000 7f3641800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f363c15d7f0 con 0x7f363c071a60 2024-04-20T10:22:44.859 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.858+0000 7f3641800640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f363c15cec0 con 0x7f363c15a130 2024-04-20T10:22:44.859 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.858+0000 7f3633e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f363c071a60 0x7f363c0739b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:44.859 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.859+0000 7f3633e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f363c071a60 0x7f363c0739b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34414/0 (socket says 172.21.15.121:34414) 2024-04-20T10:22:44.859 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.859+0000 7f3633e00640 1 -- 172.21.15.121:0/1226290317 learned_addr learned my addr 172.21.15.121:0/1226290317 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:44.859 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.859+0000 7f3633e00640 1 -- 172.21.15.121:0/1226290317 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f363c15a130 msgr2=0x7f363c073470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:44.859 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.859+0000 7f363be00640 1 --2- 172.21.15.121:0/1226290317 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f363c15a130 0x7f363c073470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:44.859 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.859+0000 7f3633e00640 1 --2- 172.21.15.121:0/1226290317 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f363c15a130 0x7f363c073470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:44.859 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.859+0000 7f3633e00640 1 -- 172.21.15.121:0/1226290317 >> v1:172.21.15.123:6789/0 conn(0x7f363c06f160 legacy=0x7f363c06f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:44.859 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.859+0000 7f3633e00640 1 -- 172.21.15.121:0/1226290317 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f363c15d1a0 con 0x7f363c071a60 2024-04-20T10:22:44.860 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.860+0000 7f3633e00640 1 --2- 172.21.15.121:0/1226290317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f363c071a60 0x7f363c0739b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f3624009870 tx=0x7f362402b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd93e389fa333a22 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:44.861 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.860+0000 7f363b400640 1 -- 172.21.15.121:0/1226290317 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3624038070 con 0x7f363c071a60 2024-04-20T10:22:44.861 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.860+0000 7f363b400640 1 -- 172.21.15.121:0/1226290317 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3624031740 con 0x7f363c071a60 2024-04-20T10:22:44.861 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.860+0000 7f363b400640 1 -- 172.21.15.121:0/1226290317 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362402f050 con 0x7f363c071a60 2024-04-20T10:22:44.861 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.861+0000 7f3641800640 1 -- 172.21.15.121:0/1226290317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f363c071a60 msgr2=0x7f363c0739b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:44.861 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.861+0000 7f3641800640 1 --2- 172.21.15.121:0/1226290317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f363c071a60 0x7f363c0739b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f3624009870 tx=0x7f362402b7e0 comp rx=0 tx=0).stop 2024-04-20T10:22:44.861 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.861+0000 7f3641800640 1 -- 172.21.15.121:0/1226290317 shutdown_connections 2024-04-20T10:22:44.861 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.861+0000 7f3641800640 1 --2- 172.21.15.121:0/1226290317 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f363c071a60 0x7f363c0739b0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:44.861 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.861+0000 7f3641800640 1 --2- 172.21.15.121:0/1226290317 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f363c15a130 0x7f363c073470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:44.861 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.861+0000 7f3641800640 1 -- 172.21.15.121:0/1226290317 >> 172.21.15.121:0/1226290317 conn(0x7f363c0cffa0 msgr2=0x7f363c0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:44.861 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.861+0000 7f3641800640 1 -- 172.21.15.121:0/1226290317 shutdown_connections 2024-04-20T10:22:44.861 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.861+0000 7f3641800640 1 -- 172.21.15.121:0/1226290317 wait complete. 2024-04-20T10:22:44.862 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.863+0000 7f3641800640 1 Processor -- start 2024-04-20T10:22:44.862 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.863+0000 7f3641800640 1 -- start start 2024-04-20T10:22:44.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.864+0000 7f3641800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f363c06f160 0x7f363c0b5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:44.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.864+0000 7f3641800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f363c071a60 0x7f363c148250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:44.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.864+0000 7f3641800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f363c15a130 0x7f363c0c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:44.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.864+0000 7f3641800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f363c06ba80 con 0x7f363c15a130 2024-04-20T10:22:44.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.864+0000 7f3641800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f363c06b900 con 0x7f363c071a60 2024-04-20T10:22:44.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.864+0000 7f3641800640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f363c06bc00 con 0x7f363c06f160 2024-04-20T10:22:44.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.864+0000 7f3640e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f363c15a130 0x7f363c0c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:44.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.864+0000 7f3640e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f363c15a130 0x7f363c0c92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34422/0 (socket says 172.21.15.121:34422) 2024-04-20T10:22:44.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.864+0000 7f3640e00640 1 -- 172.21.15.121:0/651563335 learned_addr learned my addr 172.21.15.121:0/651563335 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:44.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.864+0000 7f363be00640 1 --2- 172.21.15.121:0/651563335 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f363c06f160 0x7f363c0b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:44.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.865+0000 7f3633e00640 1 --2- 172.21.15.121:0/651563335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f363c071a60 0x7f363c148250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:44.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.865+0000 7f3640e00640 1 -- 172.21.15.121:0/651563335 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f363c06f160 msgr2=0x7f363c0b5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:44.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.865+0000 7f3640e00640 1 --2- 172.21.15.121:0/651563335 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f363c06f160 0x7f363c0b5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:44.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.865+0000 7f3640e00640 1 -- 172.21.15.121:0/651563335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f363c071a60 msgr2=0x7f363c148250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:44.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.865+0000 7f3640e00640 1 --2- 172.21.15.121:0/651563335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f363c071a60 0x7f363c148250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:44.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.865+0000 7f3640e00640 1 -- 172.21.15.121:0/651563335 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f363c0b5ca0 con 0x7f363c15a130 2024-04-20T10:22:44.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.865+0000 7f363be00640 1 --2- 172.21.15.121:0/651563335 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f363c06f160 0x7f363c0b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:44.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.865+0000 7f3640e00640 1 --2- 172.21.15.121:0/651563335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f363c15a130 0x7f363c0c92f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f363400c780 tx=0x7f363400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:44.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.865+0000 7f3638c00640 1 -- 172.21.15.121:0/651563335 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3634004b60 con 0x7f363c15a130 2024-04-20T10:22:44.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.865+0000 7f3638c00640 1 -- 172.21.15.121:0/651563335 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3634004d00 con 0x7f363c15a130 2024-04-20T10:22:44.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.865+0000 7f3638c00640 1 -- 172.21.15.121:0/651563335 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363400aa90 con 0x7f363c15a130 2024-04-20T10:22:44.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.865+0000 7f3641800640 1 -- 172.21.15.121:0/651563335 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f363c1e60b0 con 0x7f363c15a130 2024-04-20T10:22:44.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.865+0000 7f3641800640 1 -- 172.21.15.121:0/651563335 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f363c1e6670 con 0x7f363c15a130 2024-04-20T10:22:44.867 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.867+0000 7f3641800640 1 -- 172.21.15.121:0/651563335 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f363c075b30 con 0x7f363c15a130 2024-04-20T10:22:44.867 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.867+0000 7f3638c00640 1 -- 172.21.15.121:0/651563335 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f363400ac30 con 0x7f363c15a130 2024-04-20T10:22:44.867 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.868+0000 7f3638c00640 1 --2- 172.21.15.121:0/651563335 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f361806c8f0 0x7f361806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:44.868 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.868+0000 7f3638c00640 1 -- 172.21.15.121:0/651563335 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f363408bd60 con 0x7f363c15a130 2024-04-20T10:22:44.868 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.869+0000 7f363be00640 1 --2- 172.21.15.121:0/651563335 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f361806c8f0 0x7f361806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:44.868 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.869+0000 7f363be00640 1 --2- 172.21.15.121:0/651563335 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f361806c8f0 0x7f361806edb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f362c00b820 tx=0x7f362c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:44.872 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:44.873+0000 7f3638c00640 1 -- 172.21.15.121:0/651563335 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3634020070 con 0x7f363c15a130 2024-04-20T10:22:45.124 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.124+0000 7f3641800640 1 -- 172.21.15.121:0/651563335 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f363c06d010 con 0x7f363c15a130 2024-04-20T10:22:45.129 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.129+0000 7f3638c00640 1 -- 172.21.15.121:0/651563335 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f363405a2b0 con 0x7f363c15a130 2024-04-20T10:22:45.133 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.133+0000 7f3641800640 1 -- 172.21.15.121:0/651563335 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f361806c8f0 msgr2=0x7f361806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:45.133 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.133+0000 7f3641800640 1 --2- 172.21.15.121:0/651563335 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f361806c8f0 0x7f361806edb0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f362c00b820 tx=0x7f362c008040 comp rx=0 tx=0).stop 2024-04-20T10:22:45.133 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.133+0000 7f3641800640 1 -- 172.21.15.121:0/651563335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f363c15a130 msgr2=0x7f363c0c92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:45.133 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.133+0000 7f3641800640 1 --2- 172.21.15.121:0/651563335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f363c15a130 0x7f363c0c92f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f363400c780 tx=0x7f363400cc40 comp rx=0 tx=0).stop 2024-04-20T10:22:45.133 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.133+0000 7f3641800640 1 -- 172.21.15.121:0/651563335 shutdown_connections 2024-04-20T10:22:45.133 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.133+0000 7f3641800640 1 --2- 172.21.15.121:0/651563335 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f363c15a130 0x7f363c0c92f0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:45.133 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.133+0000 7f3641800640 1 --2- 172.21.15.121:0/651563335 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f361806c8f0 0x7f361806edb0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:45.133 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.133+0000 7f3641800640 1 --2- 172.21.15.121:0/651563335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f363c071a60 0x7f363c148250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:45.133 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.133+0000 7f3641800640 1 --2- 172.21.15.121:0/651563335 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f363c06f160 0x7f363c0b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:45.133 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.133+0000 7f3641800640 1 -- 172.21.15.121:0/651563335 >> 172.21.15.121:0/651563335 conn(0x7f363c0cffa0 msgr2=0x7f363c0cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:45.133 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.133+0000 7f3641800640 1 -- 172.21.15.121:0/651563335 shutdown_connections 2024-04-20T10:22:45.134 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.133+0000 7f3641800640 1 -- 172.21.15.121:0/651563335 wait complete. 2024-04-20T10:22:45.150 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-20T10:22:45.343 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.344+0000 7f01d5a00640 1 Processor -- start 2024-04-20T10:22:45.344 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.344+0000 7f01d5a00640 1 -- start start 2024-04-20T10:22:45.345 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.345+0000 7f01d5a00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01d0156c20 0x7f01d006b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:45.345 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.345+0000 7f01d5a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f01d00d0360 0x7f01d00d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:45.345 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.345+0000 7f01d5a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01d0141c60 con 0x7f01d015ca40 2024-04-20T10:22:45.345 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.345+0000 7f01d5a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f01d00adab0 con 0x7f01d00d0360 2024-04-20T10:22:45.345 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.345+0000 7f01d5a00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f01d015cf90 con 0x7f01d0156c20 2024-04-20T10:22:45.346 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.346+0000 7f01cf400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f01d00d0360 0x7f01d00d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:45.346 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.346+0000 7f01cea00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01d0156c20 0x7f01d006b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:45.346 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.346+0000 7f01c7e00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f01d015ca40 0x7f01d00ad3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.121:39666/0 (socket says 172.21.15.121:39666) 2024-04-20T10:22:45.346 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.346+0000 7f01c7e00640 1 -- 172.21.15.121:0/4116938671 learned_addr learned my addr 172.21.15.121:0/4116938671 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:45.350 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.347+0000 7f01ce000640 1 -- 172.21.15.121:0/4116938671 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 915026341 0 0) 0x7f01d0141c60 con 0x7f01d015ca40 2024-04-20T10:22:45.351 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.347+0000 7f01cea00640 1 -- 172.21.15.121:0/4116938671 >> v2:172.21.15.123:3300/0 conn(0x7f01d00d0360 msgr2=0x7f01d00d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:45.351 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.347+0000 7f01cea00640 1 --2- 172.21.15.121:0/4116938671 >> v2:172.21.15.123:3300/0 conn(0x7f01d00d0360 0x7f01d00d07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:45.351 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.347+0000 7f01cea00640 1 -- 172.21.15.121:0/4116938671 >> v1:172.21.15.123:6789/0 conn(0x7f01d015ca40 legacy=0x7f01d00ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:45.351 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.347+0000 7f01cea00640 1 -- 172.21.15.121:0/4116938671 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d015d270 con 0x7f01d0156c20 2024-04-20T10:22:45.351 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.349+0000 7f01cea00640 1 --2- 172.21.15.121:0/4116938671 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01d0156c20 0x7f01d006b900 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f01c00097a0 tx=0x7f01c002b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5c8291fb8eaaf07 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:45.351 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.349+0000 7f01ce000640 1 -- 172.21.15.121:0/4116938671 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c0035020 con 0x7f01d0156c20 2024-04-20T10:22:45.351 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.350+0000 7f01ce000640 1 -- 172.21.15.121:0/4116938671 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01c0031830 con 0x7f01d0156c20 2024-04-20T10:22:45.351 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.350+0000 7f01ce000640 1 -- 172.21.15.121:0/4116938671 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c0031c00 con 0x7f01d0156c20 2024-04-20T10:22:45.351 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.350+0000 7f01d5a00640 1 -- 172.21.15.121:0/4116938671 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01d0156c20 msgr2=0x7f01d006b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:45.351 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.350+0000 7f01d5a00640 1 --2- 172.21.15.121:0/4116938671 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01d0156c20 0x7f01d006b900 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f01c00097a0 tx=0x7f01c002b8d0 comp rx=0 tx=0).stop 2024-04-20T10:22:45.351 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.350+0000 7f01d5a00640 1 -- 172.21.15.121:0/4116938671 shutdown_connections 2024-04-20T10:22:45.352 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.350+0000 7f01d5a00640 1 --2- 172.21.15.121:0/4116938671 >> v2:172.21.15.123:3300/0 conn(0x7f01d00d0360 0x7f01d00d07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:45.352 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.350+0000 7f01d5a00640 1 --2- 172.21.15.121:0/4116938671 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01d0156c20 0x7f01d006b900 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:45.352 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.350+0000 7f01d5a00640 1 -- 172.21.15.121:0/4116938671 >> 172.21.15.121:0/4116938671 conn(0x7f01d00b3350 msgr2=0x7f01d00b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:45.352 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.350+0000 7f01d5a00640 1 -- 172.21.15.121:0/4116938671 shutdown_connections 2024-04-20T10:22:45.352 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.350+0000 7f01d5a00640 1 -- 172.21.15.121:0/4116938671 wait complete. 2024-04-20T10:22:45.354 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.352+0000 7f01d5a00640 1 Processor -- start 2024-04-20T10:22:45.354 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.352+0000 7f01d5a00640 1 -- start start 2024-04-20T10:22:45.354 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.353+0000 7f01d5a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01d00d0360 0x7f01d01e7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:45.354 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.353+0000 7f01d5a00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01d0156c20 0x7f01d014ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:45.354 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.353+0000 7f01d5a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01d015ca40 0x7f01d0097c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:45.354 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.353+0000 7f01d5a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f01d006be40 con 0x7f01d00d0360 2024-04-20T10:22:45.354 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.353+0000 7f01d5a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f01d015d830 con 0x7f01d015ca40 2024-04-20T10:22:45.354 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.353+0000 7f01d5a00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f01d006bfc0 con 0x7f01d0156c20 2024-04-20T10:22:45.354 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.354+0000 7f01cea00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01d00d0360 0x7f01d01e7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:45.354 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.354+0000 7f01cea00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01d00d0360 0x7f01d01e7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34438/0 (socket says 172.21.15.121:34438) 2024-04-20T10:22:45.354 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.354+0000 7f01cea00640 1 -- 172.21.15.121:0/4012491048 learned_addr learned my addr 172.21.15.121:0/4012491048 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:45.355 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.354+0000 7f01cf400640 1 --2- 172.21.15.121:0/4012491048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01d015ca40 0x7f01d0097c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:45.355 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.354+0000 7f01c7e00640 1 --2- 172.21.15.121:0/4012491048 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01d0156c20 0x7f01d014ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:45.355 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.354+0000 7f01cea00640 1 -- 172.21.15.121:0/4012491048 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01d0156c20 msgr2=0x7f01d014ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:45.355 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.354+0000 7f01cea00640 1 --2- 172.21.15.121:0/4012491048 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01d0156c20 0x7f01d014ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:45.355 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.354+0000 7f01cea00640 1 -- 172.21.15.121:0/4012491048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01d015ca40 msgr2=0x7f01d0097c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:45.355 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.354+0000 7f01cea00640 1 --2- 172.21.15.121:0/4012491048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01d015ca40 0x7f01d0097c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:45.355 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.354+0000 7f01cea00640 1 -- 172.21.15.121:0/4012491048 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d00981a0 con 0x7f01d00d0360 2024-04-20T10:22:45.355 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.355+0000 7f01cea00640 1 --2- 172.21.15.121:0/4012491048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01d00d0360 0x7f01d01e7ae0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f01c002bee0 tx=0x7f01c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:45.356 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.355+0000 7f01c6a00640 1 -- 172.21.15.121:0/4012491048 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c003b980 con 0x7f01d00d0360 2024-04-20T10:22:45.356 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.355+0000 7f01c6a00640 1 -- 172.21.15.121:0/4012491048 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01c003bb20 con 0x7f01d00d0360 2024-04-20T10:22:45.356 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.355+0000 7f01c6a00640 1 -- 172.21.15.121:0/4012491048 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c00453f0 con 0x7f01d00d0360 2024-04-20T10:22:45.356 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.355+0000 7f01d5a00640 1 -- 172.21.15.121:0/4012491048 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01d0098470 con 0x7f01d00d0360 2024-04-20T10:22:45.356 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.355+0000 7f01d5a00640 1 -- 172.21.15.121:0/4012491048 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01d0098a30 con 0x7f01d00d0360 2024-04-20T10:22:45.356 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.356+0000 7f01d5a00640 1 -- 172.21.15.121:0/4012491048 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01d0068980 con 0x7f01d00d0360 2024-04-20T10:22:45.357 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.357+0000 7f01c6a00640 1 -- 172.21.15.121:0/4012491048 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f01c0038020 con 0x7f01d00d0360 2024-04-20T10:22:45.357 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.358+0000 7f01c6a00640 1 --2- 172.21.15.121:0/4012491048 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f01ac06c920 0x7f01ac06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:45.357 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.358+0000 7f01c6a00640 1 -- 172.21.15.121:0/4012491048 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f01c00ad5b0 con 0x7f01d00d0360 2024-04-20T10:22:45.358 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.358+0000 7f01c7e00640 1 --2- 172.21.15.121:0/4012491048 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f01ac06c920 0x7f01ac06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:45.358 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.359+0000 7f01c7e00640 1 --2- 172.21.15.121:0/4012491048 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f01ac06c920 0x7f01ac06ede0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f01d0141e00 tx=0x7f01b8007b50 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:45.362 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.362+0000 7f01c6a00640 1 -- 172.21.15.121:0/4012491048 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f01c0040070 con 0x7f01d00d0360 2024-04-20T10:22:45.631 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.632+0000 7f01d5a00640 1 -- 172.21.15.121:0/4012491048 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f01d00cfd90 con 0x7f01d00d0360 2024-04-20T10:22:45.635 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.636+0000 7f01c6a00640 1 -- 172.21.15.121:0/4012491048 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7f01c007bcb0 con 0x7f01d00d0360 2024-04-20T10:22:45.639 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.638+0000 7f01d5a00640 1 -- 172.21.15.121:0/4012491048 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f01ac06c920 msgr2=0x7f01ac06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:45.639 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.638+0000 7f01d5a00640 1 --2- 172.21.15.121:0/4012491048 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f01ac06c920 0x7f01ac06ede0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f01d0141e00 tx=0x7f01b8007b50 comp rx=0 tx=0).stop 2024-04-20T10:22:45.639 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.638+0000 7f01d5a00640 1 -- 172.21.15.121:0/4012491048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01d00d0360 msgr2=0x7f01d01e7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:45.639 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.638+0000 7f01d5a00640 1 --2- 172.21.15.121:0/4012491048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01d00d0360 0x7f01d01e7ae0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f01c002bee0 tx=0x7f01c00046c0 comp rx=0 tx=0).stop 2024-04-20T10:22:45.639 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.638+0000 7f01d5a00640 1 -- 172.21.15.121:0/4012491048 shutdown_connections 2024-04-20T10:22:45.639 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.638+0000 7f01d5a00640 1 --2- 172.21.15.121:0/4012491048 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f01ac06c920 0x7f01ac06ede0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:45.639 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.638+0000 7f01d5a00640 1 --2- 172.21.15.121:0/4012491048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01d015ca40 0x7f01d0097c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:45.639 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.638+0000 7f01d5a00640 1 --2- 172.21.15.121:0/4012491048 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f01d0156c20 0x7f01d014ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:45.639 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.638+0000 7f01d5a00640 1 --2- 172.21.15.121:0/4012491048 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f01d00d0360 0x7f01d01e7ae0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:45.639 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.638+0000 7f01d5a00640 1 -- 172.21.15.121:0/4012491048 >> 172.21.15.121:0/4012491048 conn(0x7f01d00b3350 msgr2=0x7f01d015ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:45.639 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.639+0000 7f01d5a00640 1 -- 172.21.15.121:0/4012491048 shutdown_connections 2024-04-20T10:22:45.640 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.639+0000 7f01d5a00640 1 -- 172.21.15.121:0/4012491048 wait complete. 2024-04-20T10:22:45.656 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-20T10:22:45.855 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.855+0000 7f3626800640 1 Processor -- start 2024-04-20T10:22:45.855 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.855+0000 7f3626800640 1 -- start start 2024-04-20T10:22:45.856 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.857+0000 7f3626800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f362015e770 0x7f36200b3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:45.857 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.857+0000 7f3626800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3620162210 0x7f3620162670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:45.857 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.857+0000 7f3626800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36200d3590 con 0x7f36200b6400 2024-04-20T10:22:45.857 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.857+0000 7f3626800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f362015eb50 con 0x7f362015e770 2024-04-20T10:22:45.857 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.857+0000 7f3626800640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f36200d3870 con 0x7f3620162210 2024-04-20T10:22:45.857 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.857+0000 7f3625400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f362015e770 0x7f36200b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:45.857 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.857+0000 7f3625400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f362015e770 0x7f36200b3490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34448/0 (socket says 172.21.15.121:34448) 2024-04-20T10:22:45.857 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.857+0000 7f3625400640 1 -- 172.21.15.121:0/2301434104 learned_addr learned my addr 172.21.15.121:0/2301434104 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:45.857 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.857+0000 7f3625e00640 1 --1- 172.21.15.121:0/2301434104 >> v1:172.21.15.121:6789/0 conn(0x7f36200b6400 0x7f36200d42c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:46222/0 (socket says 172.21.15.121:46222) 2024-04-20T10:22:45.858 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.858+0000 7f3624a00640 1 --2- 172.21.15.121:0/2301434104 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3620162210 0x7f3620162670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:45.858 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.858+0000 7f3625400640 1 -- 172.21.15.121:0/2301434104 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3620162210 msgr2=0x7f3620162670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:45.858 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.858+0000 7f3625400640 1 --2- 172.21.15.121:0/2301434104 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3620162210 0x7f3620162670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:45.858 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.858+0000 7f3625400640 1 -- 172.21.15.121:0/2301434104 >> v1:172.21.15.121:6789/0 conn(0x7f36200b6400 legacy=0x7f36200d42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:45.858 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.858+0000 7f3625400640 1 -- 172.21.15.121:0/2301434104 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36200d3b50 con 0x7f362015e770 2024-04-20T10:22:45.858 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.858+0000 7f3624a00640 1 --2- 172.21.15.121:0/2301434104 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3620162210 0x7f3620162670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:45.858 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.859+0000 7f3625400640 1 --2- 172.21.15.121:0/2301434104 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f362015e770 0x7f36200b3490 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3614009870 tx=0x7f361402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1be00207afcb0c8f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:45.859 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.859+0000 7f361fe00640 1 -- 172.21.15.121:0/2301434104 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3614035020 con 0x7f362015e770 2024-04-20T10:22:45.860 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.859+0000 7f361fe00640 1 -- 172.21.15.121:0/2301434104 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3614031640 con 0x7f362015e770 2024-04-20T10:22:45.860 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.859+0000 7f361fe00640 1 -- 172.21.15.121:0/2301434104 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361402d040 con 0x7f362015e770 2024-04-20T10:22:45.860 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.860+0000 7f3626800640 1 -- 172.21.15.121:0/2301434104 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f362015e770 msgr2=0x7f36200b3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:45.860 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.860+0000 7f3626800640 1 --2- 172.21.15.121:0/2301434104 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f362015e770 0x7f36200b3490 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3614009870 tx=0x7f361402b8f0 comp rx=0 tx=0).stop 2024-04-20T10:22:45.860 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.860+0000 7f3626800640 1 -- 172.21.15.121:0/2301434104 shutdown_connections 2024-04-20T10:22:45.860 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.860+0000 7f3626800640 1 --2- 172.21.15.121:0/2301434104 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3620162210 0x7f3620162670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:45.860 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.860+0000 7f3626800640 1 --2- 172.21.15.121:0/2301434104 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f362015e770 0x7f36200b3490 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:45.860 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.860+0000 7f3626800640 1 -- 172.21.15.121:0/2301434104 >> 172.21.15.121:0/2301434104 conn(0x7f362009d6d0 msgr2=0x7f36200b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:45.860 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.860+0000 7f3626800640 1 -- 172.21.15.121:0/2301434104 shutdown_connections 2024-04-20T10:22:45.860 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.860+0000 7f3626800640 1 -- 172.21.15.121:0/2301434104 wait complete. 2024-04-20T10:22:45.861 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.862+0000 7f3626800640 1 Processor -- start 2024-04-20T10:22:45.861 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.862+0000 7f3626800640 1 -- start start 2024-04-20T10:22:45.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.863+0000 7f3626800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f36200b6400 0x7f362006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:45.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.863+0000 7f3626800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f362015e770 0x7f362015bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:45.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.863+0000 7f3626800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3620162210 0x7f36200cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:45.863 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.863+0000 7f3626800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f36200b39d0 con 0x7f362015e770 2024-04-20T10:22:45.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.863+0000 7f3626800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f36200d3e30 con 0x7f3620162210 2024-04-20T10:22:45.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.863+0000 7f3626800640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f36200b3b50 con 0x7f36200b6400 2024-04-20T10:22:45.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.863+0000 7f3624a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f362015e770 0x7f362015bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:45.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.863+0000 7f3624a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f362015e770 0x7f362015bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:34450/0 (socket says 172.21.15.121:34450) 2024-04-20T10:22:45.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.863+0000 7f3624a00640 1 -- 172.21.15.121:0/230863921 learned_addr learned my addr 172.21.15.121:0/230863921 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:45.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.864+0000 7f3625400640 1 --2- 172.21.15.121:0/230863921 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f36200b6400 0x7f362006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:45.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.864+0000 7f3625e00640 1 --2- 172.21.15.121:0/230863921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3620162210 0x7f36200cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:45.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.864+0000 7f3624a00640 1 -- 172.21.15.121:0/230863921 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f36200b6400 msgr2=0x7f362006e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:45.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.864+0000 7f3624a00640 1 --2- 172.21.15.121:0/230863921 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f36200b6400 0x7f362006e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:45.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.864+0000 7f3624a00640 1 -- 172.21.15.121:0/230863921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3620162210 msgr2=0x7f36200cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:45.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.864+0000 7f3624a00640 1 --2- 172.21.15.121:0/230863921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3620162210 0x7f36200cd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:45.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.864+0000 7f3624a00640 1 -- 172.21.15.121:0/230863921 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f362006e860 con 0x7f362015e770 2024-04-20T10:22:45.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.864+0000 7f3625e00640 1 --2- 172.21.15.121:0/230863921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3620162210 0x7f36200cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:45.864 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.864+0000 7f3624a00640 1 --2- 172.21.15.121:0/230863921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f362015e770 0x7f362015bb60 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f360c00c7f0 tx=0x7f360c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:45.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.864+0000 7f361d600640 1 -- 172.21.15.121:0/230863921 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360c015070 con 0x7f362015e770 2024-04-20T10:22:45.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.864+0000 7f361d600640 1 -- 172.21.15.121:0/230863921 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f360c007820 con 0x7f362015e770 2024-04-20T10:22:45.865 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.864+0000 7f361d600640 1 -- 172.21.15.121:0/230863921 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360c007bd0 con 0x7f362015e770 2024-04-20T10:22:45.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.864+0000 7f3626800640 1 -- 172.21.15.121:0/230863921 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f362006d280 con 0x7f362015e770 2024-04-20T10:22:45.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.864+0000 7f3626800640 1 -- 172.21.15.121:0/230863921 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f362006d840 con 0x7f362015e770 2024-04-20T10:22:45.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.866+0000 7f3626800640 1 -- 172.21.15.121:0/230863921 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3620075b30 con 0x7f362015e770 2024-04-20T10:22:45.866 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.867+0000 7f361d600640 1 -- 172.21.15.121:0/230863921 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f360c018070 con 0x7f362015e770 2024-04-20T10:22:45.867 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.867+0000 7f361d600640 1 --2- 172.21.15.121:0/230863921 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f360006c8f0 0x7f360006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:45.867 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.868+0000 7f361d600640 1 -- 172.21.15.121:0/230863921 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f360c08bb40 con 0x7f362015e770 2024-04-20T10:22:45.867 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.868+0000 7f3625400640 1 --2- 172.21.15.121:0/230863921 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f360006c8f0 0x7f360006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:45.868 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.868+0000 7f3625400640 1 --2- 172.21.15.121:0/230863921 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f360006c8f0 0x7f360006edb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3614036000 tx=0x7f361403c040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:45.871 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:45.872+0000 7f361d600640 1 -- 172.21.15.121:0/230863921 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f360c05a090 con 0x7f362015e770 2024-04-20T10:22:46.151 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.152+0000 7f3626800640 1 -- 172.21.15.121:0/230863921 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f36200cb170 con 0x7f362015e770 2024-04-20T10:22:46.156 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.156+0000 7f361d600640 1 -- 172.21.15.121:0/230863921 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f360c059990 con 0x7f362015e770 2024-04-20T10:22:46.158 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.158+0000 7f3626800640 1 -- 172.21.15.121:0/230863921 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f360006c8f0 msgr2=0x7f360006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:46.158 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.158+0000 7f3626800640 1 --2- 172.21.15.121:0/230863921 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f360006c8f0 0x7f360006edb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3614036000 tx=0x7f361403c040 comp rx=0 tx=0).stop 2024-04-20T10:22:46.158 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.158+0000 7f3626800640 1 -- 172.21.15.121:0/230863921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f362015e770 msgr2=0x7f362015bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:46.158 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.158+0000 7f3626800640 1 --2- 172.21.15.121:0/230863921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f362015e770 0x7f362015bb60 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f360c00c7f0 tx=0x7f360c00ccb0 comp rx=0 tx=0).stop 2024-04-20T10:22:46.159 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.159+0000 7f3626800640 1 -- 172.21.15.121:0/230863921 shutdown_connections 2024-04-20T10:22:46.159 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.159+0000 7f3626800640 1 --2- 172.21.15.121:0/230863921 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f360006c8f0 0x7f360006edb0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:46.159 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.159+0000 7f3626800640 1 --2- 172.21.15.121:0/230863921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3620162210 0x7f36200cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:46.159 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.159+0000 7f3626800640 1 --2- 172.21.15.121:0/230863921 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f362015e770 0x7f362015bb60 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:46.159 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.159+0000 7f3626800640 1 --2- 172.21.15.121:0/230863921 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f36200b6400 0x7f362006e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:46.159 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.159+0000 7f3626800640 1 -- 172.21.15.121:0/230863921 >> 172.21.15.121:0/230863921 conn(0x7f362009d6d0 msgr2=0x7f36200b0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:46.160 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.159+0000 7f3626800640 1 -- 172.21.15.121:0/230863921 shutdown_connections 2024-04-20T10:22:46.160 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.159+0000 7f3626800640 1 -- 172.21.15.121:0/230863921 wait complete. 2024-04-20T10:22:46.178 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-20T10:22:46.375 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.372+0000 7fe625a00640 1 Processor -- start 2024-04-20T10:22:46.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.372+0000 7fe625a00640 1 -- start start 2024-04-20T10:22:46.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.374+0000 7fe625a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe62006dcb0 0x7fe620071b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:46.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.374+0000 7fe625a00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe620072090 0x7fe6200d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:46.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.374+0000 7fe625a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6200ade40 con 0x7fe62015b3b0 2024-04-20T10:22:46.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.374+0000 7fe625a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fe62006f1b0 con 0x7fe62006dcb0 2024-04-20T10:22:46.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.374+0000 7fe625a00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe6200ae0e0 con 0x7fe620072090 2024-04-20T10:22:46.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.374+0000 7fe617e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe62006dcb0 0x7fe620071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:46.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.375+0000 7fe61f400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe620072090 0x7fe6200d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:46.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.375+0000 7fe61ea00640 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fe62015b3b0 0x7fe62006eaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.121:43250/0 (socket says 172.21.15.121:43250) 2024-04-20T10:22:46.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.375+0000 7fe61ea00640 1 -- 172.21.15.121:0/2611525027 learned_addr learned my addr 172.21.15.121:0/2611525027 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:46.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.375+0000 7fe617e00640 1 -- 172.21.15.121:0/2611525027 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe620072090 msgr2=0x7fe6200d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:46.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.375+0000 7fe617e00640 1 --2- 172.21.15.121:0/2611525027 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe620072090 0x7fe6200d0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:46.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.376+0000 7fe617e00640 1 -- 172.21.15.121:0/2611525027 >> v1:172.21.15.123:6789/0 conn(0x7fe62015b3b0 legacy=0x7fe62006eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:46.377 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.376+0000 7fe617e00640 1 -- 172.21.15.121:0/2611525027 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe62015b790 con 0x7fe62006dcb0 2024-04-20T10:22:46.378 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.377+0000 7fe617e00640 1 --2- 172.21.15.121:0/2611525027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe62006dcb0 0x7fe620071b50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe6080097a0 tx=0x7fe60802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af226f53beeea978 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:46.378 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.377+0000 7fe61e000640 1 -- 172.21.15.121:0/2611525027 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe608035020 con 0x7fe62006dcb0 2024-04-20T10:22:46.378 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.377+0000 7fe61e000640 1 -- 172.21.15.121:0/2611525027 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe608031830 con 0x7fe62006dcb0 2024-04-20T10:22:46.378 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.377+0000 7fe61e000640 1 -- 172.21.15.121:0/2611525027 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe608031c00 con 0x7fe62006dcb0 2024-04-20T10:22:46.378 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.378+0000 7fe625a00640 1 -- 172.21.15.121:0/2611525027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe62006dcb0 msgr2=0x7fe620071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:46.378 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.378+0000 7fe625a00640 1 --2- 172.21.15.121:0/2611525027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe62006dcb0 0x7fe620071b50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe6080097a0 tx=0x7fe60802b8d0 comp rx=0 tx=0).stop 2024-04-20T10:22:46.378 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.378+0000 7fe625a00640 1 -- 172.21.15.121:0/2611525027 shutdown_connections 2024-04-20T10:22:46.378 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.378+0000 7fe625a00640 1 --2- 172.21.15.121:0/2611525027 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe620072090 0x7fe6200d0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:46.378 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.378+0000 7fe625a00640 1 --2- 172.21.15.121:0/2611525027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe62006dcb0 0x7fe620071b50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:46.378 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.378+0000 7fe625a00640 1 -- 172.21.15.121:0/2611525027 >> 172.21.15.121:0/2611525027 conn(0x7fe6200b3350 msgr2=0x7fe6200b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:46.379 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.378+0000 7fe625a00640 1 -- 172.21.15.121:0/2611525027 shutdown_connections 2024-04-20T10:22:46.379 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.378+0000 7fe625a00640 1 -- 172.21.15.121:0/2611525027 wait complete. 2024-04-20T10:22:46.380 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.380+0000 7fe625a00640 1 Processor -- start 2024-04-20T10:22:46.380 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.381+0000 7fe625a00640 1 -- start start 2024-04-20T10:22:46.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.382+0000 7fe625a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe62006dcb0 0x7fe620151160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:46.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.382+0000 7fe625a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe62015b3b0 0x7fe62015dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:46.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.382+0000 7fe625a00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe620159f10 0x7fe62015a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:46.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.382+0000 7fe625a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe6200ae940 con 0x7fe62006dcb0 2024-04-20T10:22:46.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.382+0000 7fe625a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe6200ae660 con 0x7fe62015b3b0 2024-04-20T10:22:46.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.382+0000 7fe625a00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe6200aec20 con 0x7fe620159f10 2024-04-20T10:22:46.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.382+0000 7fe61ea00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe62006dcb0 0x7fe620151160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:46.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.382+0000 7fe61ea00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe62006dcb0 0x7fe620151160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:47920/0 (socket says 172.21.15.121:47920) 2024-04-20T10:22:46.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.382+0000 7fe61ea00640 1 -- 172.21.15.121:0/4012043497 learned_addr learned my addr 172.21.15.121:0/4012043497 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:46.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.382+0000 7fe61f400640 1 --2- 172.21.15.121:0/4012043497 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe620159f10 0x7fe62015a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:46.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.382+0000 7fe61ea00640 1 -- 172.21.15.121:0/4012043497 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe620159f10 msgr2=0x7fe62015a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:46.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.383+0000 7fe617e00640 1 --2- 172.21.15.121:0/4012043497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe62015b3b0 0x7fe62015dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:46.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.383+0000 7fe61ea00640 1 --2- 172.21.15.121:0/4012043497 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe620159f10 0x7fe62015a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:46.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.383+0000 7fe61ea00640 1 -- 172.21.15.121:0/4012043497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe62015b3b0 msgr2=0x7fe62015dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:46.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.383+0000 7fe61ea00640 1 --2- 172.21.15.121:0/4012043497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe62015b3b0 0x7fe62015dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:46.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.383+0000 7fe61ea00640 1 -- 172.21.15.121:0/4012043497 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe620165620 con 0x7fe62006dcb0 2024-04-20T10:22:46.384 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.383+0000 7fe61ea00640 1 --2- 172.21.15.121:0/4012043497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe62006dcb0 0x7fe620151160 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe61000c480 tx=0x7fe61000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:46.384 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.383+0000 7fe616a00640 1 -- 172.21.15.121:0/4012043497 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe610017070 con 0x7fe62006dcb0 2024-04-20T10:22:46.384 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.383+0000 7fe616a00640 1 -- 172.21.15.121:0/4012043497 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe610005ce0 con 0x7fe62006dcb0 2024-04-20T10:22:46.384 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.383+0000 7fe616a00640 1 -- 172.21.15.121:0/4012043497 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6100072f0 con 0x7fe62006dcb0 2024-04-20T10:22:46.384 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.383+0000 7fe625a00640 1 -- 172.21.15.121:0/4012043497 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe620165900 con 0x7fe62006dcb0 2024-04-20T10:22:46.384 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.383+0000 7fe625a00640 1 -- 172.21.15.121:0/4012043497 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe620165ec0 con 0x7fe62006dcb0 2024-04-20T10:22:46.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.384+0000 7fe625a00640 1 -- 172.21.15.121:0/4012043497 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe620068980 con 0x7fe62006dcb0 2024-04-20T10:22:46.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.385+0000 7fe616a00640 1 -- 172.21.15.121:0/4012043497 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe61001a070 con 0x7fe62006dcb0 2024-04-20T10:22:46.386 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.386+0000 7fe616a00640 1 --2- 172.21.15.121:0/4012043497 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe5fc06c920 0x7fe5fc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:46.386 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.386+0000 7fe616a00640 1 -- 172.21.15.121:0/4012043497 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7fe61008acc0 con 0x7fe62006dcb0 2024-04-20T10:22:46.386 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.387+0000 7fe617e00640 1 --2- 172.21.15.121:0/4012043497 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe5fc06c920 0x7fe5fc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:46.387 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.387+0000 7fe617e00640 1 --2- 172.21.15.121:0/4012043497 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe5fc06c920 0x7fe5fc06ede0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe608036000 tx=0x7fe608008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:46.390 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.391+0000 7fe616a00640 1 -- 172.21.15.121:0/4012043497 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe610091080 con 0x7fe62006dcb0 2024-04-20T10:22:46.663 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.663+0000 7fe625a00640 1 -- 172.21.15.121:0/4012043497 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7fe6201516a0 con 0x7fe62006dcb0 2024-04-20T10:22:46.667 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.668+0000 7fe616a00640 1 -- 172.21.15.121:0/4012043497 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) ==== 97+0+0 (secure 0 0 0) 0x7fe610058c50 con 0x7fe62006dcb0 2024-04-20T10:22:46.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.671+0000 7fe625a00640 1 -- 172.21.15.121:0/4012043497 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe5fc06c920 msgr2=0x7fe5fc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:46.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.671+0000 7fe625a00640 1 --2- 172.21.15.121:0/4012043497 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe5fc06c920 0x7fe5fc06ede0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe608036000 tx=0x7fe608008030 comp rx=0 tx=0).stop 2024-04-20T10:22:46.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.671+0000 7fe625a00640 1 -- 172.21.15.121:0/4012043497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe62006dcb0 msgr2=0x7fe620151160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:46.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.671+0000 7fe625a00640 1 --2- 172.21.15.121:0/4012043497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe62006dcb0 0x7fe620151160 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe61000c480 tx=0x7fe61000c940 comp rx=0 tx=0).stop 2024-04-20T10:22:46.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.671+0000 7fe625a00640 1 -- 172.21.15.121:0/4012043497 shutdown_connections 2024-04-20T10:22:46.672 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.671+0000 7fe625a00640 1 --2- 172.21.15.121:0/4012043497 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe620159f10 0x7fe62015a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:46.672 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.671+0000 7fe625a00640 1 --2- 172.21.15.121:0/4012043497 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe5fc06c920 0x7fe5fc06ede0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:46.672 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.671+0000 7fe625a00640 1 --2- 172.21.15.121:0/4012043497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe62015b3b0 0x7fe62015dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:46.672 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.671+0000 7fe625a00640 1 --2- 172.21.15.121:0/4012043497 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe62006dcb0 0x7fe620151160 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:46.672 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.672+0000 7fe625a00640 1 -- 172.21.15.121:0/4012043497 >> 172.21.15.121:0/4012043497 conn(0x7fe6200b3350 msgr2=0x7fe620073460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:46.672 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.672+0000 7fe625a00640 1 -- 172.21.15.121:0/4012043497 shutdown_connections 2024-04-20T10:22:46.672 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.672+0000 7fe625a00640 1 -- 172.21.15.121:0/4012043497 wait complete. 2024-04-20T10:22:46.690 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-20T10:22:46.891 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.891+0000 7f70aa800640 1 Processor -- start 2024-04-20T10:22:46.891 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.892+0000 7f70aa800640 1 -- start start 2024-04-20T10:22:46.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.893+0000 7f70aa800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f70a4162c00 0x7f70a40d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:46.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.893+0000 7f70aa800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70a406fc30 0x7f70a40d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:46.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.893+0000 7f70aa800640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70a406df70 con 0x7f70a406f090 2024-04-20T10:22:46.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.893+0000 7f70aa800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f70a4162fe0 con 0x7f70a4162c00 2024-04-20T10:22:46.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.893+0000 7f70aa800640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f70a406ec00 con 0x7f70a406fc30 2024-04-20T10:22:46.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.893+0000 7f70a2a00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f70a406f090 0x7f70a406b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:44834/0 (socket says 172.21.15.121:44834) 2024-04-20T10:22:46.893 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.893+0000 7f70a2a00640 1 -- 172.21.15.121:0/1273517170 learned_addr learned my addr 172.21.15.121:0/1273517170 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:46.895 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.894+0000 7f70a3e00640 1 --2- 172.21.15.121:0/1273517170 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70a406fc30 0x7f70a40d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:46.895 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.894+0000 7f70a3400640 1 --2- 172.21.15.121:0/1273517170 >> v2:172.21.15.123:3300/0 conn(0x7f70a4162c00 0x7f70a40d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:46.895 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.894+0000 7f70a2000640 1 -- 172.21.15.121:0/1273517170 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3150152529 0 0) 0x7f70a406df70 con 0x7f70a406f090 2024-04-20T10:22:46.895 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.894+0000 7f70a2000640 1 -- 172.21.15.121:0/1273517170 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7084003520 con 0x7f70a406f090 2024-04-20T10:22:46.895 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.894+0000 7f70a2000640 1 -- 172.21.15.121:0/1273517170 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 124794905 0 0) 0x7f7084003520 con 0x7f70a406f090 2024-04-20T10:22:46.895 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.894+0000 7f70a2000640 1 -- 172.21.15.121:0/1273517170 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70a406fc30 msgr2=0x7f70a40d08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:46.896 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.894+0000 7f70a2000640 1 --2- 172.21.15.121:0/1273517170 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70a406fc30 0x7f70a40d08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:46.896 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.894+0000 7f70a2000640 1 -- 172.21.15.121:0/1273517170 >> v2:172.21.15.123:3300/0 conn(0x7f70a4162c00 msgr2=0x7f70a40d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:46.896 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.894+0000 7f70a2000640 1 --2- 172.21.15.121:0/1273517170 >> v2:172.21.15.123:3300/0 conn(0x7f70a4162c00 0x7f70a40d0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:46.896 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.894+0000 7f70a2000640 1 -- 172.21.15.121:0/1273517170 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70a406e310 con 0x7f70a406f090 2024-04-20T10:22:46.896 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.894+0000 7f70a2000640 1 -- 172.21.15.121:0/1273517170 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f708c001670 con 0x7f70a406f090 2024-04-20T10:22:46.896 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.894+0000 7f70a3400640 1 --2- 172.21.15.121:0/1273517170 >> v2:172.21.15.123:3300/0 conn(0x7f70a4162c00 0x7f70a40d0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:22:46.896 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.894+0000 7f70a2000640 1 -- 172.21.15.121:0/1273517170 >> v1:172.21.15.121:6789/0 conn(0x7f70a406f090 legacy=0x7f70a406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:46.896 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.894+0000 7f70a2000640 1 --2- 172.21.15.121:0/1273517170 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70840039b0 0x7f7084003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:46.896 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.895+0000 7f70a2000640 1 --2- 172.21.15.121:0/1273517170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7084004800 0x7f7084004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:46.896 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.895+0000 7f70a2000640 1 --2- 172.21.15.121:0/1273517170 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70a406fc30 0x7f70840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:46.896 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.895+0000 7f70a2000640 1 -- 172.21.15.121:0/1273517170 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f70a406ec00 con 0x7f70840039b0 2024-04-20T10:22:46.896 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.895+0000 7f70a2000640 1 -- 172.21.15.121:0/1273517170 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f708400b9f0 con 0x7f7084004800 2024-04-20T10:22:46.896 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.895+0000 7f70a2000640 1 -- 172.21.15.121:0/1273517170 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f708400bbf0 con 0x7f70a406fc30 2024-04-20T10:22:46.896 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.895+0000 7f70a3400640 1 --2- 172.21.15.121:0/1273517170 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70840039b0 0x7f7084003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:46.897 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.895+0000 7f70a3e00640 1 --2- 172.21.15.121:0/1273517170 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70a406fc30 0x7f70840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:46.897 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.895+0000 7f70a2a00640 1 --2- 172.21.15.121:0/1273517170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7084004800 0x7f7084004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:46.897 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.895+0000 7f70a3400640 1 -- 172.21.15.121:0/1273517170 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70a406fc30 msgr2=0x7f70840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:46.897 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.895+0000 7f70a3400640 1 --2- 172.21.15.121:0/1273517170 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70a406fc30 0x7f70840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:46.897 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.895+0000 7f70a3400640 1 -- 172.21.15.121:0/1273517170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7084004800 msgr2=0x7f7084004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:46.897 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.895+0000 7f70a3400640 1 --2- 172.21.15.121:0/1273517170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7084004800 0x7f7084004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:46.897 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.895+0000 7f70a3400640 1 -- 172.21.15.121:0/1273517170 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7084003520 con 0x7f70840039b0 2024-04-20T10:22:46.897 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.895+0000 7f70a3e00640 1 --2- 172.21.15.121:0/1273517170 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70a406fc30 0x7f70840051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:46.897 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.895+0000 7f70a2a00640 1 --2- 172.21.15.121:0/1273517170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7084004800 0x7f7084004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:46.898 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.897+0000 7f70a3400640 1 --2- 172.21.15.121:0/1273517170 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70840039b0 0x7f7084003dd0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7094006fd0 tx=0x7f7084005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e525a498001f191f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:46.898 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.897+0000 7f70a2000640 1 -- 172.21.15.121:0/1273517170 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709400f040 con 0x7f70840039b0 2024-04-20T10:22:46.898 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.897+0000 7f70a2000640 1 -- 172.21.15.121:0/1273517170 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7094004230 con 0x7f70840039b0 2024-04-20T10:22:46.898 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.897+0000 7f70a2000640 1 -- 172.21.15.121:0/1273517170 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709402d040 con 0x7f70840039b0 2024-04-20T10:22:46.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.898+0000 7f70aa800640 1 -- 172.21.15.121:0/1273517170 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70840039b0 msgr2=0x7f7084003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:46.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.898+0000 7f70aa800640 1 --2- 172.21.15.121:0/1273517170 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70840039b0 0x7f7084003dd0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7094006fd0 tx=0x7f7084005830 comp rx=0 tx=0).stop 2024-04-20T10:22:46.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.898+0000 7f70a3e00640 1 -- 172.21.15.121:0/1273517170 reap_dead start 2024-04-20T10:22:46.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.898+0000 7f70aa800640 1 -- 172.21.15.121:0/1273517170 shutdown_connections 2024-04-20T10:22:46.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.898+0000 7f70aa800640 1 -- 172.21.15.121:0/1273517170 >> 172.21.15.121:0/1273517170 conn(0x7f70a40b3350 msgr2=0x7f70a40b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:46.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.898+0000 7f70aa800640 1 -- 172.21.15.121:0/1273517170 shutdown_connections 2024-04-20T10:22:46.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.898+0000 7f70aa800640 1 -- 172.21.15.121:0/1273517170 wait complete. 2024-04-20T10:22:46.899 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.900+0000 7f70aa800640 1 Processor -- start 2024-04-20T10:22:46.900 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.900+0000 7f70aa800640 1 -- start start 2024-04-20T10:22:46.901 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.901+0000 7f70aa800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70a414e530 0x7f70a41fd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:46.901 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.901+0000 7f70aa800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70a414e910 0x7f70a41f8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:46.901 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.901+0000 7f70aa800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70a41f94b0 0x7f70a41651b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:46.901 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.901+0000 7f70aa800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f70a406dbb0 con 0x7f70a414e530 2024-04-20T10:22:46.901 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.901+0000 7f70aa800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f70a406e890 con 0x7f70a414e910 2024-04-20T10:22:46.901 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.901+0000 7f70aa800640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f70a406dd30 con 0x7f70a41f94b0 2024-04-20T10:22:46.901 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.901+0000 7f70a3400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70a414e530 0x7f70a41fd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:46.902 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.901+0000 7f70a3400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70a414e530 0x7f70a41fd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:47936/0 (socket says 172.21.15.121:47936) 2024-04-20T10:22:46.902 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.901+0000 7f70a3400640 1 -- 172.21.15.121:0/2629973993 learned_addr learned my addr 172.21.15.121:0/2629973993 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:46.902 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.901+0000 7f70a3e00640 1 --2- 172.21.15.121:0/2629973993 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70a41f94b0 0x7f70a41651b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:46.902 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.901+0000 7f70a2a00640 1 --2- 172.21.15.121:0/2629973993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70a414e910 0x7f70a41f8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:46.902 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.901+0000 7f70a3400640 1 -- 172.21.15.121:0/2629973993 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70a41f94b0 msgr2=0x7f70a41651b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:46.902 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.901+0000 7f70a3400640 1 --2- 172.21.15.121:0/2629973993 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70a41f94b0 0x7f70a41651b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:46.902 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.901+0000 7f70a3400640 1 -- 172.21.15.121:0/2629973993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70a414e910 msgr2=0x7f70a41f8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:46.902 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.901+0000 7f70a3400640 1 --2- 172.21.15.121:0/2629973993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70a414e910 0x7f70a41f8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:46.902 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.902+0000 7f70a3400640 1 -- 172.21.15.121:0/2629973993 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70a41f9b00 con 0x7f70a414e530 2024-04-20T10:22:46.902 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.902+0000 7f70a3400640 1 --2- 172.21.15.121:0/2629973993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70a414e530 0x7f70a41fd080 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f709402bf00 tx=0x7f7094004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:46.903 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.902+0000 7f709b400640 1 -- 172.21.15.121:0/2629973993 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709400f000 con 0x7f70a414e530 2024-04-20T10:22:46.903 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.902+0000 7f709b400640 1 -- 172.21.15.121:0/2629973993 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f709403a440 con 0x7f70a414e530 2024-04-20T10:22:46.903 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.902+0000 7f709b400640 1 -- 172.21.15.121:0/2629973993 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709403a810 con 0x7f70a414e530 2024-04-20T10:22:46.903 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.902+0000 7f70aa800640 1 -- 172.21.15.121:0/2629973993 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70a41f9de0 con 0x7f70a414e530 2024-04-20T10:22:46.903 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.902+0000 7f70aa800640 1 -- 172.21.15.121:0/2629973993 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70a41fc880 con 0x7f70a414e530 2024-04-20T10:22:46.903 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.903+0000 7f70aa800640 1 -- 172.21.15.121:0/2629973993 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70a4068980 con 0x7f70a414e530 2024-04-20T10:22:46.904 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.904+0000 7f709b400640 1 -- 172.21.15.121:0/2629973993 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f709403a9b0 con 0x7f70a414e530 2024-04-20T10:22:46.904 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.905+0000 7f709b400640 1 --2- 172.21.15.121:0/2629973993 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f708006c8f0 0x7f708006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:46.905 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.905+0000 7f709b400640 1 -- 172.21.15.121:0/2629973993 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f70940adbd0 con 0x7f70a414e530 2024-04-20T10:22:46.905 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.905+0000 7f70a2a00640 1 --2- 172.21.15.121:0/2629973993 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f708006c8f0 0x7f708006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:46.906 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.906+0000 7f70a2a00640 1 --2- 172.21.15.121:0/2629973993 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f708006c8f0 0x7f708006edb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f70a41fd680 tx=0x7f708c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:46.909 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:46.909+0000 7f709b400640 1 -- 172.21.15.121:0/2629973993 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f709407c1b0 con 0x7f70a414e530 2024-04-20T10:22:47.164 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.164+0000 7f70aa800640 1 -- 172.21.15.121:0/2629973993 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f70a4072bf0 con 0x7f70a414e530 2024-04-20T10:22:47.168 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.169+0000 7f709b400640 1 -- 172.21.15.121:0/2629973993 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) ==== 90+0+0 (secure 0 0 0) 0x7f7094043e30 con 0x7f70a414e530 2024-04-20T10:22:47.172 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.172+0000 7f70aa800640 1 -- 172.21.15.121:0/2629973993 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f708006c8f0 msgr2=0x7f708006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:47.172 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.172+0000 7f70aa800640 1 --2- 172.21.15.121:0/2629973993 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f708006c8f0 0x7f708006edb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f70a41fd680 tx=0x7f708c009040 comp rx=0 tx=0).stop 2024-04-20T10:22:47.173 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.172+0000 7f70aa800640 1 -- 172.21.15.121:0/2629973993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70a414e530 msgr2=0x7f70a41fd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:47.173 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.172+0000 7f70aa800640 1 --2- 172.21.15.121:0/2629973993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70a414e530 0x7f70a41fd080 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f709402bf00 tx=0x7f7094004730 comp rx=0 tx=0).stop 2024-04-20T10:22:47.173 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.173+0000 7f70aa800640 1 -- 172.21.15.121:0/2629973993 shutdown_connections 2024-04-20T10:22:47.173 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.173+0000 7f70aa800640 1 --2- 172.21.15.121:0/2629973993 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f70a41f94b0 0x7f70a41651b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:47.173 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.173+0000 7f70aa800640 1 --2- 172.21.15.121:0/2629973993 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f708006c8f0 0x7f708006edb0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:47.173 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.173+0000 7f70aa800640 1 --2- 172.21.15.121:0/2629973993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70a414e910 0x7f70a41f8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:47.173 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.173+0000 7f70aa800640 1 --2- 172.21.15.121:0/2629973993 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f70a414e530 0x7f70a41fd080 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:47.173 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.173+0000 7f70aa800640 1 -- 172.21.15.121:0/2629973993 >> 172.21.15.121:0/2629973993 conn(0x7f70a40b3350 msgr2=0x7f70a40b13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:47.173 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.173+0000 7f70aa800640 1 -- 172.21.15.121:0/2629973993 shutdown_connections 2024-04-20T10:22:47.173 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.173+0000 7f70aa800640 1 -- 172.21.15.121:0/2629973993 wait complete. 2024-04-20T10:22:47.192 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:22:47.386 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.386+0000 7f3b40000640 1 Processor -- start 2024-04-20T10:22:47.386 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.386+0000 7f3b40000640 1 -- start start 2024-04-20T10:22:47.387 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.388+0000 7f3b40000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b38156c20 0x7f3b3806b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:47.388 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.388+0000 7f3b40000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3b3815ca40 0x7f3b380ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:47.388 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.388+0000 7f3b40000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3b380ad8e0 0x7f3b380d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:47.388 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.388+0000 7f3b40000640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f3b38141c60 con 0x7f3b380ad8e0 2024-04-20T10:22:47.388 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.388+0000 7f3b40000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f3b3815cf90 con 0x7f3b3815ca40 2024-04-20T10:22:47.388 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.388+0000 7f3b40000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f3b3815d270 con 0x7f3b38156c20 2024-04-20T10:22:47.389 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.389+0000 7f3b3e000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3b380ad8e0 0x7f3b380d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:47.389 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.389+0000 7f3b3e000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3b380ad8e0 0x7f3b380d0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:47948/0 (socket says 172.21.15.121:47948) 2024-04-20T10:22:47.389 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.389+0000 7f3b3e000640 1 -- 172.21.15.121:0/1158781647 learned_addr learned my addr 172.21.15.121:0/1158781647 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:47.389 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.389+0000 7f3b3d600640 1 --2- 172.21.15.121:0/1158781647 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b38156c20 0x7f3b3806b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:47.389 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.389+0000 7f3b3cc00640 1 --2- 172.21.15.121:0/1158781647 >> v2:172.21.15.123:3300/0 conn(0x7f3b3815ca40 0x7f3b380ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:47.389 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.389+0000 7f3b3e000640 1 -- 172.21.15.121:0/1158781647 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b38156c20 msgr2=0x7f3b3806b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:47.389 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.389+0000 7f3b3e000640 1 --2- 172.21.15.121:0/1158781647 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b38156c20 0x7f3b3806b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:47.390 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.389+0000 7f3b3e000640 1 -- 172.21.15.121:0/1158781647 >> v2:172.21.15.123:3300/0 conn(0x7f3b3815ca40 msgr2=0x7f3b380ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:47.390 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.389+0000 7f3b3e000640 1 --2- 172.21.15.121:0/1158781647 >> v2:172.21.15.123:3300/0 conn(0x7f3b3815ca40 0x7f3b380ad3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:47.390 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.389+0000 7f3b3e000640 1 -- 172.21.15.121:0/1158781647 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b3815d550 con 0x7f3b380ad8e0 2024-04-20T10:22:47.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.390+0000 7f3b3e000640 1 --2- 172.21.15.121:0/1158781647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b380ad8e0 0x7f3b380d0360 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f3b3806fe70 tx=0x7f3b3002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ba1c27cc34a9a51 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:47.393 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.390+0000 7f3b37e00640 1 -- 172.21.15.121:0/1158781647 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3002ca70 con 0x7f3b380ad8e0 2024-04-20T10:22:47.393 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.390+0000 7f3b37e00640 1 -- 172.21.15.121:0/1158781647 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b3002cc10 con 0x7f3b380ad8e0 2024-04-20T10:22:47.393 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.390+0000 7f3b37e00640 1 -- 172.21.15.121:0/1158781647 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b300343f0 con 0x7f3b380ad8e0 2024-04-20T10:22:47.393 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.391+0000 7f3b40000640 1 -- 172.21.15.121:0/1158781647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b380ad8e0 msgr2=0x7f3b380d0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:47.393 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.391+0000 7f3b40000640 1 --2- 172.21.15.121:0/1158781647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b380ad8e0 0x7f3b380d0360 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f3b3806fe70 tx=0x7f3b3002bed0 comp rx=0 tx=0).stop 2024-04-20T10:22:47.393 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.391+0000 7f3b40000640 1 -- 172.21.15.121:0/1158781647 shutdown_connections 2024-04-20T10:22:47.394 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.391+0000 7f3b40000640 1 --2- 172.21.15.121:0/1158781647 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b380ad8e0 0x7f3b380d0360 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:47.394 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.391+0000 7f3b40000640 1 --2- 172.21.15.121:0/1158781647 >> v2:172.21.15.123:3300/0 conn(0x7f3b3815ca40 0x7f3b380ad3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:47.394 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.391+0000 7f3b40000640 1 --2- 172.21.15.121:0/1158781647 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b38156c20 0x7f3b3806b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:47.394 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.391+0000 7f3b40000640 1 -- 172.21.15.121:0/1158781647 >> 172.21.15.121:0/1158781647 conn(0x7f3b380b3350 msgr2=0x7f3b380b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:47.394 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.391+0000 7f3b40000640 1 -- 172.21.15.121:0/1158781647 shutdown_connections 2024-04-20T10:22:47.394 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.391+0000 7f3b40000640 1 -- 172.21.15.121:0/1158781647 wait complete. 2024-04-20T10:22:47.396 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.393+0000 7f3b40000640 1 Processor -- start 2024-04-20T10:22:47.396 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.393+0000 7f3b40000640 1 -- start start 2024-04-20T10:22:47.396 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.394+0000 7f3b40000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b380ad8e0 0x7f3b380cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:47.396 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.394+0000 7f3b40000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b38156c20 0x7f3b380cf200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:47.396 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.394+0000 7f3b40000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b3815ca40 0x7f3b3809dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:47.397 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.394+0000 7f3b40000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3806bfc0 con 0x7f3b3815ca40 2024-04-20T10:22:47.397 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.394+0000 7f3b40000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3806be40 con 0x7f3b38156c20 2024-04-20T10:22:47.397 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.394+0000 7f3b40000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f3b380d08a0 con 0x7f3b380ad8e0 2024-04-20T10:22:47.397 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.394+0000 7f3b3e000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b3815ca40 0x7f3b3809dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:47.397 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.395+0000 7f3b3e000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b3815ca40 0x7f3b3809dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:47964/0 (socket says 172.21.15.121:47964) 2024-04-20T10:22:47.397 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.395+0000 7f3b3e000640 1 -- 172.21.15.121:0/2483890725 learned_addr learned my addr 172.21.15.121:0/2483890725 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:47.397 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.395+0000 7f3b3d600640 1 --2- 172.21.15.121:0/2483890725 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b380ad8e0 0x7f3b380cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:47.397 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.395+0000 7f3b3cc00640 1 --2- 172.21.15.121:0/2483890725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b38156c20 0x7f3b380cf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:47.397 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.395+0000 7f3b3e000640 1 -- 172.21.15.121:0/2483890725 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b380ad8e0 msgr2=0x7f3b380cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:47.397 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.395+0000 7f3b3e000640 1 --2- 172.21.15.121:0/2483890725 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b380ad8e0 0x7f3b380cdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:47.397 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.395+0000 7f3b3e000640 1 -- 172.21.15.121:0/2483890725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b38156c20 msgr2=0x7f3b380cf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:47.397 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.395+0000 7f3b3e000640 1 --2- 172.21.15.121:0/2483890725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b38156c20 0x7f3b380cf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:47.397 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.395+0000 7f3b3e000640 1 -- 172.21.15.121:0/2483890725 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b380ce160 con 0x7f3b3815ca40 2024-04-20T10:22:47.397 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.395+0000 7f3b3cc00640 1 --2- 172.21.15.121:0/2483890725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b38156c20 0x7f3b380cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:47.398 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.395+0000 7f3b3e000640 1 --2- 172.21.15.121:0/2483890725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b3815ca40 0x7f3b3809dc10 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3b30036040 tx=0x7f3b30004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:47.398 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.396+0000 7f3b35600640 1 -- 172.21.15.121:0/2483890725 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b30043020 con 0x7f3b3815ca40 2024-04-20T10:22:47.398 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.396+0000 7f3b35600640 1 -- 172.21.15.121:0/2483890725 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b3003e450 con 0x7f3b3815ca40 2024-04-20T10:22:47.398 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.396+0000 7f3b35600640 1 -- 172.21.15.121:0/2483890725 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b30009510 con 0x7f3b3815ca40 2024-04-20T10:22:47.398 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.396+0000 7f3b40000640 1 -- 172.21.15.121:0/2483890725 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b380ce440 con 0x7f3b3815ca40 2024-04-20T10:22:47.398 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.396+0000 7f3b40000640 1 -- 172.21.15.121:0/2483890725 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b380cea00 con 0x7f3b3815ca40 2024-04-20T10:22:47.398 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.397+0000 7f3b40000640 1 -- 172.21.15.121:0/2483890725 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b38074010 con 0x7f3b3815ca40 2024-04-20T10:22:47.399 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.398+0000 7f3b35600640 1 -- 172.21.15.121:0/2483890725 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3b30046020 con 0x7f3b3815ca40 2024-04-20T10:22:47.399 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.399+0000 7f3b35600640 1 --2- 172.21.15.121:0/2483890725 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f3b1c06c8f0 0x7f3b1c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:47.399 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.399+0000 7f3b35600640 1 -- 172.21.15.121:0/2483890725 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f3b300ae2a0 con 0x7f3b3815ca40 2024-04-20T10:22:47.399 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.399+0000 7f3b3d600640 1 --2- 172.21.15.121:0/2483890725 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f3b1c06c8f0 0x7f3b1c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:47.399 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.400+0000 7f3b3d600640 1 --2- 172.21.15.121:0/2483890725 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f3b1c06c8f0 0x7f3b1c06edb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c006fd0 tx=0x7f3b2c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:47.403 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.403+0000 7f3b35600640 1 -- 172.21.15.121:0/2483890725 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3b3004d030 con 0x7f3b3815ca40 2024-04-20T10:22:47.590 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.590+0000 7f3b40000640 1 -- 172.21.15.121:0/2483890725 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3b380b2bc0 con 0x7f3b3815ca40 2024-04-20T10:22:47.592 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.592+0000 7f3b35600640 1 -- 172.21.15.121:0/2483890725 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+13200 (secure 0 0 0) 0x7f3b3007c760 con 0x7f3b3815ca40 2024-04-20T10:22:47.592 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:22:47.592 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":35,"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","created":"2024-04-20T10:20:38.076076+0000","modified":"2024-04-20T10:22:43.643134+0000","last_up_change":"2024-04-20T10:20:50.162700+0000","last_in_change":"2024-04-20T10:20:41.510382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:20:51.141365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"629a7b8c-a40a-4a2a-a3cc-d9f357774310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6809","nonce":1044588835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6811","nonce":1044588835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6815","nonce":1044588835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6813","nonce":1044588835}]},"public_addr":"172.21.15.121:6809/1044588835","cluster_addr":"172.21.15.121:6811/1044588835","heartbeat_back_addr":"172.21.15.121:6815/1044588835","heartbeat_front_addr":"172.21.15.121:6813/1044588835","state":["exists","up"]},{"osd":1,"uuid":"f2570399-aa50-4ff3-bf5d-3d09207290db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6801","nonce":3505199624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6803","nonce":3505199624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6807","nonce":3505199624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6805","nonce":3505199624}]},"public_addr":"172.21.15.121:6801/3505199624","cluster_addr":"172.21.15.121:6803/3505199624","heartbeat_back_addr":"172.21.15.121:6807/3505199624","heartbeat_front_addr":"172.21.15.121:6805/3505199624","state":["exists","up"]},{"osd":2,"uuid":"50893807-dc30-49f4-b41f-739366626658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6817","nonce":3656724207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6819","nonce":3656724207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6827","nonce":3656724207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6822","nonce":3656724207}]},"public_addr":"172.21.15.121:6817/3656724207","cluster_addr":"172.21.15.121:6819/3656724207","heartbeat_back_addr":"172.21.15.121:6827/3656724207","heartbeat_front_addr":"172.21.15.121:6822/3656724207","state":["exists","up"]},{"osd":3,"uuid":"dadb01bc-5491-4ffc-8824-d48a057240b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6823","nonce":2232342832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6826","nonce":2232342832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6831","nonce":2232342832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6829","nonce":2232342832}]},"public_addr":"172.21.15.121:6823/2232342832","cluster_addr":"172.21.15.121:6826/2232342832","heartbeat_back_addr":"172.21.15.121:6831/2232342832","heartbeat_front_addr":"172.21.15.121:6829/2232342832","state":["exists","up"]},{"osd":4,"uuid":"b3f3b4a1-b375-4526-b69e-344ab24c2da1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6801","nonce":99353111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6803","nonce":99353111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6807","nonce":99353111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6805","nonce":99353111}]},"public_addr":"172.21.15.123:6801/99353111","cluster_addr":"172.21.15.123:6803/99353111","heartbeat_back_addr":"172.21.15.123:6807/99353111","heartbeat_front_addr":"172.21.15.123:6805/99353111","state":["exists","up"]},{"osd":5,"uuid":"2ee24a08-1624-4961-9642-d7b510282484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6818","nonce":2504374688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6822","nonce":2504374688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6830","nonce":2504374688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6826","nonce":2504374688}]},"public_addr":"172.21.15.123:6818/2504374688","cluster_addr":"172.21.15.123:6822/2504374688","heartbeat_back_addr":"172.21.15.123:6830/2504374688","heartbeat_front_addr":"172.21.15.123:6826/2504374688","state":["exists","up"]},{"osd":6,"uuid":"186118cc-6128-49de-acca-55260945c1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6819","nonce":4139299599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6823","nonce":4139299599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6829","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6831","nonce":4139299599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6827","nonce":4139299599}]},"public_addr":"172.21.15.123:6819/4139299599","cluster_addr":"172.21.15.123:6823/4139299599","heartbeat_back_addr":"172.21.15.123:6831/4139299599","heartbeat_front_addr":"172.21.15.123:6827/4139299599","state":["exists","up"]},{"osd":7,"uuid":"a00aac6e-4bd9-415a-a93d-ae85210865ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6809","nonce":4043981069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6811","nonce":4043981069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6815","nonce":4043981069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6813","nonce":4043981069}]},"public_addr":"172.21.15.123:6809/4043981069","cluster_addr":"172.21.15.123:6811/4043981069","heartbeat_back_addr":"172.21.15.123:6815/4043981069","heartbeat_front_addr":"172.21.15.123:6813/4043981069","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.904540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.958585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.616367+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.885244+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.103127+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.167248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.884348+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.862356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:22:47.594 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.594+0000 7f3b40000640 1 -- 172.21.15.121:0/2483890725 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f3b1c06c8f0 msgr2=0x7f3b1c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:47.595 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.594+0000 7f3b40000640 1 --2- 172.21.15.121:0/2483890725 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f3b1c06c8f0 0x7f3b1c06edb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c006fd0 tx=0x7f3b2c008040 comp rx=0 tx=0).stop 2024-04-20T10:22:47.595 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.595+0000 7f3b40000640 1 -- 172.21.15.121:0/2483890725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b3815ca40 msgr2=0x7f3b3809dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:47.595 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.595+0000 7f3b40000640 1 --2- 172.21.15.121:0/2483890725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b3815ca40 0x7f3b3809dc10 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3b30036040 tx=0x7f3b30004a40 comp rx=0 tx=0).stop 2024-04-20T10:22:47.595 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.595+0000 7f3b40000640 1 -- 172.21.15.121:0/2483890725 shutdown_connections 2024-04-20T10:22:47.595 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.595+0000 7f3b40000640 1 --2- 172.21.15.121:0/2483890725 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3b3815ca40 0x7f3b3809dc10 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:47.595 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.595+0000 7f3b40000640 1 --2- 172.21.15.121:0/2483890725 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f3b1c06c8f0 0x7f3b1c06edb0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:47.595 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.595+0000 7f3b40000640 1 --2- 172.21.15.121:0/2483890725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b38156c20 0x7f3b380cf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:47.595 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.595+0000 7f3b40000640 1 --2- 172.21.15.121:0/2483890725 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3b380ad8e0 0x7f3b380cdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:47.595 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.595+0000 7f3b40000640 1 -- 172.21.15.121:0/2483890725 >> 172.21.15.121:0/2483890725 conn(0x7f3b380b3350 msgr2=0x7f3b38155430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:47.595 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.595+0000 7f3b40000640 1 -- 172.21.15.121:0/2483890725 shutdown_connections 2024-04-20T10:22:47.595 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.595+0000 7f3b40000640 1 -- 172.21.15.121:0/2483890725 wait complete. 2024-04-20T10:22:47.612 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:20:51.141365+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T10:22:47.612 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:22:47.810 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.810+0000 7f1c13200640 1 Processor -- start 2024-04-20T10:22:47.810 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.811+0000 7f1c13200640 1 -- start start 2024-04-20T10:22:47.812 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.813+0000 7f1c13200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1c1415c940 0x7f1c14160940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:47.813 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.813+0000 7f1c13200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1c1406d100 0x7f1c1406d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:47.813 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.813+0000 7f1c13200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c1406da40 con 0x7f1c140afc00 2024-04-20T10:22:47.813 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.813+0000 7f1c13200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f1c1415ce40 con 0x7f1c1415c940 2024-04-20T10:22:47.813 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.813+0000 7f1c13200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f1c1415d120 con 0x7f1c1406d100 2024-04-20T10:22:47.813 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.813+0000 7f1c11400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1c1415c940 0x7f1c14160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:47.813 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.813+0000 7f1c11400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f1c1415c940 0x7f1c14160940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:47970/0 (socket says 172.21.15.121:47970) 2024-04-20T10:22:47.813 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.813+0000 7f1c11400640 1 -- 172.21.15.121:0/2055988288 learned_addr learned my addr 172.21.15.121:0/2055988288 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:47.813 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.813+0000 7f1c12800640 1 --2- 172.21.15.121:0/2055988288 >> v2:172.21.15.123:3300/0 conn(0x7f1c1406d100 0x7f1c1406d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:47.814 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.814+0000 7f1c11400640 1 -- 172.21.15.121:0/2055988288 >> v2:172.21.15.123:3300/0 conn(0x7f1c1406d100 msgr2=0x7f1c1406d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:47.814 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.814+0000 7f1c11400640 1 --2- 172.21.15.121:0/2055988288 >> v2:172.21.15.123:3300/0 conn(0x7f1c1406d100 0x7f1c1406d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:47.814 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.814+0000 7f1c11400640 1 -- 172.21.15.121:0/2055988288 >> v1:172.21.15.123:6789/0 conn(0x7f1c140afc00 legacy=0x7f1c140ce6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:47.814 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.814+0000 7f1c11400640 1 -- 172.21.15.121:0/2055988288 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c1415d400 con 0x7f1c1415c940 2024-04-20T10:22:47.814 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.814+0000 7f1c12800640 1 --2- 172.21.15.121:0/2055988288 >> v2:172.21.15.123:3300/0 conn(0x7f1c1406d100 0x7f1c1406d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:47.814 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.815+0000 7f1c11400640 1 --2- 172.21.15.121:0/2055988288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c1415c940 0x7f1c14160940 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f1c140d0650 tx=0x7f1bfc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9b4451ecd96f67 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:47.815 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.815+0000 7f1c10a00640 1 -- 172.21.15.121:0/2055988288 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc035020 con 0x7f1c1415c940 2024-04-20T10:22:47.815 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.815+0000 7f1c10a00640 1 -- 172.21.15.121:0/2055988288 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bfc031870 con 0x7f1c1415c940 2024-04-20T10:22:47.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.815+0000 7f1c10a00640 1 -- 172.21.15.121:0/2055988288 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc031c40 con 0x7f1c1415c940 2024-04-20T10:22:47.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.816+0000 7f1c13200640 1 -- 172.21.15.121:0/2055988288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c1415c940 msgr2=0x7f1c14160940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:47.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.816+0000 7f1c13200640 1 --2- 172.21.15.121:0/2055988288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c1415c940 0x7f1c14160940 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f1c140d0650 tx=0x7f1bfc02b910 comp rx=0 tx=0).stop 2024-04-20T10:22:47.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.816+0000 7f1c13200640 1 -- 172.21.15.121:0/2055988288 shutdown_connections 2024-04-20T10:22:47.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.816+0000 7f1c13200640 1 --2- 172.21.15.121:0/2055988288 >> v2:172.21.15.123:3300/0 conn(0x7f1c1406d100 0x7f1c1406d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:47.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.816+0000 7f1c13200640 1 --2- 172.21.15.121:0/2055988288 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c1415c940 0x7f1c14160940 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:47.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.816+0000 7f1c13200640 1 -- 172.21.15.121:0/2055988288 >> 172.21.15.121:0/2055988288 conn(0x7f1c140cd220 msgr2=0x7f1c140cd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:47.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.816+0000 7f1c13200640 1 -- 172.21.15.121:0/2055988288 shutdown_connections 2024-04-20T10:22:47.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.816+0000 7f1c13200640 1 -- 172.21.15.121:0/2055988288 wait complete. 2024-04-20T10:22:47.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.818+0000 7f1c13200640 1 Processor -- start 2024-04-20T10:22:47.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.818+0000 7f1c13200640 1 -- start start 2024-04-20T10:22:47.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.820+0000 7f1c13200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1c1406d100 0x7f1c14154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:47.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.820+0000 7f1c13200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c140afc00 0x7f1c1415a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:47.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.820+0000 7f1c13200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c1415c940 0x7f1c141f5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:47.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.820+0000 7f1c13200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1406faa0 con 0x7f1c140afc00 2024-04-20T10:22:47.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.820+0000 7f1c13200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1c1406e360 con 0x7f1c1415c940 2024-04-20T10:22:47.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.820+0000 7f1c13200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f1c14070010 con 0x7f1c1406d100 2024-04-20T10:22:47.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.820+0000 7f1c11400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c140afc00 0x7f1c1415a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:47.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.820+0000 7f1c11400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c140afc00 0x7f1c1415a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:47976/0 (socket says 172.21.15.121:47976) 2024-04-20T10:22:47.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.820+0000 7f1c11400640 1 -- 172.21.15.121:0/1061324030 learned_addr learned my addr 172.21.15.121:0/1061324030 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:47.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.820+0000 7f1c11400640 1 -- 172.21.15.121:0/1061324030 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1c1406d100 msgr2=0x7f1c14154ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:47.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.820+0000 7f1c11400640 1 --2- 172.21.15.121:0/1061324030 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1c1406d100 0x7f1c14154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:47.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.820+0000 7f1c11400640 1 -- 172.21.15.121:0/1061324030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c1415c940 msgr2=0x7f1c141f5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:47.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.820+0000 7f1c11400640 1 --2- 172.21.15.121:0/1061324030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c1415c940 0x7f1c141f5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:47.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.820+0000 7f1c11400640 1 -- 172.21.15.121:0/1061324030 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c1415b190 con 0x7f1c140afc00 2024-04-20T10:22:47.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.821+0000 7f1c11400640 1 --2- 172.21.15.121:0/1061324030 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c140afc00 0x7f1c1415a940 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc000c00 tx=0x7f1bfc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:47.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.821+0000 7f1c0a000640 1 -- 172.21.15.121:0/1061324030 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc040070 con 0x7f1c140afc00 2024-04-20T10:22:47.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.821+0000 7f1c0a000640 1 -- 172.21.15.121:0/1061324030 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bfc03a420 con 0x7f1c140afc00 2024-04-20T10:22:47.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.821+0000 7f1c13200640 1 -- 172.21.15.121:0/1061324030 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c1415b470 con 0x7f1c140afc00 2024-04-20T10:22:47.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.821+0000 7f1c0a000640 1 -- 172.21.15.121:0/1061324030 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc035040 con 0x7f1c140afc00 2024-04-20T10:22:47.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.821+0000 7f1c13200640 1 -- 172.21.15.121:0/1061324030 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c1415ae80 con 0x7f1c140afc00 2024-04-20T10:22:47.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.822+0000 7f1c13200640 1 -- 172.21.15.121:0/1061324030 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c1415b860 con 0x7f1c140afc00 2024-04-20T10:22:47.823 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.823+0000 7f1c0a000640 1 -- 172.21.15.121:0/1061324030 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1bfc04b060 con 0x7f1c140afc00 2024-04-20T10:22:47.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.824+0000 7f1c0a000640 1 --2- 172.21.15.121:0/1061324030 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f1bf006c8f0 0x7f1bf006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:47.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.824+0000 7f1c0a000640 1 -- 172.21.15.121:0/1061324030 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f1bfc0ad8c0 con 0x7f1c140afc00 2024-04-20T10:22:47.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.825+0000 7f1c11e00640 1 --2- 172.21.15.121:0/1061324030 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f1bf006c8f0 0x7f1bf006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:47.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.825+0000 7f1c11e00640 1 --2- 172.21.15.121:0/1061324030 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f1bf006c8f0 0x7f1bf006edb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1c04001560 tx=0x7f1c04007990 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:47.828 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:47.829+0000 7f1c0a000640 1 -- 172.21.15.121:0/1061324030 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1bfc07bf40 con 0x7f1c140afc00 2024-04-20T10:22:48.025 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.025+0000 7f1c13200640 1 -- 172.21.15.121:0/1061324030 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1c1415be10 con 0x7f1c140afc00 2024-04-20T10:22:48.025 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.026+0000 7f1c0a000640 1 -- 172.21.15.121:0/1061324030 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) ==== 93+0+10 (secure 0 0 0) 0x7f1c1415be10 con 0x7f1c140afc00 2024-04-20T10:22:48.025 INFO:teuthology.orchestra.run.smithi121.stdout:pg_num: 1 2024-04-20T10:22:48.028 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.028+0000 7f1c13200640 1 -- 172.21.15.121:0/1061324030 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f1bf006c8f0 msgr2=0x7f1bf006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:48.028 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.028+0000 7f1c13200640 1 --2- 172.21.15.121:0/1061324030 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f1bf006c8f0 0x7f1bf006edb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1c04001560 tx=0x7f1c04007990 comp rx=0 tx=0).stop 2024-04-20T10:22:48.028 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.028+0000 7f1c13200640 1 -- 172.21.15.121:0/1061324030 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c140afc00 msgr2=0x7f1c1415a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:48.028 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.028+0000 7f1c13200640 1 --2- 172.21.15.121:0/1061324030 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c140afc00 0x7f1c1415a940 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc000c00 tx=0x7f1bfc0046c0 comp rx=0 tx=0).stop 2024-04-20T10:22:48.028 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.028+0000 7f1c13200640 1 -- 172.21.15.121:0/1061324030 shutdown_connections 2024-04-20T10:22:48.028 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.028+0000 7f1c13200640 1 --2- 172.21.15.121:0/1061324030 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f1bf006c8f0 0x7f1bf006edb0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:48.028 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.028+0000 7f1c13200640 1 --2- 172.21.15.121:0/1061324030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c1415c940 0x7f1c141f5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:48.028 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.028+0000 7f1c13200640 1 --2- 172.21.15.121:0/1061324030 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1c140afc00 0x7f1c1415a940 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:48.028 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.028+0000 7f1c13200640 1 --2- 172.21.15.121:0/1061324030 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1c1406d100 0x7f1c14154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:48.029 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.028+0000 7f1c13200640 1 -- 172.21.15.121:0/1061324030 >> 172.21.15.121:0/1061324030 conn(0x7f1c140cd220 msgr2=0x7f1c140b1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:48.029 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.028+0000 7f1c13200640 1 -- 172.21.15.121:0/1061324030 shutdown_connections 2024-04-20T10:22:48.029 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.029+0000 7f1c13200640 1 -- 172.21.15.121:0/1061324030 wait complete. 2024-04-20T10:22:48.046 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-20T10:22:48.239 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.240+0000 7f5b83e00640 1 Processor -- start 2024-04-20T10:22:48.240 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.240+0000 7f5b83e00640 1 -- start start 2024-04-20T10:22:48.241 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.242+0000 7f5b83e00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5b8415b400 0x7f5b84072d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:48.242 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.242+0000 7f5b83e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5b8415eec0 0x7f5b8415f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:48.242 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.242+0000 7f5b83e00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b8415f870 0x7f5b8415b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:48.242 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.242+0000 7f5b83e00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f5b8406d280 con 0x7f5b8415eec0 2024-04-20T10:22:48.242 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.242+0000 7f5b83e00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f5b8415fcd0 con 0x7f5b8415b400 2024-04-20T10:22:48.242 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.242+0000 7f5b83e00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f5b8406ccc0 con 0x7f5b8415f870 2024-04-20T10:22:48.242 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.242+0000 7f5b7a000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5b8415eec0 0x7f5b8415f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:48.242 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.242+0000 7f5b7a000640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f5b8415eec0 0x7f5b8415f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:47980/0 (socket says 172.21.15.121:47980) 2024-04-20T10:22:48.242 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.242+0000 7f5b7a000640 1 -- 172.21.15.121:0/627188035 learned_addr learned my addr 172.21.15.121:0/627188035 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:48.243 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.243+0000 7f5b7a000640 1 -- 172.21.15.121:0/627188035 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b8415f870 msgr2=0x7f5b8415b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:48.243 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.243+0000 7f5b83400640 1 --2- 172.21.15.121:0/627188035 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b8415f870 0x7f5b8415b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:48.243 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.243+0000 7f5b82a00640 1 --2- 172.21.15.121:0/627188035 >> v2:172.21.15.123:3300/0 conn(0x7f5b8415b400 0x7f5b84072d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:48.243 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.243+0000 7f5b7a000640 1 --2- 172.21.15.121:0/627188035 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b8415f870 0x7f5b8415b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:48.243 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.243+0000 7f5b7a000640 1 -- 172.21.15.121:0/627188035 >> v2:172.21.15.123:3300/0 conn(0x7f5b8415b400 msgr2=0x7f5b84072d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:48.243 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.243+0000 7f5b7a000640 1 --2- 172.21.15.121:0/627188035 >> v2:172.21.15.123:3300/0 conn(0x7f5b8415b400 0x7f5b84072d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:48.244 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.243+0000 7f5b7a000640 1 -- 172.21.15.121:0/627188035 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b8406cfa0 con 0x7f5b8415eec0 2024-04-20T10:22:48.244 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.244+0000 7f5b7a000640 1 --2- 172.21.15.121:0/627188035 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b8415eec0 0x7f5b8415f330 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5b84074060 tx=0x7f5b6c02e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe8317af27077e6f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:48.245 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.244+0000 7f5b82000640 1 -- 172.21.15.121:0/627188035 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c03b070 con 0x7f5b8415eec0 2024-04-20T10:22:48.245 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.244+0000 7f5b82000640 1 -- 172.21.15.121:0/627188035 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b6c036440 con 0x7f5b8415eec0 2024-04-20T10:22:48.245 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.244+0000 7f5b82000640 1 -- 172.21.15.121:0/627188035 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c036810 con 0x7f5b8415eec0 2024-04-20T10:22:48.245 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.245+0000 7f5b83e00640 1 -- 172.21.15.121:0/627188035 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b8415eec0 msgr2=0x7f5b8415f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:48.245 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.245+0000 7f5b83e00640 1 --2- 172.21.15.121:0/627188035 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b8415eec0 0x7f5b8415f330 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5b84074060 tx=0x7f5b6c02e0e0 comp rx=0 tx=0).stop 2024-04-20T10:22:48.245 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.245+0000 7f5b83e00640 1 -- 172.21.15.121:0/627188035 shutdown_connections 2024-04-20T10:22:48.245 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.245+0000 7f5b83e00640 1 --2- 172.21.15.121:0/627188035 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b8415f870 0x7f5b8415b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:48.245 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.245+0000 7f5b83e00640 1 --2- 172.21.15.121:0/627188035 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b8415eec0 0x7f5b8415f330 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:48.245 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.245+0000 7f5b83e00640 1 --2- 172.21.15.121:0/627188035 >> v2:172.21.15.123:3300/0 conn(0x7f5b8415b400 0x7f5b84072d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:48.245 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.245+0000 7f5b83e00640 1 -- 172.21.15.121:0/627188035 >> 172.21.15.121:0/627188035 conn(0x7f5b840d0040 msgr2=0x7f5b840d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:48.245 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.245+0000 7f5b83e00640 1 -- 172.21.15.121:0/627188035 shutdown_connections 2024-04-20T10:22:48.246 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.245+0000 7f5b83e00640 1 -- 172.21.15.121:0/627188035 wait complete. 2024-04-20T10:22:48.246 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.247+0000 7f5b83e00640 1 Processor -- start 2024-04-20T10:22:48.246 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.247+0000 7f5b83e00640 1 -- start start 2024-04-20T10:22:48.247 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.248+0000 7f5b83e00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b8415b400 0x7f5b841f5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:48.248 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.248+0000 7f5b83e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b8415eec0 0x7f5b840b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:48.248 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.248+0000 7f5b83e00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b8415f870 0x7f5b840b42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:48.248 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.248+0000 7f5b83e00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f5b84073240 con 0x7f5b8415eec0 2024-04-20T10:22:48.248 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.248+0000 7f5b83e00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5b8406e720 con 0x7f5b8415b400 2024-04-20T10:22:48.248 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.248+0000 7f5b83e00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f5b8415bd20 con 0x7f5b8415f870 2024-04-20T10:22:48.248 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.248+0000 7f5b7a000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b8415eec0 0x7f5b840b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:48.248 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.248+0000 7f5b7a000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b8415eec0 0x7f5b840b3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:47996/0 (socket says 172.21.15.121:47996) 2024-04-20T10:22:48.248 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.248+0000 7f5b7a000640 1 -- 172.21.15.121:0/1041568588 learned_addr learned my addr 172.21.15.121:0/1041568588 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:48.248 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.248+0000 7f5b7a000640 1 -- 172.21.15.121:0/1041568588 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b8415f870 msgr2=0x7f5b840b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:48.248 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.248+0000 7f5b7a000640 1 --2- 172.21.15.121:0/1041568588 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b8415f870 0x7f5b840b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:48.248 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.248+0000 7f5b7a000640 1 -- 172.21.15.121:0/1041568588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b8415b400 msgr2=0x7f5b841f5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:48.248 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.248+0000 7f5b7a000640 1 --2- 172.21.15.121:0/1041568588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b8415b400 0x7f5b841f5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:48.249 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.248+0000 7f5b7a000640 1 -- 172.21.15.121:0/1041568588 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b84157100 con 0x7f5b8415eec0 2024-04-20T10:22:48.249 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.249+0000 7f5b7a000640 1 --2- 172.21.15.121:0/1041568588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b8415eec0 0x7f5b840b3d80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c02e120 tx=0x7f5b6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:48.250 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.249+0000 7f5b7b400640 1 -- 172.21.15.121:0/1041568588 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c03b030 con 0x7f5b8415eec0 2024-04-20T10:22:48.250 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.249+0000 7f5b7b400640 1 -- 172.21.15.121:0/1041568588 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b6c03d070 con 0x7f5b8415eec0 2024-04-20T10:22:48.250 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.249+0000 7f5b7b400640 1 -- 172.21.15.121:0/1041568588 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c034bc0 con 0x7f5b8415eec0 2024-04-20T10:22:48.250 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.249+0000 7f5b83e00640 1 -- 172.21.15.121:0/1041568588 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b840cc650 con 0x7f5b8415eec0 2024-04-20T10:22:48.250 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.249+0000 7f5b83e00640 1 -- 172.21.15.121:0/1041568588 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b840ccc10 con 0x7f5b8415eec0 2024-04-20T10:22:48.250 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.250+0000 7f5b83e00640 1 -- 172.21.15.121:0/1041568588 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b840d0a80 con 0x7f5b8415eec0 2024-04-20T10:22:48.250 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.251+0000 7f5b7b400640 1 -- 172.21.15.121:0/1041568588 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5b6c03f030 con 0x7f5b8415eec0 2024-04-20T10:22:48.252 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.252+0000 7f5b7b400640 1 --2- 172.21.15.121:0/1041568588 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f5b6006c8f0 0x7f5b6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:48.252 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.252+0000 7f5b7b400640 1 -- 172.21.15.121:0/1041568588 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f5b6c0ad810 con 0x7f5b8415eec0 2024-04-20T10:22:48.252 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.252+0000 7f5b82a00640 1 --2- 172.21.15.121:0/1041568588 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f5b6006c8f0 0x7f5b6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:48.252 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.253+0000 7f5b82a00640 1 --2- 172.21.15.121:0/1041568588 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f5b6006c8f0 0x7f5b6006edb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5b74007130 tx=0x7f5b74007310 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:48.256 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.257+0000 7f5b7b400640 1 -- 172.21.15.121:0/1041568588 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5b840d0a80 con 0x7f5b8415eec0 2024-04-20T10:22:48.453 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.453+0000 7f5b83e00640 1 -- 172.21.15.121:0/1041568588 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f5b841573e0 con 0x7f5b8415eec0 2024-04-20T10:22:48.453 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.454+0000 7f5b7b400640 1 -- 172.21.15.121:0/1041568588 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v35) ==== 84+0+65 (secure 0 0 0) 0x7f5b6c07bd60 con 0x7f5b8415eec0 2024-04-20T10:22:48.454 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:22:48.454 INFO:teuthology.orchestra.run.smithi121.stdout:[ 2024-04-20T10:22:48.454 INFO:teuthology.orchestra.run.smithi121.stdout: { 2024-04-20T10:22:48.454 INFO:teuthology.orchestra.run.smithi121.stdout: "poolnum": 1, 2024-04-20T10:22:48.454 INFO:teuthology.orchestra.run.smithi121.stdout: "poolname": ".mgr" 2024-04-20T10:22:48.454 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T10:22:48.454 INFO:teuthology.orchestra.run.smithi121.stdout:] 2024-04-20T10:22:48.456 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.456+0000 7f5b83e00640 1 -- 172.21.15.121:0/1041568588 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f5b6006c8f0 msgr2=0x7f5b6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:48.456 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.456+0000 7f5b83e00640 1 --2- 172.21.15.121:0/1041568588 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f5b6006c8f0 0x7f5b6006edb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5b74007130 tx=0x7f5b74007310 comp rx=0 tx=0).stop 2024-04-20T10:22:48.456 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.456+0000 7f5b83e00640 1 -- 172.21.15.121:0/1041568588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b8415eec0 msgr2=0x7f5b840b3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:48.457 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.456+0000 7f5b83e00640 1 --2- 172.21.15.121:0/1041568588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b8415eec0 0x7f5b840b3d80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c02e120 tx=0x7f5b6c0046c0 comp rx=0 tx=0).stop 2024-04-20T10:22:48.457 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.456+0000 7f5b83e00640 1 -- 172.21.15.121:0/1041568588 shutdown_connections 2024-04-20T10:22:48.457 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.456+0000 7f5b83e00640 1 --2- 172.21.15.121:0/1041568588 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f5b8415f870 0x7f5b840b42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:48.457 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.456+0000 7f5b83e00640 1 --2- 172.21.15.121:0/1041568588 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f5b8415eec0 0x7f5b840b3d80 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:48.457 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.456+0000 7f5b83e00640 1 --2- 172.21.15.121:0/1041568588 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f5b6006c8f0 0x7f5b6006edb0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:48.457 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.456+0000 7f5b83e00640 1 --2- 172.21.15.121:0/1041568588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b8415b400 0x7f5b841f5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:48.457 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.457+0000 7f5b83e00640 1 -- 172.21.15.121:0/1041568588 >> 172.21.15.121:0/1041568588 conn(0x7f5b840d0040 msgr2=0x7f5b840afa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:48.457 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.457+0000 7f5b83e00640 1 -- 172.21.15.121:0/1041568588 shutdown_connections 2024-04-20T10:22:48.457 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.457+0000 7f5b83e00640 1 -- 172.21.15.121:0/1041568588 wait complete. 2024-04-20T10:22:48.473 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-20T10:22:48.473 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-20T10:22:48.667 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.667+0000 7f895e200640 1 Processor -- start 2024-04-20T10:22:48.667 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.667+0000 7f895e200640 1 -- start start 2024-04-20T10:22:48.668 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.668+0000 7f895e200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f89580d2840 0x7f89581617d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:48.668 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.668+0000 7f895e200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8958161d10 0x7f89580cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:48.668 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.668+0000 7f895e200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89580d2e30 con 0x7f89580d2460 2024-04-20T10:22:48.668 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.669+0000 7f895e200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f89580d3110 con 0x7f89580d2840 2024-04-20T10:22:48.668 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.669+0000 7f895e200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f89580d33f0 con 0x7f8958161d10 2024-04-20T10:22:48.669 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.669+0000 7f894fe00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f89580d2840 0x7f89581617d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:48.669 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.669+0000 7f894fe00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f89580d2840 0x7f89581617d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:48010/0 (socket says 172.21.15.121:48010) 2024-04-20T10:22:48.669 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.669+0000 7f894fe00640 1 -- 172.21.15.121:0/3536781946 learned_addr learned my addr 172.21.15.121:0/3536781946 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:48.669 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.669+0000 7f895d800640 1 --2- 172.21.15.121:0/3536781946 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8958161d10 0x7f89580cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:48.669 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.669+0000 7f894fe00640 1 -- 172.21.15.121:0/3536781946 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8958161d10 msgr2=0x7f89580cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:48.670 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.669+0000 7f894fe00640 1 --2- 172.21.15.121:0/3536781946 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8958161d10 0x7f89580cfa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:48.670 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.669+0000 7f894fe00640 1 -- 172.21.15.121:0/3536781946 >> v1:172.21.15.123:6789/0 conn(0x7f89580d2460 legacy=0x7f89580d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:48.670 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.669+0000 7f894fe00640 1 -- 172.21.15.121:0/3536781946 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89580d36d0 con 0x7f89580d2840 2024-04-20T10:22:48.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.670+0000 7f894fe00640 1 --2- 172.21.15.121:0/3536781946 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89580d2840 0x7f89581617d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8940009870 tx=0x7f894002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94fa405b9f47d14e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:48.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.671+0000 7f8957e00640 1 -- 172.21.15.121:0/3536781946 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8940034070 con 0x7f89580d2840 2024-04-20T10:22:48.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.671+0000 7f8957e00640 1 -- 172.21.15.121:0/3536781946 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8940030840 con 0x7f89580d2840 2024-04-20T10:22:48.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.671+0000 7f8957e00640 1 -- 172.21.15.121:0/3536781946 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8940030c10 con 0x7f89580d2840 2024-04-20T10:22:48.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.671+0000 7f895e200640 1 -- 172.21.15.121:0/3536781946 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89580d2840 msgr2=0x7f89581617d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:48.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.671+0000 7f895e200640 1 --2- 172.21.15.121:0/3536781946 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89580d2840 0x7f89581617d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f8940009870 tx=0x7f894002b980 comp rx=0 tx=0).stop 2024-04-20T10:22:48.671 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.671+0000 7f895e200640 1 -- 172.21.15.121:0/3536781946 shutdown_connections 2024-04-20T10:22:48.672 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.671+0000 7f895e200640 1 --2- 172.21.15.121:0/3536781946 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f8958161d10 0x7f89580cfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:48.672 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.671+0000 7f895e200640 1 --2- 172.21.15.121:0/3536781946 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89580d2840 0x7f89581617d0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:48.672 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.671+0000 7f895e200640 1 -- 172.21.15.121:0/3536781946 >> 172.21.15.121:0/3536781946 conn(0x7f89580b7570 msgr2=0x7f89580b7980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:48.672 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.672+0000 7f895e200640 1 -- 172.21.15.121:0/3536781946 shutdown_connections 2024-04-20T10:22:48.672 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.672+0000 7f895e200640 1 -- 172.21.15.121:0/3536781946 wait complete. 2024-04-20T10:22:48.672 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.673+0000 7f895e200640 1 Processor -- start 2024-04-20T10:22:48.673 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.673+0000 7f895e200640 1 -- start start 2024-04-20T10:22:48.674 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.674+0000 7f895e200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89580d2460 0x7f89580d1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:48.675 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.674+0000 7f895e200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f89580d2840 0x7f89580cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:48.675 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.674+0000 7f895e200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8958161d10 0x7f8958158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:48.675 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.674+0000 7f895e200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f89580d5010 con 0x7f89580d2460 2024-04-20T10:22:48.675 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.674+0000 7f895e200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8958162160 con 0x7f8958161d10 2024-04-20T10:22:48.675 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.674+0000 7f895e200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f89580d44d0 con 0x7f89580d2840 2024-04-20T10:22:48.675 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.674+0000 7f895ce00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89580d2460 0x7f89580d1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:48.675 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.674+0000 7f895ce00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89580d2460 0x7f89580d1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:48016/0 (socket says 172.21.15.121:48016) 2024-04-20T10:22:48.675 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.674+0000 7f895ce00640 1 -- 172.21.15.121:0/1232850267 learned_addr learned my addr 172.21.15.121:0/1232850267 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:48.675 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.675+0000 7f894fe00640 1 --2- 172.21.15.121:0/1232850267 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f89580d2840 0x7f89580cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:48.675 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.675+0000 7f895d800640 1 --2- 172.21.15.121:0/1232850267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8958161d10 0x7f8958158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:48.675 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.675+0000 7f895ce00640 1 -- 172.21.15.121:0/1232850267 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f89580d2840 msgr2=0x7f89580cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:48.675 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.675+0000 7f895ce00640 1 --2- 172.21.15.121:0/1232850267 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f89580d2840 0x7f89580cab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:48.675 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.675+0000 7f895ce00640 1 -- 172.21.15.121:0/1232850267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8958161d10 msgr2=0x7f8958158650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:48.675 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.675+0000 7f895ce00640 1 --2- 172.21.15.121:0/1232850267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8958161d10 0x7f8958158650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:48.676 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.675+0000 7f895ce00640 1 -- 172.21.15.121:0/1232850267 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89580d18b0 con 0x7f89580d2460 2024-04-20T10:22:48.676 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.675+0000 7f895ce00640 1 --2- 172.21.15.121:0/1232850267 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89580d2460 0x7f89580d1210 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f894800c430 tx=0x7f894800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:48.676 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.675+0000 7f8955600640 1 -- 172.21.15.121:0/1232850267 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8948011790 con 0x7f89580d2460 2024-04-20T10:22:48.676 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.676+0000 7f8955600640 1 -- 172.21.15.121:0/1232850267 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8948017070 con 0x7f89580d2460 2024-04-20T10:22:48.677 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.676+0000 7f895e200640 1 -- 172.21.15.121:0/1232850267 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89580b4260 con 0x7f89580d2460 2024-04-20T10:22:48.677 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.676+0000 7f8955600640 1 -- 172.21.15.121:0/1232850267 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8948011bb0 con 0x7f89580d2460 2024-04-20T10:22:48.677 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.676+0000 7f895e200640 1 -- 172.21.15.121:0/1232850267 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8958158b90 con 0x7f89580d2460 2024-04-20T10:22:48.677 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.677+0000 7f895e200640 1 -- 172.21.15.121:0/1232850267 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89580b4050 con 0x7f89580d2460 2024-04-20T10:22:48.677 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.678+0000 7f8955600640 1 -- 172.21.15.121:0/1232850267 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f894801c460 con 0x7f89580d2460 2024-04-20T10:22:48.678 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.679+0000 7f8955600640 1 --2- 172.21.15.121:0/1232850267 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f893406c920 0x7f893406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:48.678 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.679+0000 7f8955600640 1 -- 172.21.15.121:0/1232850267 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f894808b8d0 con 0x7f89580d2460 2024-04-20T10:22:48.678 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.679+0000 7f894fe00640 1 --2- 172.21.15.121:0/1232850267 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f893406c920 0x7f893406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:48.679 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.680+0000 7f894fe00640 1 --2- 172.21.15.121:0/1232850267 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f893406c920 0x7f893406ede0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8940036000 tx=0x7f894003b040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:48.682 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.683+0000 7f8955600640 1 -- 172.21.15.121:0/1232850267 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8948059e20 con 0x7f89580d2460 2024-04-20T10:22:48.879 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:48.879+0000 7f895e200640 1 -- 172.21.15.121:0/1232850267 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f8958075b30 con 0x7f89580d2460 2024-04-20T10:22:49.161 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.161+0000 7f8955600640 1 -- 172.21.15.121:0/1232850267 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v36) ==== 138+0+0 (secure 0 0 0) 0x7f8948059720 con 0x7f89580d2460 2024-04-20T10:22:49.161 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'cephfs_metadata' created 2024-04-20T10:22:49.165 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.165+0000 7f895e200640 1 -- 172.21.15.121:0/1232850267 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f893406c920 msgr2=0x7f893406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:49.165 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.165+0000 7f895e200640 1 --2- 172.21.15.121:0/1232850267 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f893406c920 0x7f893406ede0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8940036000 tx=0x7f894003b040 comp rx=0 tx=0).stop 2024-04-20T10:22:49.165 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.165+0000 7f895e200640 1 -- 172.21.15.121:0/1232850267 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89580d2460 msgr2=0x7f89580d1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:49.165 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.165+0000 7f895e200640 1 --2- 172.21.15.121:0/1232850267 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89580d2460 0x7f89580d1210 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f894800c430 tx=0x7f894800c8f0 comp rx=0 tx=0).stop 2024-04-20T10:22:49.165 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.165+0000 7f895e200640 1 -- 172.21.15.121:0/1232850267 shutdown_connections 2024-04-20T10:22:49.165 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.166+0000 7f895e200640 1 --2- 172.21.15.121:0/1232850267 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f893406c920 0x7f893406ede0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:49.165 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.166+0000 7f895e200640 1 --2- 172.21.15.121:0/1232850267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8958161d10 0x7f8958158650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:49.165 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.166+0000 7f895e200640 1 --2- 172.21.15.121:0/1232850267 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f89580d2840 0x7f89580cab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:49.165 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.166+0000 7f895e200640 1 --2- 172.21.15.121:0/1232850267 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f89580d2460 0x7f89580d1210 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:49.165 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.166+0000 7f895e200640 1 -- 172.21.15.121:0/1232850267 >> 172.21.15.121:0/1232850267 conn(0x7f89580b7570 msgr2=0x7f895815c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:49.166 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.166+0000 7f895e200640 1 -- 172.21.15.121:0/1232850267 shutdown_connections 2024-04-20T10:22:49.166 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.166+0000 7f895e200640 1 -- 172.21.15.121:0/1232850267 wait complete. 2024-04-20T10:22:49.193 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-20T10:22:49.381 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.381+0000 7f7610c00640 1 Processor -- start 2024-04-20T10:22:49.381 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.382+0000 7f7610c00640 1 -- start start 2024-04-20T10:22:49.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.383+0000 7f7610c00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f760c159e90 0x7f760c0cca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:49.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.383+0000 7f7610c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f760c15ff80 0x7f760c0b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:49.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.383+0000 7f7610c00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f760c0b2000 0x7f760c0d15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:49.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.383+0000 7f7610c00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f760c160480 con 0x7f760c0b2000 2024-04-20T10:22:49.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.383+0000 7f7610c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f760c160760 con 0x7f760c15ff80 2024-04-20T10:22:49.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.383+0000 7f7610c00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f760c160a40 con 0x7f760c159e90 2024-04-20T10:22:49.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.383+0000 7f760be00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f760c0b2000 0x7f760c0d15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:49.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.384+0000 7f760be00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f760c0b2000 0x7f760c0d15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:48030/0 (socket says 172.21.15.121:48030) 2024-04-20T10:22:49.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.384+0000 7f760be00640 1 -- 172.21.15.121:0/1002228751 learned_addr learned my addr 172.21.15.121:0/1002228751 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:49.384 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.384+0000 7f760b400640 1 --2- 172.21.15.121:0/1002228751 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f760c159e90 0x7f760c0cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:49.384 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.384+0000 7f760aa00640 1 --2- 172.21.15.121:0/1002228751 >> v2:172.21.15.123:3300/0 conn(0x7f760c15ff80 0x7f760c0b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:49.384 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.384+0000 7f760be00640 1 -- 172.21.15.121:0/1002228751 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f760c159e90 msgr2=0x7f760c0cca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:49.384 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.384+0000 7f760be00640 1 --2- 172.21.15.121:0/1002228751 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f760c159e90 0x7f760c0cca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:49.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.384+0000 7f760be00640 1 -- 172.21.15.121:0/1002228751 >> v2:172.21.15.123:3300/0 conn(0x7f760c15ff80 msgr2=0x7f760c0b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:49.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.384+0000 7f760be00640 1 --2- 172.21.15.121:0/1002228751 >> v2:172.21.15.123:3300/0 conn(0x7f760c15ff80 0x7f760c0b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:49.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.384+0000 7f760be00640 1 -- 172.21.15.121:0/1002228751 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f760c160d20 con 0x7f760c0b2000 2024-04-20T10:22:49.386 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.385+0000 7f760be00640 1 --2- 172.21.15.121:0/1002228751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f760c0b2000 0x7f760c0d15a0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f760c0d32c0 tx=0x7f760402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df3eb01a1c3feca5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:49.386 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.386+0000 7f760a000640 1 -- 172.21.15.121:0/1002228751 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760402c9e0 con 0x7f760c0b2000 2024-04-20T10:22:49.386 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.386+0000 7f760a000640 1 -- 172.21.15.121:0/1002228751 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f760402cb80 con 0x7f760c0b2000 2024-04-20T10:22:49.386 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.386+0000 7f760a000640 1 -- 172.21.15.121:0/1002228751 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76040343f0 con 0x7f760c0b2000 2024-04-20T10:22:49.386 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.386+0000 7f7610c00640 1 -- 172.21.15.121:0/1002228751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f760c0b2000 msgr2=0x7f760c0d15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:49.386 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.386+0000 7f7610c00640 1 --2- 172.21.15.121:0/1002228751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f760c0b2000 0x7f760c0d15a0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f760c0d32c0 tx=0x7f760402be40 comp rx=0 tx=0).stop 2024-04-20T10:22:49.386 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.386+0000 7f7610c00640 1 -- 172.21.15.121:0/1002228751 shutdown_connections 2024-04-20T10:22:49.386 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.386+0000 7f7610c00640 1 --2- 172.21.15.121:0/1002228751 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f760c0b2000 0x7f760c0d15a0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:49.386 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.386+0000 7f7610c00640 1 --2- 172.21.15.121:0/1002228751 >> v2:172.21.15.123:3300/0 conn(0x7f760c15ff80 0x7f760c0b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:49.387 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.386+0000 7f7610c00640 1 --2- 172.21.15.121:0/1002228751 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f760c159e90 0x7f760c0cca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:49.387 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.386+0000 7f7610c00640 1 -- 172.21.15.121:0/1002228751 >> 172.21.15.121:0/1002228751 conn(0x7f760c0cece0 msgr2=0x7f760c0cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:49.387 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.386+0000 7f7610c00640 1 -- 172.21.15.121:0/1002228751 shutdown_connections 2024-04-20T10:22:49.387 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.386+0000 7f7610c00640 1 -- 172.21.15.121:0/1002228751 wait complete. 2024-04-20T10:22:49.387 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.388+0000 7f7610c00640 1 Processor -- start 2024-04-20T10:22:49.388 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.388+0000 7f7610c00640 1 -- start start 2024-04-20T10:22:49.390 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.389+0000 7f7610c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f760c159e90 0x7f760c0cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:49.390 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.389+0000 7f7610c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f760c15ff80 0x7f760c0704c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:49.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.389+0000 7f7610c00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f760c070a00 0x7f760c14ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:49.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.389+0000 7f7610c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f760c0ccfc0 con 0x7f760c159e90 2024-04-20T10:22:49.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.389+0000 7f7610c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f760c15c940 con 0x7f760c15ff80 2024-04-20T10:22:49.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.389+0000 7f7610c00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f760c0cd140 con 0x7f760c070a00 2024-04-20T10:22:49.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.389+0000 7f760b400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f760c159e90 0x7f760c0cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:49.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.390+0000 7f760b400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f760c159e90 0x7f760c0cbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:48036/0 (socket says 172.21.15.121:48036) 2024-04-20T10:22:49.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.390+0000 7f760b400640 1 -- 172.21.15.121:0/49914990 learned_addr learned my addr 172.21.15.121:0/49914990 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:49.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.390+0000 7f760aa00640 1 --2- 172.21.15.121:0/49914990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f760c15ff80 0x7f760c0704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:49.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.390+0000 7f760be00640 1 --2- 172.21.15.121:0/49914990 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f760c070a00 0x7f760c14ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:49.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.390+0000 7f760b400640 1 -- 172.21.15.121:0/49914990 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f760c070a00 msgr2=0x7f760c14ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:49.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.390+0000 7f760b400640 1 --2- 172.21.15.121:0/49914990 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f760c070a00 0x7f760c14ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:49.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.390+0000 7f760b400640 1 -- 172.21.15.121:0/49914990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f760c15ff80 msgr2=0x7f760c0704c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:49.392 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.390+0000 7f760b400640 1 --2- 172.21.15.121:0/49914990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f760c15ff80 0x7f760c0704c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:49.392 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.390+0000 7f760b400640 1 -- 172.21.15.121:0/49914990 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f760c06e2b0 con 0x7f760c159e90 2024-04-20T10:22:49.392 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.390+0000 7f760aa00640 1 --2- 172.21.15.121:0/49914990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f760c15ff80 0x7f760c0704c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:49.392 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.390+0000 7f760b400640 1 --2- 172.21.15.121:0/49914990 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f760c159e90 0x7f760c0cbcb0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f75fc00d800 tx=0x7f75fc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:49.392 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.391+0000 7f7603400640 1 -- 172.21.15.121:0/49914990 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75fc018020 con 0x7f760c159e90 2024-04-20T10:22:49.392 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.391+0000 7f7603400640 1 -- 172.21.15.121:0/49914990 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75fc0096a0 con 0x7f760c159e90 2024-04-20T10:22:49.392 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.391+0000 7f7603400640 1 -- 172.21.15.121:0/49914990 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75fc009a70 con 0x7f760c159e90 2024-04-20T10:22:49.392 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.391+0000 7f7610c00640 1 -- 172.21.15.121:0/49914990 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f760c06e590 con 0x7f760c159e90 2024-04-20T10:22:49.392 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.391+0000 7f7610c00640 1 -- 172.21.15.121:0/49914990 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f760c06eb50 con 0x7f760c159e90 2024-04-20T10:22:49.393 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.392+0000 7f7610c00640 1 -- 172.21.15.121:0/49914990 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f760c070e10 con 0x7f760c159e90 2024-04-20T10:22:49.393 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.393+0000 7f7603400640 1 -- 172.21.15.121:0/49914990 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f75fc01b020 con 0x7f760c159e90 2024-04-20T10:22:49.393 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.394+0000 7f7603400640 1 --2- 172.21.15.121:0/49914990 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f75ec06c8f0 0x7f75ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:49.394 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.394+0000 7f7603400640 1 -- 172.21.15.121:0/49914990 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5619+0+0 (secure 0 0 0) 0x7f75fc08bd00 con 0x7f760c159e90 2024-04-20T10:22:49.394 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.394+0000 7f760aa00640 1 --2- 172.21.15.121:0/49914990 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f75ec06c8f0 0x7f75ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:49.394 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.395+0000 7f760aa00640 1 --2- 172.21.15.121:0/49914990 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f75ec06c8f0 0x7f75ec06edb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f760c157fc0 tx=0x7f75f8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:49.398 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.399+0000 7f7603400640 1 -- 172.21.15.121:0/49914990 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f760c070e10 con 0x7f760c159e90 2024-04-20T10:22:49.602 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:49.602+0000 7f7610c00640 1 -- 172.21.15.121:0/49914990 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f760c0b34c0 con 0x7f760c159e90 2024-04-20T10:22:50.168 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.168+0000 7f7603400640 1 -- 172.21.15.121:0/49914990 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v37) ==== 170+0+0 (secure 0 0 0) 0x7f75fc022030 con 0x7f760c159e90 2024-04-20T10:22:50.168 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'cephfs_data' created 2024-04-20T10:22:50.171 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.171+0000 7f7610c00640 1 -- 172.21.15.121:0/49914990 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f75ec06c8f0 msgr2=0x7f75ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:50.171 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.171+0000 7f7610c00640 1 --2- 172.21.15.121:0/49914990 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f75ec06c8f0 0x7f75ec06edb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f760c157fc0 tx=0x7f75f8008040 comp rx=0 tx=0).stop 2024-04-20T10:22:50.171 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.171+0000 7f7610c00640 1 -- 172.21.15.121:0/49914990 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f760c159e90 msgr2=0x7f760c0cbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:50.171 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.171+0000 7f7610c00640 1 --2- 172.21.15.121:0/49914990 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f760c159e90 0x7f760c0cbcb0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f75fc00d800 tx=0x7f75fc00dcc0 comp rx=0 tx=0).stop 2024-04-20T10:22:50.172 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.172+0000 7f7610c00640 1 -- 172.21.15.121:0/49914990 shutdown_connections 2024-04-20T10:22:50.172 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.172+0000 7f7610c00640 1 --2- 172.21.15.121:0/49914990 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f760c070a00 0x7f760c14ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:50.172 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.172+0000 7f7610c00640 1 --2- 172.21.15.121:0/49914990 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f75ec06c8f0 0x7f75ec06edb0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:50.172 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.172+0000 7f7610c00640 1 --2- 172.21.15.121:0/49914990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f760c15ff80 0x7f760c0704c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:50.172 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.172+0000 7f7610c00640 1 --2- 172.21.15.121:0/49914990 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f760c159e90 0x7f760c0cbcb0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:50.172 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.172+0000 7f7610c00640 1 -- 172.21.15.121:0/49914990 >> 172.21.15.121:0/49914990 conn(0x7f760c0cece0 msgr2=0x7f760c15a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:50.172 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.172+0000 7f7610c00640 1 -- 172.21.15.121:0/49914990 shutdown_connections 2024-04-20T10:22:50.172 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.172+0000 7f7610c00640 1 -- 172.21.15.121:0/49914990 wait complete. 2024-04-20T10:22:50.199 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-20T10:22:50.379 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.379+0000 7f3710c00640 1 Processor -- start 2024-04-20T10:22:50.379 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.379+0000 7f3710c00640 1 -- start start 2024-04-20T10:22:50.380 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.381+0000 7f3710c00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f370c156c20 0x7f370c06b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:50.380 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.381+0000 7f3710c00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f370c141c60 con 0x7f370c0d0360 2024-04-20T10:22:50.380 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.381+0000 7f3710c00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f370c0adab0 con 0x7f370c15ca40 2024-04-20T10:22:50.381 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.381+0000 7f3710c00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f370c15cf90 con 0x7f370c156c20 2024-04-20T10:22:50.381 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.381+0000 7f370a800640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f370c0d0360 0x7f370c142140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:44840/0 (socket says 172.21.15.121:44840) 2024-04-20T10:22:50.381 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.381+0000 7f370a800640 1 -- 172.21.15.121:0/3808368726 learned_addr learned my addr 172.21.15.121:0/3808368726 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:50.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.381+0000 7f3709e00640 1 --2- 172.21.15.121:0/3808368726 >> v2:172.21.15.123:3300/0 conn(0x7f370c156c20 0x7f370c06b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:50.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.382+0000 7f3708a00640 1 -- 172.21.15.121:0/3808368726 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 212619082 0 0) 0x7f370c141c60 con 0x7f370c0d0360 2024-04-20T10:22:50.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.382+0000 7f3708a00640 1 -- 172.21.15.121:0/3808368726 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f36f0003520 con 0x7f370c0d0360 2024-04-20T10:22:50.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.382+0000 7f3708a00640 1 -- 172.21.15.121:0/3808368726 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1724111711 0 0) 0x7f36f0003520 con 0x7f370c0d0360 2024-04-20T10:22:50.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.382+0000 7f3708a00640 1 -- 172.21.15.121:0/3808368726 >> v2:172.21.15.123:3300/0 conn(0x7f370c156c20 msgr2=0x7f370c06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:50.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.382+0000 7f3708a00640 1 --2- 172.21.15.121:0/3808368726 >> v2:172.21.15.123:3300/0 conn(0x7f370c156c20 0x7f370c06b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:50.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.382+0000 7f3708a00640 1 -- 172.21.15.121:0/3808368726 >> v1:172.21.15.123:6789/0 conn(0x7f370c15ca40 legacy=0x7f370c0ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:50.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.382+0000 7f3708a00640 1 -- 172.21.15.121:0/3808368726 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f370c15d270 con 0x7f370c0d0360 2024-04-20T10:22:50.384 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.382+0000 7f3708a00640 1 -- 172.21.15.121:0/3808368726 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f37040019f0 con 0x7f370c0d0360 2024-04-20T10:22:50.384 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.382+0000 7f3708a00640 1 -- 172.21.15.121:0/3808368726 >> v1:172.21.15.121:6789/0 conn(0x7f370c0d0360 legacy=0x7f370c142140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:50.384 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.382+0000 7f3708a00640 1 --2- 172.21.15.121:0/3808368726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36f00039b0 0x7f36f0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:50.384 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.382+0000 7f3708a00640 1 --2- 172.21.15.121:0/3808368726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36f0004800 0x7f36f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:50.384 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.382+0000 7f3708a00640 1 --2- 172.21.15.121:0/3808368726 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f36f00051c0 0x7f36f000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:50.384 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.382+0000 7f3708a00640 1 -- 172.21.15.121:0/3808368726 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f370c15cf90 con 0x7f36f0004800 2024-04-20T10:22:50.384 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.382+0000 7f3708a00640 1 -- 172.21.15.121:0/3808368726 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f36f000c060 con 0x7f36f00039b0 2024-04-20T10:22:50.384 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.382+0000 7f3708a00640 1 -- 172.21.15.121:0/3808368726 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f36f000c260 con 0x7f36f00051c0 2024-04-20T10:22:50.384 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.383+0000 7f3709400640 1 --2- 172.21.15.121:0/3808368726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36f0004800 0x7f36f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:50.384 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.383+0000 7f3709e00640 1 --2- 172.21.15.121:0/3808368726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36f00039b0 0x7f36f0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:50.384 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.383+0000 7f3709400640 1 -- 172.21.15.121:0/3808368726 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f36f00051c0 msgr2=0x7f36f000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:50.384 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.383+0000 7f370a800640 1 --2- 172.21.15.121:0/3808368726 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f36f00051c0 0x7f36f000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:50.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.383+0000 7f3709400640 1 --2- 172.21.15.121:0/3808368726 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f36f00051c0 0x7f36f000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:50.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.383+0000 7f3709400640 1 -- 172.21.15.121:0/3808368726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36f00039b0 msgr2=0x7f36f0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:50.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.383+0000 7f3709400640 1 --2- 172.21.15.121:0/3808368726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36f00039b0 0x7f36f0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:50.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.383+0000 7f3709400640 1 -- 172.21.15.121:0/3808368726 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f0003520 con 0x7f36f0004800 2024-04-20T10:22:50.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.383+0000 7f370a800640 1 --2- 172.21.15.121:0/3808368726 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f36f00051c0 0x7f36f000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:50.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.383+0000 7f370a800640 1 -- 172.21.15.121:0/3808368726 reap_dead start 2024-04-20T10:22:50.386 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.384+0000 7f3709400640 1 --2- 172.21.15.121:0/3808368726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36f0004800 0x7f36f0004c80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f36f8001750 tx=0x7f36f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dde2f5374719bca2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:50.386 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.385+0000 7f3708a00640 1 -- 172.21.15.121:0/3808368726 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f8001370 con 0x7f36f0004800 2024-04-20T10:22:50.386 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.385+0000 7f3708a00640 1 -- 172.21.15.121:0/3808368726 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36f80052d0 con 0x7f36f0004800 2024-04-20T10:22:50.386 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.385+0000 7f3708a00640 1 -- 172.21.15.121:0/3808368726 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f8003830 con 0x7f36f0004800 2024-04-20T10:22:50.386 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.385+0000 7f3710c00640 1 -- 172.21.15.121:0/3808368726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36f0004800 msgr2=0x7f36f0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:50.386 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.385+0000 7f3710c00640 1 --2- 172.21.15.121:0/3808368726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36f0004800 0x7f36f0004c80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f36f8001750 tx=0x7f36f0005830 comp rx=0 tx=0).stop 2024-04-20T10:22:50.386 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.385+0000 7f3710c00640 1 -- 172.21.15.121:0/3808368726 shutdown_connections 2024-04-20T10:22:50.387 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.385+0000 7f3710c00640 1 --2- 172.21.15.121:0/3808368726 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f36f0004800 0x7f36f0004c80 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:50.387 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.385+0000 7f3710c00640 1 -- 172.21.15.121:0/3808368726 >> 172.21.15.121:0/3808368726 conn(0x7f370c0b3350 msgr2=0x7f370c0b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:50.387 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.385+0000 7f3710c00640 1 -- 172.21.15.121:0/3808368726 shutdown_connections 2024-04-20T10:22:50.387 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.385+0000 7f3710c00640 1 -- 172.21.15.121:0/3808368726 wait complete. 2024-04-20T10:22:50.388 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.387+0000 7f3710c00640 1 Processor -- start 2024-04-20T10:22:50.388 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.387+0000 7f3710c00640 1 -- start start 2024-04-20T10:22:50.388 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.388+0000 7f3710c00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36f0004800 0x7f370c0cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:50.388 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.388+0000 7f3710c00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f370c091d00 0x7f370c1f47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:50.388 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.388+0000 7f3710c00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f370c154270 0x7f370c1546f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:50.388 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.388+0000 7f3710c00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f370c0d07d0 con 0x7f370c091d00 2024-04-20T10:22:50.388 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.388+0000 7f3710c00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f370c15d830 con 0x7f36f0004800 2024-04-20T10:22:50.388 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.388+0000 7f3710c00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f370c0d0950 con 0x7f370c154270 2024-04-20T10:22:50.389 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.388+0000 7f3709400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f370c091d00 0x7f370c1f47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:50.389 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.389+0000 7f3709400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f370c091d00 0x7f370c1f47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:48048/0 (socket says 172.21.15.121:48048) 2024-04-20T10:22:50.390 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.389+0000 7f3709400640 1 -- 172.21.15.121:0/2532858933 learned_addr learned my addr 172.21.15.121:0/2532858933 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:50.390 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.389+0000 7f3709e00640 1 --2- 172.21.15.121:0/2532858933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36f0004800 0x7f370c0cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:50.390 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.389+0000 7f370a800640 1 --2- 172.21.15.121:0/2532858933 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f370c154270 0x7f370c1546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:50.390 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.389+0000 7f3709400640 1 -- 172.21.15.121:0/2532858933 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f370c154270 msgr2=0x7f370c1546f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:50.390 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.389+0000 7f3709400640 1 --2- 172.21.15.121:0/2532858933 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f370c154270 0x7f370c1546f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:50.390 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.389+0000 7f3709400640 1 -- 172.21.15.121:0/2532858933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36f0004800 msgr2=0x7f370c0cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:50.390 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.389+0000 7f3709400640 1 --2- 172.21.15.121:0/2532858933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36f0004800 0x7f370c0cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:50.390 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.389+0000 7f3709400640 1 -- 172.21.15.121:0/2532858933 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f370c0b0cc0 con 0x7f370c091d00 2024-04-20T10:22:50.390 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.390+0000 7f3709400640 1 --2- 172.21.15.121:0/2532858933 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f370c091d00 0x7f370c1f47b0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f36f802bcd0 tx=0x7f36f8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:50.390 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.390+0000 7f3702000640 1 -- 172.21.15.121:0/2532858933 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f8041070 con 0x7f370c091d00 2024-04-20T10:22:50.390 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.390+0000 7f3702000640 1 -- 172.21.15.121:0/2532858933 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36f8005480 con 0x7f370c091d00 2024-04-20T10:22:50.390 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.390+0000 7f3702000640 1 -- 172.21.15.121:0/2532858933 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f8007a30 con 0x7f370c091d00 2024-04-20T10:22:50.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.390+0000 7f3710c00640 1 -- 172.21.15.121:0/2532858933 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f370c1f4cf0 con 0x7f370c091d00 2024-04-20T10:22:50.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.390+0000 7f3710c00640 1 -- 172.21.15.121:0/2532858933 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f370c0cdeb0 con 0x7f370c091d00 2024-04-20T10:22:50.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.391+0000 7f3710c00640 1 -- 172.21.15.121:0/2532858933 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f370c068980 con 0x7f370c091d00 2024-04-20T10:22:50.392 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.393+0000 7f3702000640 1 -- 172.21.15.121:0/2532858933 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f36f8044070 con 0x7f370c091d00 2024-04-20T10:22:50.393 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.393+0000 7f3702000640 1 --2- 172.21.15.121:0/2532858933 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f36ec06c8f0 0x7f36ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:50.393 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.394+0000 7f3702000640 1 -- 172.21.15.121:0/2532858933 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6001+0+0 (secure 0 0 0) 0x7f36f80ad780 con 0x7f370c091d00 2024-04-20T10:22:50.393 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.394+0000 7f3709e00640 1 --2- 172.21.15.121:0/2532858933 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f36ec06c8f0 0x7f36ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:50.394 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.394+0000 7f3709e00640 1 --2- 172.21.15.121:0/2532858933 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f36ec06c8f0 0x7f36ec06edb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f36fc009b40 tx=0x7f36fc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:50.398 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.399+0000 7f3702000640 1 -- 172.21.15.121:0/2532858933 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f36f807b4c0 con 0x7f370c091d00 2024-04-20T10:22:50.638 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:50.638+0000 7f3710c00640 1 -- 172.21.15.121:0/2532858933 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f370c0cfd90 con 0x7f370c091d00 2024-04-20T10:22:51.172 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.172+0000 7f3702000640 1 -- 172.21.15.121:0/2532858933 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v13) ==== 172+0+0 (secure 0 0 0) 0x7f36f807f7c0 con 0x7f370c091d00 2024-04-20T10:22:51.172 INFO:teuthology.orchestra.run.smithi121.stderr:new fs with metadata pool 5 and data pool 6 2024-04-20T10:22:51.175 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.175+0000 7f3710c00640 1 -- 172.21.15.121:0/2532858933 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f36ec06c8f0 msgr2=0x7f36ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:51.175 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.175+0000 7f3710c00640 1 --2- 172.21.15.121:0/2532858933 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f36ec06c8f0 0x7f36ec06edb0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f36fc009b40 tx=0x7f36fc008040 comp rx=0 tx=0).stop 2024-04-20T10:22:51.175 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.176+0000 7f3710c00640 1 -- 172.21.15.121:0/2532858933 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f370c091d00 msgr2=0x7f370c1f47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:51.175 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.176+0000 7f3710c00640 1 --2- 172.21.15.121:0/2532858933 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f370c091d00 0x7f370c1f47b0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f36f802bcd0 tx=0x7f36f8001120 comp rx=0 tx=0).stop 2024-04-20T10:22:51.176 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.176+0000 7f3710c00640 1 -- 172.21.15.121:0/2532858933 shutdown_connections 2024-04-20T10:22:51.176 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.176+0000 7f3710c00640 1 --2- 172.21.15.121:0/2532858933 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f370c154270 0x7f370c1546f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:51.176 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.176+0000 7f3710c00640 1 --2- 172.21.15.121:0/2532858933 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f370c091d00 0x7f370c1f47b0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:51.176 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.176+0000 7f3710c00640 1 --2- 172.21.15.121:0/2532858933 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f36ec06c8f0 0x7f36ec06edb0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:51.176 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.176+0000 7f3710c00640 1 --2- 172.21.15.121:0/2532858933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36f0004800 0x7f370c0cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:51.176 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.176+0000 7f3710c00640 1 -- 172.21.15.121:0/2532858933 >> 172.21.15.121:0/2532858933 conn(0x7f370c0b3350 msgr2=0x7f370c15db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:51.176 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.176+0000 7f3710c00640 1 -- 172.21.15.121:0/2532858933 shutdown_connections 2024-04-20T10:22:51.176 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.176+0000 7f3710c00640 1 -- 172.21.15.121:0/2532858933 wait complete. 2024-04-20T10:22:51.213 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:22:51.389 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.389+0000 7ff0e9400640 1 Processor -- start 2024-04-20T10:22:51.389 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.389+0000 7ff0e9400640 1 -- start start 2024-04-20T10:22:51.390 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.391+0000 7ff0e9400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0e415a7b0 0x7ff0e406da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:51.390 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.391+0000 7ff0e9400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0e406b900 con 0x7ff0e406dfd0 2024-04-20T10:22:51.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.391+0000 7ff0e9400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0e406bba0 con 0x7ff0e4071c90 2024-04-20T10:22:51.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.391+0000 7ff0e9400640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ff0e40b50a0 con 0x7ff0e415a7b0 2024-04-20T10:22:51.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.391+0000 7ff0db400640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7ff0e406dfd0 0x7ff0e406c960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:44850/0 (socket says 172.21.15.121:44850) 2024-04-20T10:22:51.391 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.391+0000 7ff0db400640 1 -- 172.21.15.121:0/1026709933 learned_addr learned my addr 172.21.15.121:0/1026709933 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:51.393 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.391+0000 7ff0e3e00640 1 --2- 172.21.15.121:0/1026709933 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0e415a7b0 0x7ff0e406da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:51.393 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.392+0000 7ff0e3400640 1 -- 172.21.15.121:0/1026709933 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3936145517 0 0) 0x7ff0e406b900 con 0x7ff0e406dfd0 2024-04-20T10:22:51.393 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.392+0000 7ff0e3400640 1 -- 172.21.15.121:0/1026709933 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0c8003520 con 0x7ff0e406dfd0 2024-04-20T10:22:51.393 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.392+0000 7ff0e3400640 1 -- 172.21.15.121:0/1026709933 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3603846867 0 0) 0x7ff0c8003520 con 0x7ff0e406dfd0 2024-04-20T10:22:51.393 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.392+0000 7ff0e3400640 1 -- 172.21.15.121:0/1026709933 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0e415a7b0 msgr2=0x7ff0e406da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:51.393 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.392+0000 7ff0e3400640 1 --2- 172.21.15.121:0/1026709933 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0e415a7b0 0x7ff0e406da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:51.393 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.392+0000 7ff0e3400640 1 -- 172.21.15.121:0/1026709933 >> v1:172.21.15.123:6789/0 conn(0x7ff0e4071c90 legacy=0x7ff0e40b3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:51.393 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.392+0000 7ff0e3400640 1 -- 172.21.15.121:0/1026709933 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0e406be80 con 0x7ff0e406dfd0 2024-04-20T10:22:51.393 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.392+0000 7ff0e3400640 1 -- 172.21.15.121:0/1026709933 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7ff0cc001670 con 0x7ff0e406dfd0 2024-04-20T10:22:51.393 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.392+0000 7ff0e3400640 1 -- 172.21.15.121:0/1026709933 >> v1:172.21.15.121:6789/0 conn(0x7ff0e406dfd0 legacy=0x7ff0e406c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:51.393 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.392+0000 7ff0e3400640 1 --2- 172.21.15.121:0/1026709933 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0e415a7b0 0x7ff0c80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:51.393 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.392+0000 7ff0e3400640 1 --2- 172.21.15.121:0/1026709933 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0c8003f60 0x7ff0c8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:51.394 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.392+0000 7ff0e3400640 1 --2- 172.21.15.121:0/1026709933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0c8004d40 0x7ff0c80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:51.394 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.392+0000 7ff0e3400640 1 -- 172.21.15.121:0/1026709933 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e40b50a0 con 0x7ff0c8003f60 2024-04-20T10:22:51.394 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.392+0000 7ff0e3400640 1 -- 172.21.15.121:0/1026709933 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c800dcb0 con 0x7ff0c8004d40 2024-04-20T10:22:51.394 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.392+0000 7ff0e3400640 1 -- 172.21.15.121:0/1026709933 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ff0c800deb0 con 0x7ff0e415a7b0 2024-04-20T10:22:51.394 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.393+0000 7ff0db400640 1 --2- 172.21.15.121:0/1026709933 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0c8003f60 0x7ff0c8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:51.394 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.393+0000 7ff0e8a00640 1 --2- 172.21.15.121:0/1026709933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0c8004d40 0x7ff0c80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:51.394 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.393+0000 7ff0e3e00640 1 --2- 172.21.15.121:0/1026709933 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0e415a7b0 0x7ff0c80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:51.394 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.393+0000 7ff0db400640 1 -- 172.21.15.121:0/1026709933 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0e415a7b0 msgr2=0x7ff0c80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:51.394 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.393+0000 7ff0db400640 1 --2- 172.21.15.121:0/1026709933 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0e415a7b0 0x7ff0c80039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:51.394 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.393+0000 7ff0db400640 1 -- 172.21.15.121:0/1026709933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0c8004d40 msgr2=0x7ff0c80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:51.394 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.393+0000 7ff0db400640 1 --2- 172.21.15.121:0/1026709933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0c8004d40 0x7ff0c80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:51.394 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.393+0000 7ff0db400640 1 -- 172.21.15.121:0/1026709933 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0c8003520 con 0x7ff0c8003f60 2024-04-20T10:22:51.395 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.394+0000 7ff0db400640 1 --2- 172.21.15.121:0/1026709933 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0c8003f60 0x7ff0c8004800 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00c240 tx=0x7ff0c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2c8e26659f1ad2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:51.396 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.394+0000 7ff0e3400640 1 -- 172.21.15.121:0/1026709933 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc007cf0 con 0x7ff0c8003f60 2024-04-20T10:22:51.396 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.394+0000 7ff0e3400640 1 -- 172.21.15.121:0/1026709933 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0cc030070 con 0x7ff0c8003f60 2024-04-20T10:22:51.396 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.394+0000 7ff0e3400640 1 -- 172.21.15.121:0/1026709933 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc005d70 con 0x7ff0c8003f60 2024-04-20T10:22:51.396 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.395+0000 7ff0e9400640 1 -- 172.21.15.121:0/1026709933 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0c8003f60 msgr2=0x7ff0c8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:51.396 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.395+0000 7ff0e9400640 1 --2- 172.21.15.121:0/1026709933 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0c8003f60 0x7ff0c8004800 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00c240 tx=0x7ff0c8005830 comp rx=0 tx=0).stop 2024-04-20T10:22:51.396 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.395+0000 7ff0e8a00640 1 -- 172.21.15.121:0/1026709933 reap_dead start 2024-04-20T10:22:51.396 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.395+0000 7ff0e9400640 1 -- 172.21.15.121:0/1026709933 shutdown_connections 2024-04-20T10:22:51.396 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.395+0000 7ff0e9400640 1 -- 172.21.15.121:0/1026709933 >> 172.21.15.121:0/1026709933 conn(0x7ff0e40d0040 msgr2=0x7ff0e40d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:51.396 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.395+0000 7ff0e9400640 1 -- 172.21.15.121:0/1026709933 shutdown_connections 2024-04-20T10:22:51.396 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.395+0000 7ff0e9400640 1 -- 172.21.15.121:0/1026709933 wait complete. 2024-04-20T10:22:51.397 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.397+0000 7ff0e9400640 1 Processor -- start 2024-04-20T10:22:51.397 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.397+0000 7ff0e9400640 1 -- start start 2024-04-20T10:22:51.399 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.398+0000 7ff0e9400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0c8003f60 0x7ff0e4150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:51.399 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.398+0000 7ff0e9400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0e41526d0 0x7ff0e4152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:51.399 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.398+0000 7ff0e9400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0e409cbc0 0x7ff0e409d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:51.399 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.398+0000 7ff0e9400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e406d070 con 0x7ff0e41526d0 2024-04-20T10:22:51.399 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.398+0000 7ff0e9400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e406c440 con 0x7ff0c8003f60 2024-04-20T10:22:51.399 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.398+0000 7ff0e9400640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ff0e406d1f0 con 0x7ff0e409cbc0 2024-04-20T10:22:51.399 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.398+0000 7ff0db400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0e41526d0 0x7ff0e4152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:51.399 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.398+0000 7ff0db400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0e41526d0 0x7ff0e4152b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:48054/0 (socket says 172.21.15.121:48054) 2024-04-20T10:22:51.399 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.398+0000 7ff0db400640 1 -- 172.21.15.121:0/4229899963 learned_addr learned my addr 172.21.15.121:0/4229899963 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:51.399 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.398+0000 7ff0e8a00640 1 --2- 172.21.15.121:0/4229899963 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0e409cbc0 0x7ff0e409d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:51.399 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.399+0000 7ff0e3e00640 1 --2- 172.21.15.121:0/4229899963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0c8003f60 0x7ff0e4150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:51.399 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.399+0000 7ff0db400640 1 -- 172.21.15.121:0/4229899963 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0e409cbc0 msgr2=0x7ff0e409d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:51.400 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.399+0000 7ff0db400640 1 --2- 172.21.15.121:0/4229899963 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0e409cbc0 0x7ff0e409d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:51.400 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.399+0000 7ff0db400640 1 -- 172.21.15.121:0/4229899963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0c8003f60 msgr2=0x7ff0e4150bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:51.400 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.399+0000 7ff0db400640 1 --2- 172.21.15.121:0/4229899963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0c8003f60 0x7ff0e4150bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:51.400 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.399+0000 7ff0db400640 1 -- 172.21.15.121:0/4229899963 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0e4153250 con 0x7ff0e41526d0 2024-04-20T10:22:51.400 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.399+0000 7ff0e8a00640 1 --2- 172.21.15.121:0/4229899963 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0e409cbc0 0x7ff0e409d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:51.400 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.399+0000 7ff0db400640 1 --2- 172.21.15.121:0/4229899963 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0e41526d0 0x7ff0e4152b50 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00c240 tx=0x7ff0e406c050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:51.400 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.399+0000 7ff0e0c00640 1 -- 172.21.15.121:0/4229899963 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc001c30 con 0x7ff0e41526d0 2024-04-20T10:22:51.400 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.399+0000 7ff0e0c00640 1 -- 172.21.15.121:0/4229899963 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0cc004bf0 con 0x7ff0e41526d0 2024-04-20T10:22:51.400 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.399+0000 7ff0e9400640 1 -- 172.21.15.121:0/4229899963 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0e4153530 con 0x7ff0e41526d0 2024-04-20T10:22:51.400 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.399+0000 7ff0e0c00640 1 -- 172.21.15.121:0/4229899963 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc007070 con 0x7ff0e41526d0 2024-04-20T10:22:51.400 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.399+0000 7ff0e9400640 1 -- 172.21.15.121:0/4229899963 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0e4146db0 con 0x7ff0e41526d0 2024-04-20T10:22:51.401 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.400+0000 7ff0e9400640 1 -- 172.21.15.121:0/4229899963 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0e409d580 con 0x7ff0e41526d0 2024-04-20T10:22:51.401 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.401+0000 7ff0e0c00640 1 -- 172.21.15.121:0/4229899963 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff0cc007210 con 0x7ff0e41526d0 2024-04-20T10:22:51.402 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.402+0000 7ff0e0c00640 1 --2- 172.21.15.121:0/4229899963 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff0c006c920 0x7ff0c006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:51.402 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.402+0000 7ff0e0c00640 1 -- 172.21.15.121:0/4229899963 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7ff0cc0acde0 con 0x7ff0e41526d0 2024-04-20T10:22:51.402 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.403+0000 7ff0e3e00640 1 --2- 172.21.15.121:0/4229899963 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff0c006c920 0x7ff0c006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:51.403 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.403+0000 7ff0e3e00640 1 --2- 172.21.15.121:0/4229899963 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff0c006c920 0x7ff0c006ede0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4009910 tx=0x7ff0d4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:51.406 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.407+0000 7ff0e0c00640 1 -- 172.21.15.121:0/4229899963 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff0cc07a990 con 0x7ff0e41526d0 2024-04-20T10:22:51.606 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.607+0000 7ff0e9400640 1 -- 172.21.15.121:0/4229899963 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff0e4072e70 con 0x7ff0e41526d0 2024-04-20T10:22:51.609 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.610+0000 7ff0e0c00640 1 -- 172.21.15.121:0/4229899963 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16991 (secure 0 0 0) 0x7ff0e4072e70 con 0x7ff0e41526d0 2024-04-20T10:22:51.610 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:22:51.610 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":38,"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","created":"2024-04-20T10:20:38.076076+0000","modified":"2024-04-20T10:22:50.639532+0000","last_up_change":"2024-04-20T10:20:50.162700+0000","last_in_change":"2024-04-20T10:20:41.510382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:20:51.141365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:22:48.893174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-20T10:22:49.616949+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"629a7b8c-a40a-4a2a-a3cc-d9f357774310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6809","nonce":1044588835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6811","nonce":1044588835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6815","nonce":1044588835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6813","nonce":1044588835}]},"public_addr":"172.21.15.121:6809/1044588835","cluster_addr":"172.21.15.121:6811/1044588835","heartbeat_back_addr":"172.21.15.121:6815/1044588835","heartbeat_front_addr":"172.21.15.121:6813/1044588835","state":["exists","up"]},{"osd":1,"uuid":"f2570399-aa50-4ff3-bf5d-3d09207290db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6801","nonce":3505199624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6803","nonce":3505199624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6807","nonce":3505199624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6805","nonce":3505199624}]},"public_addr":"172.21.15.121:6801/3505199624","cluster_addr":"172.21.15.121:6803/3505199624","heartbeat_back_addr":"172.21.15.121:6807/3505199624","heartbeat_front_addr":"172.21.15.121:6805/3505199624","state":["exists","up"]},{"osd":2,"uuid":"50893807-dc30-49f4-b41f-739366626658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6817","nonce":3656724207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6819","nonce":3656724207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6827","nonce":3656724207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6822","nonce":3656724207}]},"public_addr":"172.21.15.121:6817/3656724207","cluster_addr":"172.21.15.121:6819/3656724207","heartbeat_back_addr":"172.21.15.121:6827/3656724207","heartbeat_front_addr":"172.21.15.121:6822/3656724207","state":["exists","up"]},{"osd":3,"uuid":"dadb01bc-5491-4ffc-8824-d48a057240b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6823","nonce":2232342832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6826","nonce":2232342832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6831","nonce":2232342832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6829","nonce":2232342832}]},"public_addr":"172.21.15.121:6823/2232342832","cluster_addr":"172.21.15.121:6826/2232342832","heartbeat_back_addr":"172.21.15.121:6831/2232342832","heartbeat_front_addr":"172.21.15.121:6829/2232342832","state":["exists","up"]},{"osd":4,"uuid":"b3f3b4a1-b375-4526-b69e-344ab24c2da1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6801","nonce":99353111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6803","nonce":99353111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6807","nonce":99353111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6805","nonce":99353111}]},"public_addr":"172.21.15.123:6801/99353111","cluster_addr":"172.21.15.123:6803/99353111","heartbeat_back_addr":"172.21.15.123:6807/99353111","heartbeat_front_addr":"172.21.15.123:6805/99353111","state":["exists","up"]},{"osd":5,"uuid":"2ee24a08-1624-4961-9642-d7b510282484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6818","nonce":2504374688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6822","nonce":2504374688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6830","nonce":2504374688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6826","nonce":2504374688}]},"public_addr":"172.21.15.123:6818/2504374688","cluster_addr":"172.21.15.123:6822/2504374688","heartbeat_back_addr":"172.21.15.123:6830/2504374688","heartbeat_front_addr":"172.21.15.123:6826/2504374688","state":["exists","up"]},{"osd":6,"uuid":"186118cc-6128-49de-acca-55260945c1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6819","nonce":4139299599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6823","nonce":4139299599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6829","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6831","nonce":4139299599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6827","nonce":4139299599}]},"public_addr":"172.21.15.123:6819/4139299599","cluster_addr":"172.21.15.123:6823/4139299599","heartbeat_back_addr":"172.21.15.123:6831/4139299599","heartbeat_front_addr":"172.21.15.123:6827/4139299599","state":["exists","up"]},{"osd":7,"uuid":"a00aac6e-4bd9-415a-a93d-ae85210865ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6809","nonce":4043981069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6811","nonce":4043981069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6815","nonce":4043981069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6813","nonce":4043981069}]},"public_addr":"172.21.15.123:6809/4043981069","cluster_addr":"172.21.15.123:6811/4043981069","heartbeat_back_addr":"172.21.15.123:6815/4043981069","heartbeat_front_addr":"172.21.15.123:6813/4043981069","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.904540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.958585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.616367+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.885244+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.103127+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.167248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.884348+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.862356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:22:51.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.613+0000 7ff0e9400640 1 -- 172.21.15.121:0/4229899963 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff0c006c920 msgr2=0x7ff0c006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:51.613 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.613+0000 7ff0e9400640 1 --2- 172.21.15.121:0/4229899963 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff0c006c920 0x7ff0c006ede0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4009910 tx=0x7ff0d4008040 comp rx=0 tx=0).stop 2024-04-20T10:22:51.613 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.613+0000 7ff0e9400640 1 -- 172.21.15.121:0/4229899963 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0e41526d0 msgr2=0x7ff0e4152b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:51.613 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.613+0000 7ff0e9400640 1 --2- 172.21.15.121:0/4229899963 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0e41526d0 0x7ff0e4152b50 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00c240 tx=0x7ff0e406c050 comp rx=0 tx=0).stop 2024-04-20T10:22:51.613 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.613+0000 7ff0e9400640 1 -- 172.21.15.121:0/4229899963 shutdown_connections 2024-04-20T10:22:51.613 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.613+0000 7ff0e9400640 1 --2- 172.21.15.121:0/4229899963 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff0e409cbc0 0x7ff0e409d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:51.613 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.613+0000 7ff0e9400640 1 --2- 172.21.15.121:0/4229899963 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff0e41526d0 0x7ff0e4152b50 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:51.613 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.613+0000 7ff0e9400640 1 --2- 172.21.15.121:0/4229899963 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff0c006c920 0x7ff0c006ede0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:51.613 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.613+0000 7ff0e9400640 1 --2- 172.21.15.121:0/4229899963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0c8003f60 0x7ff0e4150bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:51.613 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.613+0000 7ff0e9400640 1 -- 172.21.15.121:0/4229899963 >> 172.21.15.121:0/4229899963 conn(0x7ff0e40d0040 msgr2=0x7ff0e40adcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:51.613 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.613+0000 7ff0e9400640 1 -- 172.21.15.121:0/4229899963 shutdown_connections 2024-04-20T10:22:51.613 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.613+0000 7ff0e9400640 1 -- 172.21.15.121:0/4229899963 wait complete. 2024-04-20T10:22:51.629 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:22:51.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.816+0000 7fe50ee00640 1 Processor -- start 2024-04-20T10:22:51.816 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.816+0000 7fe50ee00640 1 -- start start 2024-04-20T10:22:51.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.818+0000 7fe50ee00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe508159e10 0x7fe508071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:51.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.818+0000 7fe50ee00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe50815a1f0 0x7fe508071fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:51.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.818+0000 7fe50ee00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe50815cc90 con 0x7fe50806da90 2024-04-20T10:22:51.817 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.818+0000 7fe50ee00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fe508072900 con 0x7fe508159e10 2024-04-20T10:22:51.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.818+0000 7fe50ee00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe50815ca40 con 0x7fe50815a1f0 2024-04-20T10:22:51.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.819+0000 7fe50d000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe50815a1f0 0x7fe508071fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:51.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.819+0000 7fe50d000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe50815a1f0 0x7fe508071fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3301/0 says I am v2:172.21.15.121:53534/0 (socket says 172.21.15.121:53534) 2024-04-20T10:22:51.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.819+0000 7fe50da00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe508159e10 0x7fe508071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:51.818 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.819+0000 7fe50d000640 1 -- 172.21.15.121:0/4245619427 learned_addr learned my addr 172.21.15.121:0/4245619427 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:51.819 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.819+0000 7fe50d000640 1 -- 172.21.15.121:0/4245619427 >> v2:172.21.15.123:3300/0 conn(0x7fe508159e10 msgr2=0x7fe508071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:51.819 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.819+0000 7fe50d000640 1 --2- 172.21.15.121:0/4245619427 >> v2:172.21.15.123:3300/0 conn(0x7fe508159e10 0x7fe508071a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:51.819 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.819+0000 7fe50d000640 1 -- 172.21.15.121:0/4245619427 >> v1:172.21.15.123:6789/0 conn(0x7fe50806da90 legacy=0x7fe508160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:51.819 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.819+0000 7fe50d000640 1 -- 172.21.15.121:0/4245619427 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe50815cf70 con 0x7fe50815a1f0 2024-04-20T10:22:51.820 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.821+0000 7fe50d000640 1 --2- 172.21.15.121:0/4245619427 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe50815a1f0 0x7fe508071fc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe50815cbc0 tx=0x7fe4f402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39ddcc003981f84d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:51.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.821+0000 7fe507e00640 1 -- 172.21.15.121:0/4245619427 <== mon.2 v2:172.21.15.123:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4034070 con 0x7fe50815a1f0 2024-04-20T10:22:51.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.821+0000 7fe507e00640 1 -- 172.21.15.121:0/4245619427 <== mon.2 v2:172.21.15.123:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4f4030840 con 0x7fe50815a1f0 2024-04-20T10:22:51.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.821+0000 7fe507e00640 1 -- 172.21.15.121:0/4245619427 <== mon.2 v2:172.21.15.123:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4030c10 con 0x7fe50815a1f0 2024-04-20T10:22:51.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.822+0000 7fe50ee00640 1 -- 172.21.15.121:0/4245619427 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe50815a1f0 msgr2=0x7fe508071fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:51.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.822+0000 7fe50ee00640 1 --2- 172.21.15.121:0/4245619427 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe50815a1f0 0x7fe508071fc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe50815cbc0 tx=0x7fe4f402b980 comp rx=0 tx=0).stop 2024-04-20T10:22:51.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.822+0000 7fe50ee00640 1 -- 172.21.15.121:0/4245619427 shutdown_connections 2024-04-20T10:22:51.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.822+0000 7fe50ee00640 1 --2- 172.21.15.121:0/4245619427 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe50815a1f0 0x7fe508071fc0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:51.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.822+0000 7fe50ee00640 1 --2- 172.21.15.121:0/4245619427 >> v2:172.21.15.123:3300/0 conn(0x7fe508159e10 0x7fe508071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:51.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.822+0000 7fe50ee00640 1 -- 172.21.15.121:0/4245619427 >> 172.21.15.121:0/4245619427 conn(0x7fe5080d0040 msgr2=0x7fe5080d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:51.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.822+0000 7fe50ee00640 1 -- 172.21.15.121:0/4245619427 shutdown_connections 2024-04-20T10:22:51.822 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.822+0000 7fe50ee00640 1 -- 172.21.15.121:0/4245619427 wait complete. 2024-04-20T10:22:51.823 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.824+0000 7fe50ee00640 1 Processor -- start 2024-04-20T10:22:51.823 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.824+0000 7fe50ee00640 1 -- start start 2024-04-20T10:22:51.825 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.825+0000 7fe50ee00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe50806da90 0x7fe50815c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:51.825 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.825+0000 7fe50ee00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe508159e10 0x7fe5081584d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:51.825 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.825+0000 7fe50ee00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe50815a1f0 0x7fe508149530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:51.825 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.825+0000 7fe50ee00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe50815d810 con 0x7fe508159e10 2024-04-20T10:22:51.825 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.825+0000 7fe50ee00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe50815d530 con 0x7fe50815a1f0 2024-04-20T10:22:51.825 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.825+0000 7fe50ee00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe50806b900 con 0x7fe50806da90 2024-04-20T10:22:51.825 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.825+0000 7fe50d000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe508159e10 0x7fe5081584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:51.825 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.825+0000 7fe50d000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe508159e10 0x7fe5081584d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:48064/0 (socket says 172.21.15.121:48064) 2024-04-20T10:22:51.825 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.825+0000 7fe50d000640 1 -- 172.21.15.121:0/2171073282 learned_addr learned my addr 172.21.15.121:0/2171073282 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:51.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.826+0000 7fe50da00640 1 --2- 172.21.15.121:0/2171073282 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe50806da90 0x7fe50815c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:51.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.826+0000 7fe50e400640 1 --2- 172.21.15.121:0/2171073282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe50815a1f0 0x7fe508149530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:51.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.826+0000 7fe50d000640 1 -- 172.21.15.121:0/2171073282 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe50806da90 msgr2=0x7fe50815c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:51.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.826+0000 7fe50d000640 1 --2- 172.21.15.121:0/2171073282 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe50806da90 0x7fe50815c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:51.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.826+0000 7fe50d000640 1 -- 172.21.15.121:0/2171073282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe50815a1f0 msgr2=0x7fe508149530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:51.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.826+0000 7fe50d000640 1 --2- 172.21.15.121:0/2171073282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe50815a1f0 0x7fe508149530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:51.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.826+0000 7fe50d000640 1 -- 172.21.15.121:0/2171073282 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5080b35e0 con 0x7fe508159e10 2024-04-20T10:22:51.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.826+0000 7fe50d000640 1 --2- 172.21.15.121:0/2171073282 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe508159e10 0x7fe5081584d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe4f402b9c0 tx=0x7fe4f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:51.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.827+0000 7fe505600640 1 -- 172.21.15.121:0/2171073282 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4034040 con 0x7fe508159e10 2024-04-20T10:22:51.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.827+0000 7fe505600640 1 -- 172.21.15.121:0/2171073282 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4f403ad20 con 0x7fe508159e10 2024-04-20T10:22:51.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.827+0000 7fe505600640 1 -- 172.21.15.121:0/2171073282 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4044610 con 0x7fe508159e10 2024-04-20T10:22:51.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.827+0000 7fe50ee00640 1 -- 172.21.15.121:0/2171073282 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5080b38c0 con 0x7fe508159e10 2024-04-20T10:22:51.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.827+0000 7fe50ee00640 1 -- 172.21.15.121:0/2171073282 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5080b3e80 con 0x7fe508159e10 2024-04-20T10:22:51.828 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.828+0000 7fe50ee00640 1 -- 172.21.15.121:0/2171073282 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe508075b60 con 0x7fe508159e10 2024-04-20T10:22:51.828 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.829+0000 7fe505600640 1 -- 172.21.15.121:0/2171073282 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe4f404b060 con 0x7fe508159e10 2024-04-20T10:22:51.829 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.830+0000 7fe505600640 1 --2- 172.21.15.121:0/2171073282 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe4e806c8f0 0x7fe4e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:51.829 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.830+0000 7fe505600640 1 -- 172.21.15.121:0/2171073282 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7fe4f40add50 con 0x7fe508159e10 2024-04-20T10:22:51.829 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.830+0000 7fe50da00640 1 --2- 172.21.15.121:0/2171073282 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe4e806c8f0 0x7fe4e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:51.830 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.831+0000 7fe50da00640 1 --2- 172.21.15.121:0/2171073282 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe4e806c8f0 0x7fe4e806edb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc009ad0 tx=0x7fe4fc008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:51.833 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:51.834+0000 7fe505600640 1 -- 172.21.15.121:0/2171073282 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe4f403f070 con 0x7fe508159e10 2024-04-20T10:22:52.036 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.037+0000 7fe50ee00640 1 -- 172.21.15.121:0/2171073282 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe508073720 con 0x7fe508159e10 2024-04-20T10:22:52.038 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.039+0000 7fe505600640 1 -- 172.21.15.121:0/2171073282 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+16991 (secure 0 0 0) 0x7fe4f4080e20 con 0x7fe508159e10 2024-04-20T10:22:52.039 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:22:52.039 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":38,"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","created":"2024-04-20T10:20:38.076076+0000","modified":"2024-04-20T10:22:50.639532+0000","last_up_change":"2024-04-20T10:20:50.162700+0000","last_in_change":"2024-04-20T10:20:41.510382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:20:51.141365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:22:48.893174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-20T10:22:49.616949+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"629a7b8c-a40a-4a2a-a3cc-d9f357774310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6809","nonce":1044588835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6811","nonce":1044588835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6815","nonce":1044588835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6813","nonce":1044588835}]},"public_addr":"172.21.15.121:6809/1044588835","cluster_addr":"172.21.15.121:6811/1044588835","heartbeat_back_addr":"172.21.15.121:6815/1044588835","heartbeat_front_addr":"172.21.15.121:6813/1044588835","state":["exists","up"]},{"osd":1,"uuid":"f2570399-aa50-4ff3-bf5d-3d09207290db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6801","nonce":3505199624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6803","nonce":3505199624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6807","nonce":3505199624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6805","nonce":3505199624}]},"public_addr":"172.21.15.121:6801/3505199624","cluster_addr":"172.21.15.121:6803/3505199624","heartbeat_back_addr":"172.21.15.121:6807/3505199624","heartbeat_front_addr":"172.21.15.121:6805/3505199624","state":["exists","up"]},{"osd":2,"uuid":"50893807-dc30-49f4-b41f-739366626658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6817","nonce":3656724207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6819","nonce":3656724207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6827","nonce":3656724207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6822","nonce":3656724207}]},"public_addr":"172.21.15.121:6817/3656724207","cluster_addr":"172.21.15.121:6819/3656724207","heartbeat_back_addr":"172.21.15.121:6827/3656724207","heartbeat_front_addr":"172.21.15.121:6822/3656724207","state":["exists","up"]},{"osd":3,"uuid":"dadb01bc-5491-4ffc-8824-d48a057240b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6823","nonce":2232342832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6826","nonce":2232342832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6831","nonce":2232342832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6829","nonce":2232342832}]},"public_addr":"172.21.15.121:6823/2232342832","cluster_addr":"172.21.15.121:6826/2232342832","heartbeat_back_addr":"172.21.15.121:6831/2232342832","heartbeat_front_addr":"172.21.15.121:6829/2232342832","state":["exists","up"]},{"osd":4,"uuid":"b3f3b4a1-b375-4526-b69e-344ab24c2da1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6801","nonce":99353111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6803","nonce":99353111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6807","nonce":99353111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6805","nonce":99353111}]},"public_addr":"172.21.15.123:6801/99353111","cluster_addr":"172.21.15.123:6803/99353111","heartbeat_back_addr":"172.21.15.123:6807/99353111","heartbeat_front_addr":"172.21.15.123:6805/99353111","state":["exists","up"]},{"osd":5,"uuid":"2ee24a08-1624-4961-9642-d7b510282484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6818","nonce":2504374688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6822","nonce":2504374688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6830","nonce":2504374688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6826","nonce":2504374688}]},"public_addr":"172.21.15.123:6818/2504374688","cluster_addr":"172.21.15.123:6822/2504374688","heartbeat_back_addr":"172.21.15.123:6830/2504374688","heartbeat_front_addr":"172.21.15.123:6826/2504374688","state":["exists","up"]},{"osd":6,"uuid":"186118cc-6128-49de-acca-55260945c1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6819","nonce":4139299599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6823","nonce":4139299599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6829","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6831","nonce":4139299599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6827","nonce":4139299599}]},"public_addr":"172.21.15.123:6819/4139299599","cluster_addr":"172.21.15.123:6823/4139299599","heartbeat_back_addr":"172.21.15.123:6831/4139299599","heartbeat_front_addr":"172.21.15.123:6827/4139299599","state":["exists","up"]},{"osd":7,"uuid":"a00aac6e-4bd9-415a-a93d-ae85210865ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6809","nonce":4043981069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6811","nonce":4043981069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6815","nonce":4043981069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6813","nonce":4043981069}]},"public_addr":"172.21.15.123:6809/4043981069","cluster_addr":"172.21.15.123:6811/4043981069","heartbeat_back_addr":"172.21.15.123:6815/4043981069","heartbeat_front_addr":"172.21.15.123:6813/4043981069","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.904540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.958585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.616367+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.885244+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.103127+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.167248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.884348+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.862356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:22:52.042 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.042+0000 7fe50ee00640 1 -- 172.21.15.121:0/2171073282 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe4e806c8f0 msgr2=0x7fe4e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:52.042 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.042+0000 7fe50ee00640 1 --2- 172.21.15.121:0/2171073282 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe4e806c8f0 0x7fe4e806edb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc009ad0 tx=0x7fe4fc008040 comp rx=0 tx=0).stop 2024-04-20T10:22:52.042 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.042+0000 7fe50ee00640 1 -- 172.21.15.121:0/2171073282 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe508159e10 msgr2=0x7fe5081584d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:52.042 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.042+0000 7fe50ee00640 1 --2- 172.21.15.121:0/2171073282 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe508159e10 0x7fe5081584d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fe4f402b9c0 tx=0x7fe4f40046c0 comp rx=0 tx=0).stop 2024-04-20T10:22:52.042 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.042+0000 7fe50ee00640 1 -- 172.21.15.121:0/2171073282 shutdown_connections 2024-04-20T10:22:52.042 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.042+0000 7fe50ee00640 1 --2- 172.21.15.121:0/2171073282 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe4e806c8f0 0x7fe4e806edb0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:52.042 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.042+0000 7fe50ee00640 1 --2- 172.21.15.121:0/2171073282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe50815a1f0 0x7fe508149530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:52.042 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.042+0000 7fe50ee00640 1 --2- 172.21.15.121:0/2171073282 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe508159e10 0x7fe5081584d0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:52.042 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.042+0000 7fe50ee00640 1 --2- 172.21.15.121:0/2171073282 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe50806da90 0x7fe50815c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:52.043 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.042+0000 7fe50ee00640 1 -- 172.21.15.121:0/2171073282 >> 172.21.15.121:0/2171073282 conn(0x7fe5080d0040 msgr2=0x7fe5080676c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:52.043 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.042+0000 7fe50ee00640 1 -- 172.21.15.121:0/2171073282 shutdown_connections 2024-04-20T10:22:52.043 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.042+0000 7fe50ee00640 1 -- 172.21.15.121:0/2171073282 wait complete. 2024-04-20T10:22:52.060 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-20T10:22:52.254 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.254+0000 7f181fe00640 1 Processor -- start 2024-04-20T10:22:52.254 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.255+0000 7f181fe00640 1 -- start start 2024-04-20T10:22:52.255 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.256+0000 7f181fe00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f18201607d0 0x7f1820160bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:52.256 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.256+0000 7f181fe00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f18200cd040 0x7f18200b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:52.256 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.256+0000 7f181fe00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18200d0380 con 0x7f18200b3d60 2024-04-20T10:22:52.256 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.256+0000 7f181fe00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f18200b4700 con 0x7f18200cd040 2024-04-20T10:22:52.256 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.256+0000 7f181fe00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f18200704c0 con 0x7f18201607d0 2024-04-20T10:22:52.256 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.257+0000 7f181f400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f18200cd040 0x7f18200b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:52.256 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.257+0000 7f181f400640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f18200cd040 0x7f18200b41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.121:55446/0 (socket says 172.21.15.121:55446) 2024-04-20T10:22:52.256 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.257+0000 7f181f400640 1 -- 172.21.15.121:0/2273624858 learned_addr learned my addr 172.21.15.121:0/2273624858 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:52.257 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.257+0000 7f181ea00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f18201607d0 0x7f1820160bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:52.257 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.258+0000 7f181f400640 1 -- 172.21.15.121:0/2273624858 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f18201607d0 msgr2=0x7f1820160bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:52.257 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.258+0000 7f181f400640 1 --2- 172.21.15.121:0/2273624858 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f18201607d0 0x7f1820160bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:52.257 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.258+0000 7f181f400640 1 -- 172.21.15.121:0/2273624858 >> v1:172.21.15.123:6789/0 conn(0x7f18200b3d60 legacy=0x7f18200cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:52.257 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.258+0000 7f181f400640 1 -- 172.21.15.121:0/2273624858 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18200d0660 con 0x7f18200cd040 2024-04-20T10:22:52.259 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.259+0000 7f181f400640 1 --2- 172.21.15.121:0/2273624858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18200cd040 0x7f18200b41c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1818009e50 tx=0x7f181802bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57deb61c5d929a78 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:52.259 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.260+0000 7f181e000640 1 -- 172.21.15.121:0/2273624858 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181802cb00 con 0x7f18200cd040 2024-04-20T10:22:52.259 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.260+0000 7f181e000640 1 -- 172.21.15.121:0/2273624858 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f181802cca0 con 0x7f18200cd040 2024-04-20T10:22:52.259 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.260+0000 7f181e000640 1 -- 172.21.15.121:0/2273624858 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1818034610 con 0x7f18200cd040 2024-04-20T10:22:52.260 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.260+0000 7f181fe00640 1 -- 172.21.15.121:0/2273624858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18200cd040 msgr2=0x7f18200b41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:52.260 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.260+0000 7f181fe00640 1 --2- 172.21.15.121:0/2273624858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18200cd040 0x7f18200b41c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1818009e50 tx=0x7f181802bf60 comp rx=0 tx=0).stop 2024-04-20T10:22:52.260 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.260+0000 7f181fe00640 1 -- 172.21.15.121:0/2273624858 shutdown_connections 2024-04-20T10:22:52.260 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.260+0000 7f181fe00640 1 --2- 172.21.15.121:0/2273624858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18200cd040 0x7f18200b41c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:52.260 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.260+0000 7f181fe00640 1 --2- 172.21.15.121:0/2273624858 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f18201607d0 0x7f1820160bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:52.260 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.260+0000 7f181fe00640 1 -- 172.21.15.121:0/2273624858 >> 172.21.15.121:0/2273624858 conn(0x7f18200ced60 msgr2=0x7f18200cf170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:52.260 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.261+0000 7f181fe00640 1 -- 172.21.15.121:0/2273624858 shutdown_connections 2024-04-20T10:22:52.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.261+0000 7f181fe00640 1 -- 172.21.15.121:0/2273624858 wait complete. 2024-04-20T10:22:52.261 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.262+0000 7f181fe00640 1 Processor -- start 2024-04-20T10:22:52.262 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.262+0000 7f181fe00640 1 -- start start 2024-04-20T10:22:52.263 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.263+0000 7f181fe00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f18200b3d60 0x7f1820159220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:52.264 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.263+0000 7f181fe00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18200cd040 0x7f182006e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:52.264 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.263+0000 7f181fe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18201607d0 0x7f18200b76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:52.264 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.263+0000 7f181fe00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f18200d1140 con 0x7f18201607d0 2024-04-20T10:22:52.264 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.263+0000 7f181fe00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f18200d0c20 con 0x7f18200cd040 2024-04-20T10:22:52.264 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.263+0000 7f181fe00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f18200c7d00 con 0x7f18200b3d60 2024-04-20T10:22:52.264 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.263+0000 7f181f400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18201607d0 0x7f18200b76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:52.264 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.264+0000 7f181f400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18201607d0 0x7f18200b76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:48070/0 (socket says 172.21.15.121:48070) 2024-04-20T10:22:52.264 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.264+0000 7f181f400640 1 -- 172.21.15.121:0/146175192 learned_addr learned my addr 172.21.15.121:0/146175192 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:52.264 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.264+0000 7f1817e00640 1 --2- 172.21.15.121:0/146175192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18200cd040 0x7f182006e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:52.264 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.264+0000 7f181ea00640 1 --2- 172.21.15.121:0/146175192 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f18200b3d60 0x7f1820159220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:52.264 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.264+0000 7f181f400640 1 -- 172.21.15.121:0/146175192 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f18200b3d60 msgr2=0x7f1820159220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:52.264 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.264+0000 7f181f400640 1 --2- 172.21.15.121:0/146175192 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f18200b3d60 0x7f1820159220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:52.264 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.264+0000 7f181f400640 1 -- 172.21.15.121:0/146175192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18200cd040 msgr2=0x7f182006e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:52.264 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.264+0000 7f181f400640 1 --2- 172.21.15.121:0/146175192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18200cd040 0x7f182006e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:52.264 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.264+0000 7f181f400640 1 -- 172.21.15.121:0/146175192 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18200c7fe0 con 0x7f18201607d0 2024-04-20T10:22:52.265 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.264+0000 7f181ea00640 1 --2- 172.21.15.121:0/146175192 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f18200b3d60 0x7f1820159220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:52.265 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.264+0000 7f181f400640 1 --2- 172.21.15.121:0/146175192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18201607d0 0x7f18200b76d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f1818036040 tx=0x7f1818004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:52.265 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.265+0000 7f1816a00640 1 -- 172.21.15.121:0/146175192 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1818043020 con 0x7f18201607d0 2024-04-20T10:22:52.265 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.265+0000 7f1816a00640 1 -- 172.21.15.121:0/146175192 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f181802ce50 con 0x7f18201607d0 2024-04-20T10:22:52.266 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.265+0000 7f1816a00640 1 -- 172.21.15.121:0/146175192 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f181803e690 con 0x7f18201607d0 2024-04-20T10:22:52.266 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.265+0000 7f181fe00640 1 -- 172.21.15.121:0/146175192 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18200c82c0 con 0x7f18201607d0 2024-04-20T10:22:52.266 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.265+0000 7f181fe00640 1 -- 172.21.15.121:0/146175192 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1820159760 con 0x7f18201607d0 2024-04-20T10:22:52.266 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.266+0000 7f181fe00640 1 -- 172.21.15.121:0/146175192 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18200c87e0 con 0x7f18201607d0 2024-04-20T10:22:52.266 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.267+0000 7f1816a00640 1 -- 172.21.15.121:0/146175192 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1818046020 con 0x7f18201607d0 2024-04-20T10:22:52.267 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.268+0000 7f1816a00640 1 --2- 172.21.15.121:0/146175192 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f17fc06c920 0x7f17fc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:52.267 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.268+0000 7f1816a00640 1 -- 172.21.15.121:0/146175192 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f18180ae0a0 con 0x7f18201607d0 2024-04-20T10:22:52.268 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.268+0000 7f181ea00640 1 --2- 172.21.15.121:0/146175192 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f17fc06c920 0x7f17fc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:52.268 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.269+0000 7f181ea00640 1 --2- 172.21.15.121:0/146175192 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f17fc06c920 0x7f17fc06ede0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1810009ac0 tx=0x7f1810008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:52.272 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.272+0000 7f1816a00640 1 -- 172.21.15.121:0/146175192 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f18200c87e0 con 0x7f18201607d0 2024-04-20T10:22:52.502 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:52.501+0000 7f181fe00640 1 -- 172.21.15.121:0/146175192 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f1820075b30 con 0x7f18201607d0 2024-04-20T10:22:53.204 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.204+0000 7f1816a00640 1 -- 172.21.15.121:0/146175192 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f181804d030 con 0x7f18201607d0 2024-04-20T10:22:53.207 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.208+0000 7f181fe00640 1 -- 172.21.15.121:0/146175192 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f17fc06c920 msgr2=0x7f17fc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:53.208 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.208+0000 7f181fe00640 1 --2- 172.21.15.121:0/146175192 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f17fc06c920 0x7f17fc06ede0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1810009ac0 tx=0x7f1810008040 comp rx=0 tx=0).stop 2024-04-20T10:22:53.208 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.208+0000 7f181fe00640 1 -- 172.21.15.121:0/146175192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18201607d0 msgr2=0x7f18200b76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:53.208 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.208+0000 7f181fe00640 1 --2- 172.21.15.121:0/146175192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18201607d0 0x7f18200b76d0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f1818036040 tx=0x7f1818004a40 comp rx=0 tx=0).stop 2024-04-20T10:22:53.208 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.208+0000 7f181fe00640 1 -- 172.21.15.121:0/146175192 shutdown_connections 2024-04-20T10:22:53.208 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.208+0000 7f181fe00640 1 --2- 172.21.15.121:0/146175192 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f18201607d0 0x7f18200b76d0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:53.209 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.208+0000 7f181fe00640 1 --2- 172.21.15.121:0/146175192 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f17fc06c920 0x7f17fc06ede0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:53.209 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.208+0000 7f181fe00640 1 --2- 172.21.15.121:0/146175192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18200cd040 0x7f182006e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:53.209 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.208+0000 7f181fe00640 1 --2- 172.21.15.121:0/146175192 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f18200b3d60 0x7f1820159220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:53.209 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.208+0000 7f181fe00640 1 -- 172.21.15.121:0/146175192 >> 172.21.15.121:0/146175192 conn(0x7f18200ced60 msgr2=0x7f18200b35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:53.209 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.209+0000 7f181fe00640 1 -- 172.21.15.121:0/146175192 shutdown_connections 2024-04-20T10:22:53.209 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.209+0000 7f181fe00640 1 -- 172.21.15.121:0/146175192 wait complete. 2024-04-20T10:22:53.231 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-20T10:22:53.231 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-20T10:22:53.434 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.434+0000 7f7752200640 1 Processor -- start 2024-04-20T10:22:53.434 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.435+0000 7f7752200640 1 -- start start 2024-04-20T10:22:53.436 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.436+0000 7f7752200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f774c0d1a00 0x7f774c0b1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:53.436 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.436+0000 7f7752200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f774c0b17d0 0x7f774c0d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:53.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.436+0000 7f7752200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f774c0d1e60 con 0x7f774c0d4270 2024-04-20T10:22:53.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.436+0000 7f7752200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f774c0d40c0 con 0x7f774c0b17d0 2024-04-20T10:22:53.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.436+0000 7f7752200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f774c0d3820 con 0x7f774c0d1a00 2024-04-20T10:22:53.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.437+0000 7f774be00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f774c0b17d0 0x7f774c0d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:53.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.437+0000 7f7743e00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f774c0d1a00 0x7f774c0b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:53.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.437+0000 7f774be00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f774c0b17d0 0x7f774c0d0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.121:55466/0 (socket says 172.21.15.121:55466) 2024-04-20T10:22:53.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.437+0000 7f774be00640 1 -- 172.21.15.121:0/2137545640 learned_addr learned my addr 172.21.15.121:0/2137545640 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:53.437 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.438+0000 7f774be00640 1 -- 172.21.15.121:0/2137545640 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f774c0d1a00 msgr2=0x7f774c0b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:53.438 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.438+0000 7f774be00640 1 --2- 172.21.15.121:0/2137545640 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f774c0d1a00 0x7f774c0b1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:53.438 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.438+0000 7f774be00640 1 -- 172.21.15.121:0/2137545640 >> v1:172.21.15.123:6789/0 conn(0x7f774c0d4270 legacy=0x7f774c0d4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:53.438 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.438+0000 7f774be00640 1 -- 172.21.15.121:0/2137545640 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f774c0d3b00 con 0x7f774c0b17d0 2024-04-20T10:22:53.439 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.439+0000 7f774be00640 1 --2- 172.21.15.121:0/2137545640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f774c0b17d0 0x7f774c0d0380 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7744009e50 tx=0x7f774402bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33b48956d477f07c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:53.439 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.440+0000 7f774aa00640 1 -- 172.21.15.121:0/2137545640 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774402cb00 con 0x7f774c0b17d0 2024-04-20T10:22:53.439 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.440+0000 7f774aa00640 1 -- 172.21.15.121:0/2137545640 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f774402cca0 con 0x7f774c0b17d0 2024-04-20T10:22:53.439 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.440+0000 7f774aa00640 1 -- 172.21.15.121:0/2137545640 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7744034610 con 0x7f774c0b17d0 2024-04-20T10:22:53.440 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.441+0000 7f7752200640 1 -- 172.21.15.121:0/2137545640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f774c0b17d0 msgr2=0x7f774c0d0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:53.440 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.441+0000 7f7752200640 1 --2- 172.21.15.121:0/2137545640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f774c0b17d0 0x7f774c0d0380 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7744009e50 tx=0x7f774402bf60 comp rx=0 tx=0).stop 2024-04-20T10:22:53.440 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.441+0000 7f7752200640 1 -- 172.21.15.121:0/2137545640 shutdown_connections 2024-04-20T10:22:53.440 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.441+0000 7f7752200640 1 --2- 172.21.15.121:0/2137545640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f774c0b17d0 0x7f774c0d0380 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:53.441 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.441+0000 7f7752200640 1 --2- 172.21.15.121:0/2137545640 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f774c0d1a00 0x7f774c0b1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:53.441 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.441+0000 7f7752200640 1 -- 172.21.15.121:0/2137545640 >> 172.21.15.121:0/2137545640 conn(0x7f774c0b99c0 msgr2=0x7f774c0b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:53.441 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.441+0000 7f7752200640 1 -- 172.21.15.121:0/2137545640 shutdown_connections 2024-04-20T10:22:53.441 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.441+0000 7f7752200640 1 -- 172.21.15.121:0/2137545640 wait complete. 2024-04-20T10:22:53.442 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.443+0000 7f7752200640 1 Processor -- start 2024-04-20T10:22:53.442 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.443+0000 7f7752200640 1 -- start start 2024-04-20T10:22:53.443 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.444+0000 7f7752200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f774c0b17d0 0x7f774c1f3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:53.443 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.444+0000 7f7752200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f774c0d1a00 0x7f774c0c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:53.443 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.444+0000 7f7752200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f774c0d4270 0x7f774c0b5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:53.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.444+0000 7f7752200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f774c0d53b0 con 0x7f774c0d1a00 2024-04-20T10:22:53.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.444+0000 7f7752200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f774c0cfe60 con 0x7f774c0b17d0 2024-04-20T10:22:53.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.444+0000 7f7752200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f774c161f70 con 0x7f774c0d4270 2024-04-20T10:22:53.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.444+0000 7f7743e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f774c0d1a00 0x7f774c0c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:53.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.444+0000 7f7743e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f774c0d1a00 0x7f774c0c9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:48086/0 (socket says 172.21.15.121:48086) 2024-04-20T10:22:53.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.444+0000 7f7743e00640 1 -- 172.21.15.121:0/993716454 learned_addr learned my addr 172.21.15.121:0/993716454 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:53.445 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.444+0000 7f774b400640 1 --2- 172.21.15.121:0/993716454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f774c0b17d0 0x7f774c1f3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:53.446 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.444+0000 7f774be00640 1 --2- 172.21.15.121:0/993716454 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f774c0d4270 0x7f774c0b5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:53.446 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.444+0000 7f7743e00640 1 -- 172.21.15.121:0/993716454 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f774c0d4270 msgr2=0x7f774c0b5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:53.446 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.444+0000 7f7743e00640 1 --2- 172.21.15.121:0/993716454 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f774c0d4270 0x7f774c0b5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:53.446 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.445+0000 7f7743e00640 1 -- 172.21.15.121:0/993716454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f774c0b17d0 msgr2=0x7f774c1f3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:53.446 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.445+0000 7f7743e00640 1 --2- 172.21.15.121:0/993716454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f774c0b17d0 0x7f774c1f3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:53.446 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.445+0000 7f7743e00640 1 -- 172.21.15.121:0/993716454 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f774c0b40e0 con 0x7f774c0d1a00 2024-04-20T10:22:53.446 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.445+0000 7f774be00640 1 --2- 172.21.15.121:0/993716454 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f774c0d4270 0x7f774c0b5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:53.446 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.445+0000 7f7743e00640 1 --2- 172.21.15.121:0/993716454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f774c0d1a00 0x7f774c0c9c30 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f773400e9e0 tx=0x7f773400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:53.446 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.445+0000 7f7743400640 1 -- 172.21.15.121:0/993716454 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7734017070 con 0x7f774c0d1a00 2024-04-20T10:22:53.446 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.445+0000 7f7743400640 1 -- 172.21.15.121:0/993716454 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f773400c9d0 con 0x7f774c0d1a00 2024-04-20T10:22:53.446 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.445+0000 7f7743400640 1 -- 172.21.15.121:0/993716454 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f773400cdc0 con 0x7f774c0d1a00 2024-04-20T10:22:53.446 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.445+0000 7f7752200640 1 -- 172.21.15.121:0/993716454 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f774c0b4380 con 0x7f774c0d1a00 2024-04-20T10:22:53.446 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.445+0000 7f7752200640 1 -- 172.21.15.121:0/993716454 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f774c0b4940 con 0x7f774c0d1a00 2024-04-20T10:22:53.447 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.447+0000 7f7752200640 1 -- 172.21.15.121:0/993716454 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f774c068980 con 0x7f774c0d1a00 2024-04-20T10:22:53.447 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.448+0000 7f7743400640 1 -- 172.21.15.121:0/993716454 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f773401a070 con 0x7f774c0d1a00 2024-04-20T10:22:53.448 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.449+0000 7f7743400640 1 --2- 172.21.15.121:0/993716454 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f772806c8f0 0x7f772806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:53.448 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.449+0000 7f7743400640 1 -- 172.21.15.121:0/993716454 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f773408bcf0 con 0x7f774c0d1a00 2024-04-20T10:22:53.448 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.449+0000 7f774b400640 1 --2- 172.21.15.121:0/993716454 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f772806c8f0 0x7f772806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:53.449 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.449+0000 7f774b400640 1 --2- 172.21.15.121:0/993716454 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f772806c8f0 0x7f772806edb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f773c001640 tx=0x7f773c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:53.452 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.453+0000 7f7743400640 1 -- 172.21.15.121:0/993716454 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7734059f80 con 0x7f774c0d1a00 2024-04-20T10:22:53.696 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:53.696+0000 7f7752200640 1 -- 172.21.15.121:0/993716454 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f774c1f3d60 con 0x7f774c0d1a00 2024-04-20T10:22:54.217 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.217+0000 7f7743400640 1 -- 172.21.15.121:0/993716454 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7f7734059880 con 0x7f774c0d1a00 2024-04-20T10:22:54.221 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.221+0000 7f7752200640 1 -- 172.21.15.121:0/993716454 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f772806c8f0 msgr2=0x7f772806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:54.221 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.221+0000 7f7752200640 1 --2- 172.21.15.121:0/993716454 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f772806c8f0 0x7f772806edb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f773c001640 tx=0x7f773c009040 comp rx=0 tx=0).stop 2024-04-20T10:22:54.221 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.221+0000 7f7752200640 1 -- 172.21.15.121:0/993716454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f774c0d1a00 msgr2=0x7f774c0c9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:54.221 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.221+0000 7f7752200640 1 --2- 172.21.15.121:0/993716454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f774c0d1a00 0x7f774c0c9c30 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f773400e9e0 tx=0x7f773400eea0 comp rx=0 tx=0).stop 2024-04-20T10:22:54.221 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.221+0000 7f7752200640 1 -- 172.21.15.121:0/993716454 shutdown_connections 2024-04-20T10:22:54.221 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.221+0000 7f7752200640 1 --2- 172.21.15.121:0/993716454 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f774c0d4270 0x7f774c0b5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:54.221 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.221+0000 7f7752200640 1 --2- 172.21.15.121:0/993716454 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f774c0d1a00 0x7f774c0c9c30 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:54.221 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.221+0000 7f7752200640 1 --2- 172.21.15.121:0/993716454 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f772806c8f0 0x7f772806edb0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:54.221 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.221+0000 7f7752200640 1 --2- 172.21.15.121:0/993716454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f774c0b17d0 0x7f774c1f3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:54.221 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.221+0000 7f7752200640 1 -- 172.21.15.121:0/993716454 >> 172.21.15.121:0/993716454 conn(0x7f774c0b99c0 msgr2=0x7f774c0d01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:54.221 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.221+0000 7f7752200640 1 -- 172.21.15.121:0/993716454 shutdown_connections 2024-04-20T10:22:54.221 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.221+0000 7f7752200640 1 -- 172.21.15.121:0/993716454 wait complete. 2024-04-20T10:22:54.244 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-20T10:22:54.244 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:22:54.438 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.439+0000 7fdee7200640 1 Processor -- start 2024-04-20T10:22:54.439 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.439+0000 7fdee7200640 1 -- start start 2024-04-20T10:22:54.441 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.440+0000 7fdee7200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdee815a1f0 0x7fdee8072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:54.441 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.441+0000 7fdee7200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdee8072190 0x7fdee8073420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:54.441 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.441+0000 7fdee7200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdee815cc90 con 0x7fdee8159e10 2024-04-20T10:22:54.441 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.441+0000 7fdee7200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fdee8072900 con 0x7fdee815a1f0 2024-04-20T10:22:54.441 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.441+0000 7fdee7200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fdee815ca40 con 0x7fdee8072190 2024-04-20T10:22:54.441 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.441+0000 7fdedfe00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdee815a1f0 0x7fdee8072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:54.441 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.441+0000 7fdedfe00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdee815a1f0 0x7fdee8072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:48088/0 (socket says 172.21.15.121:48088) 2024-04-20T10:22:54.441 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.441+0000 7fdedfe00640 1 -- 172.21.15.121:0/3568298978 learned_addr learned my addr 172.21.15.121:0/3568298978 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:54.442 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.441+0000 7fdee6800640 1 --2- 172.21.15.121:0/3568298978 >> v2:172.21.15.123:3300/0 conn(0x7fdee8072190 0x7fdee8073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:54.442 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.441+0000 7fdedfe00640 1 -- 172.21.15.121:0/3568298978 >> v2:172.21.15.123:3300/0 conn(0x7fdee8072190 msgr2=0x7fdee8073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:54.442 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.441+0000 7fdedfe00640 1 --2- 172.21.15.121:0/3568298978 >> v2:172.21.15.123:3300/0 conn(0x7fdee8072190 0x7fdee8073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:54.442 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.441+0000 7fdedfe00640 1 -- 172.21.15.121:0/3568298978 >> v1:172.21.15.123:6789/0 conn(0x7fdee8159e10 legacy=0x7fdee8071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:54.442 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.442+0000 7fdedfe00640 1 -- 172.21.15.121:0/3568298978 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee815cf70 con 0x7fdee815a1f0 2024-04-20T10:22:54.442 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.443+0000 7fdedfe00640 1 --2- 172.21.15.121:0/3568298978 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdee815a1f0 0x7fdee8072ee0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fdee815daf0 tx=0x7fded002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba13b4b3df386ca server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:54.443 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.443+0000 7fdee5400640 1 -- 172.21.15.121:0/3568298978 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded0035020 con 0x7fdee815a1f0 2024-04-20T10:22:54.443 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.443+0000 7fdee5400640 1 -- 172.21.15.121:0/3568298978 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fded0031870 con 0x7fdee815a1f0 2024-04-20T10:22:54.443 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.443+0000 7fdee5400640 1 -- 172.21.15.121:0/3568298978 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded0031c40 con 0x7fdee815a1f0 2024-04-20T10:22:54.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.444+0000 7fdee7200640 1 -- 172.21.15.121:0/3568298978 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdee815a1f0 msgr2=0x7fdee8072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:54.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.444+0000 7fdee7200640 1 --2- 172.21.15.121:0/3568298978 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdee815a1f0 0x7fdee8072ee0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fdee815daf0 tx=0x7fded002b910 comp rx=0 tx=0).stop 2024-04-20T10:22:54.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.444+0000 7fdee7200640 1 -- 172.21.15.121:0/3568298978 shutdown_connections 2024-04-20T10:22:54.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.444+0000 7fdee7200640 1 --2- 172.21.15.121:0/3568298978 >> v2:172.21.15.123:3300/0 conn(0x7fdee8072190 0x7fdee8073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:54.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.444+0000 7fdee7200640 1 --2- 172.21.15.121:0/3568298978 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdee815a1f0 0x7fdee8072ee0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:54.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.444+0000 7fdee7200640 1 -- 172.21.15.121:0/3568298978 >> 172.21.15.121:0/3568298978 conn(0x7fdee80d0040 msgr2=0x7fdee80d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:54.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.444+0000 7fdee7200640 1 -- 172.21.15.121:0/3568298978 shutdown_connections 2024-04-20T10:22:54.444 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.444+0000 7fdee7200640 1 -- 172.21.15.121:0/3568298978 wait complete. 2024-04-20T10:22:54.445 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.446+0000 7fdee7200640 1 Processor -- start 2024-04-20T10:22:54.446 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.446+0000 7fdee7200640 1 -- start start 2024-04-20T10:22:54.447 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.447+0000 7fdee7200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdee8072190 0x7fdee80cba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:54.447 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.447+0000 7fdee7200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdee8159e10 0x7fdee809d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:54.447 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.447+0000 7fdee7200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdee815a1f0 0x7fdee81f6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:54.447 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.447+0000 7fdee7200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdee815d810 con 0x7fdee8159e10 2024-04-20T10:22:54.447 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.447+0000 7fdee7200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdee815d530 con 0x7fdee8072190 2024-04-20T10:22:54.447 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.447+0000 7fdee7200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fdee806da90 con 0x7fdee815a1f0 2024-04-20T10:22:54.447 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.448+0000 7fdedfe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdee8159e10 0x7fdee809d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:54.447 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.448+0000 7fdedfe00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdee8159e10 0x7fdee809d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:48104/0 (socket says 172.21.15.121:48104) 2024-04-20T10:22:54.448 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.448+0000 7fdedfe00640 1 -- 172.21.15.121:0/2993630174 learned_addr learned my addr 172.21.15.121:0/2993630174 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:54.449 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.448+0000 7fdee5e00640 1 --2- 172.21.15.121:0/2993630174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdee8072190 0x7fdee80cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:54.449 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.448+0000 7fdee6800640 1 --2- 172.21.15.121:0/2993630174 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdee815a1f0 0x7fdee81f6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:54.449 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.448+0000 7fdedfe00640 1 -- 172.21.15.121:0/2993630174 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdee815a1f0 msgr2=0x7fdee81f6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:54.449 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.448+0000 7fdedfe00640 1 --2- 172.21.15.121:0/2993630174 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdee815a1f0 0x7fdee81f6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:54.449 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.448+0000 7fdedfe00640 1 -- 172.21.15.121:0/2993630174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdee8072190 msgr2=0x7fdee80cba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:54.449 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.448+0000 7fdedfe00640 1 --2- 172.21.15.121:0/2993630174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdee8072190 0x7fdee80cba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:54.449 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.448+0000 7fdedfe00640 1 -- 172.21.15.121:0/2993630174 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdee81f7070 con 0x7fdee8159e10 2024-04-20T10:22:54.449 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.448+0000 7fdee5e00640 1 --2- 172.21.15.121:0/2993630174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdee8072190 0x7fdee80cba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:54.450 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.448+0000 7fdedfe00640 1 --2- 172.21.15.121:0/2993630174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdee8159e10 0x7fdee809d2d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fded002b950 tx=0x7fded00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:54.450 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.449+0000 7fdede000640 1 -- 172.21.15.121:0/2993630174 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded0040070 con 0x7fdee8159e10 2024-04-20T10:22:54.450 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.449+0000 7fdede000640 1 -- 172.21.15.121:0/2993630174 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fded0030d00 con 0x7fdee8159e10 2024-04-20T10:22:54.450 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.449+0000 7fdede000640 1 -- 172.21.15.121:0/2993630174 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded0035040 con 0x7fdee8159e10 2024-04-20T10:22:54.450 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.449+0000 7fdee7200640 1 -- 172.21.15.121:0/2993630174 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdee81586f0 con 0x7fdee8159e10 2024-04-20T10:22:54.450 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.449+0000 7fdee7200640 1 -- 172.21.15.121:0/2993630174 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdee8158cb0 con 0x7fdee8159e10 2024-04-20T10:22:54.451 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.450+0000 7fdee7200640 1 -- 172.21.15.121:0/2993630174 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdee8075b60 con 0x7fdee8159e10 2024-04-20T10:22:54.451 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.451+0000 7fdede000640 1 -- 172.21.15.121:0/2993630174 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fded0038020 con 0x7fdee8159e10 2024-04-20T10:22:54.451 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.452+0000 7fdede000640 1 --2- 172.21.15.121:0/2993630174 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fdec406c8f0 0x7fdec406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:54.452 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.452+0000 7fdede000640 1 -- 172.21.15.121:0/2993630174 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fded00adb10 con 0x7fdee8159e10 2024-04-20T10:22:54.452 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.452+0000 7fdee5e00640 1 --2- 172.21.15.121:0/2993630174 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fdec406c8f0 0x7fdec406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:54.452 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.453+0000 7fdee5e00640 1 --2- 172.21.15.121:0/2993630174 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fdec406c8f0 0x7fdec406edb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fded8003eb0 tx=0x7fded8009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:54.456 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.457+0000 7fdede000640 1 -- 172.21.15.121:0/2993630174 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fded007bd90 con 0x7fdee8159e10 2024-04-20T10:22:54.650 INFO:tasks.ceph.mds.c.smithi121.stderr:2024-04-20T10:22:54.650+0000 7f4d98a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T10:22:54.695 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.695+0000 7fdee7200640 1 -- 172.21.15.121:0/2993630174 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdee8074dd0 con 0x7fdee8159e10 2024-04-20T10:22:54.696 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.696+0000 7fdede000640 1 -- 172.21.15.121:0/2993630174 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5202 (secure 0 0 0) 0x7fded007b690 con 0x7fdee8159e10 2024-04-20T10:22:54.696 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:22:54.696 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4715,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/788634820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":788634820},{"type":"v1","addr":"172.21.15.123:6835","nonce":788634820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4942,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/331480838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":331480838},{"type":"v1","addr":"172.21.15.123:6833","nonce":331480838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5562,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/2525740087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2525740087},{"type":"v1","addr":"172.21.15.121:6835","nonce":2525740087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:22:50.639487+0000","modified":"2024-04-20T10:22:53.697379+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5568},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5568":{"gid":5568,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.121:6837/257724942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":257724942},{"type":"v1","addr":"172.21.15.121:6837","nonce":257724942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5568,"qdb_cluster":[5568]},"id":2}]} 2024-04-20T10:22:54.697 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 17 2024-04-20T10:22:54.699 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.699+0000 7fdee7200640 1 -- 172.21.15.121:0/2993630174 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fdec406c8f0 msgr2=0x7fdec406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:54.699 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.699+0000 7fdee7200640 1 --2- 172.21.15.121:0/2993630174 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fdec406c8f0 0x7fdec406edb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fded8003eb0 tx=0x7fded8009040 comp rx=0 tx=0).stop 2024-04-20T10:22:54.699 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.699+0000 7fdee7200640 1 -- 172.21.15.121:0/2993630174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdee8159e10 msgr2=0x7fdee809d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:54.699 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.699+0000 7fdee7200640 1 --2- 172.21.15.121:0/2993630174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdee8159e10 0x7fdee809d2d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fded002b950 tx=0x7fded00046c0 comp rx=0 tx=0).stop 2024-04-20T10:22:54.699 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.699+0000 7fdee7200640 1 -- 172.21.15.121:0/2993630174 shutdown_connections 2024-04-20T10:22:54.700 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.699+0000 7fdee7200640 1 --2- 172.21.15.121:0/2993630174 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdee815a1f0 0x7fdee81f6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:54.700 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.699+0000 7fdee7200640 1 --2- 172.21.15.121:0/2993630174 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdee8159e10 0x7fdee809d2d0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:54.700 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.699+0000 7fdee7200640 1 --2- 172.21.15.121:0/2993630174 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fdec406c8f0 0x7fdec406edb0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:54.700 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.699+0000 7fdee7200640 1 --2- 172.21.15.121:0/2993630174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdee8072190 0x7fdee80cba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:54.700 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.699+0000 7fdee7200640 1 -- 172.21.15.121:0/2993630174 >> 172.21.15.121:0/2993630174 conn(0x7fdee80d0040 msgr2=0x7fdee8071a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:54.700 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.699+0000 7fdee7200640 1 -- 172.21.15.121:0/2993630174 shutdown_connections 2024-04-20T10:22:54.700 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.699+0000 7fdee7200640 1 -- 172.21.15.121:0/2993630174 wait complete. 2024-04-20T10:22:54.718 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:22:54.921 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.921+0000 7efc48200640 1 Processor -- start 2024-04-20T10:22:54.921 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.921+0000 7efc48200640 1 -- start start 2024-04-20T10:22:54.922 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.923+0000 7efc48200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc4015b3b0 0x7efc4006eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:54.922 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.923+0000 7efc48200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc400ade40 con 0x7efc4006dcb0 2024-04-20T10:22:54.922 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.923+0000 7efc48200640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc400ae0e0 con 0x7efc400d0360 2024-04-20T10:22:54.923 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.923+0000 7efc48200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7efc4015b790 con 0x7efc4015b3b0 2024-04-20T10:22:54.923 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.923+0000 7efc44e00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7efc4006dcb0 0x7efc40071b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:44856/0 (socket says 172.21.15.121:44856) 2024-04-20T10:22:54.923 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.923+0000 7efc44e00640 1 -- 172.21.15.121:0/3502897404 learned_addr learned my addr 172.21.15.121:0/3502897404 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:54.925 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.923+0000 7efc45800640 1 --2- 172.21.15.121:0/3502897404 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc4015b3b0 0x7efc4006eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:54.925 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.923+0000 7efc3fe00640 1 -- 172.21.15.121:0/3502897404 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2496633619 0 0) 0x7efc400ade40 con 0x7efc4006dcb0 2024-04-20T10:22:54.925 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.924+0000 7efc3fe00640 1 -- 172.21.15.121:0/3502897404 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efc28003520 con 0x7efc4006dcb0 2024-04-20T10:22:54.925 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.924+0000 7efc3fe00640 1 -- 172.21.15.121:0/3502897404 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2493410803 0 0) 0x7efc28003520 con 0x7efc4006dcb0 2024-04-20T10:22:54.925 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.924+0000 7efc3fe00640 1 -- 172.21.15.121:0/3502897404 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc4015b3b0 msgr2=0x7efc4006eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:54.925 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.924+0000 7efc3fe00640 1 --2- 172.21.15.121:0/3502897404 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc4015b3b0 0x7efc4006eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:54.925 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.924+0000 7efc3fe00640 1 -- 172.21.15.121:0/3502897404 >> v1:172.21.15.123:6789/0 conn(0x7efc400d0360 legacy=0x7efc400d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:54.925 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.924+0000 7efc3fe00640 1 -- 172.21.15.121:0/3502897404 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc40072260 con 0x7efc4006dcb0 2024-04-20T10:22:54.926 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.924+0000 7efc3fe00640 1 -- 172.21.15.121:0/3502897404 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7efc2c001670 con 0x7efc4006dcb0 2024-04-20T10:22:54.926 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.924+0000 7efc3fe00640 1 -- 172.21.15.121:0/3502897404 >> v1:172.21.15.121:6789/0 conn(0x7efc4006dcb0 legacy=0x7efc40071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:54.926 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.924+0000 7efc3fe00640 1 --2- 172.21.15.121:0/3502897404 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc4015b3b0 0x7efc28003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:54.926 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.924+0000 7efc3fe00640 1 --2- 172.21.15.121:0/3502897404 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc28003f80 0x7efc28004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:54.926 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.924+0000 7efc3fe00640 1 --2- 172.21.15.121:0/3502897404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc28004d40 0x7efc280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:54.926 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.924+0000 7efc3fe00640 1 -- 172.21.15.121:0/3502897404 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efc4015b790 con 0x7efc28003f80 2024-04-20T10:22:54.926 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.924+0000 7efc3fe00640 1 -- 172.21.15.121:0/3502897404 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efc2800bad0 con 0x7efc28004d40 2024-04-20T10:22:54.926 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.924+0000 7efc3fe00640 1 -- 172.21.15.121:0/3502897404 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7efc2800bcd0 con 0x7efc4015b3b0 2024-04-20T10:22:54.926 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.925+0000 7efc44e00640 1 --2- 172.21.15.121:0/3502897404 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc28003f80 0x7efc28004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:54.926 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.925+0000 7efc45800640 1 --2- 172.21.15.121:0/3502897404 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc4015b3b0 0x7efc28003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:54.926 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.925+0000 7efc46200640 1 --2- 172.21.15.121:0/3502897404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc28004d40 0x7efc280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:54.926 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.925+0000 7efc44e00640 1 -- 172.21.15.121:0/3502897404 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc4015b3b0 msgr2=0x7efc28003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:54.927 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.925+0000 7efc44e00640 1 --2- 172.21.15.121:0/3502897404 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc4015b3b0 0x7efc28003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:54.927 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.925+0000 7efc44e00640 1 -- 172.21.15.121:0/3502897404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc28004d40 msgr2=0x7efc280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:54.927 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.925+0000 7efc44e00640 1 --2- 172.21.15.121:0/3502897404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc28004d40 0x7efc280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:54.927 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.925+0000 7efc44e00640 1 -- 172.21.15.121:0/3502897404 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc28003520 con 0x7efc28003f80 2024-04-20T10:22:54.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.927+0000 7efc44e00640 1 --2- 172.21.15.121:0/3502897404 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc28003f80 0x7efc28004800 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7efc2c00c210 tx=0x7efc28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d210d20d545d085e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:54.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.927+0000 7efc3fe00640 1 -- 172.21.15.121:0/3502897404 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c007d10 con 0x7efc28003f80 2024-04-20T10:22:54.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.927+0000 7efc3fe00640 1 -- 172.21.15.121:0/3502897404 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc2c030070 con 0x7efc28003f80 2024-04-20T10:22:54.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.927+0000 7efc3fe00640 1 -- 172.21.15.121:0/3502897404 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c005d70 con 0x7efc28003f80 2024-04-20T10:22:54.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.927+0000 7efc48200640 1 -- 172.21.15.121:0/3502897404 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc28003f80 msgr2=0x7efc28004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:54.928 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.927+0000 7efc48200640 1 --2- 172.21.15.121:0/3502897404 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc28003f80 0x7efc28004800 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7efc2c00c210 tx=0x7efc28005830 comp rx=0 tx=0).stop 2024-04-20T10:22:54.929 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.928+0000 7efc46200640 1 -- 172.21.15.121:0/3502897404 reap_dead start 2024-04-20T10:22:54.929 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.928+0000 7efc48200640 1 -- 172.21.15.121:0/3502897404 shutdown_connections 2024-04-20T10:22:54.929 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.928+0000 7efc48200640 1 -- 172.21.15.121:0/3502897404 >> 172.21.15.121:0/3502897404 conn(0x7efc400b3350 msgr2=0x7efc400b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:54.929 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.928+0000 7efc48200640 1 -- 172.21.15.121:0/3502897404 shutdown_connections 2024-04-20T10:22:54.929 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.928+0000 7efc48200640 1 -- 172.21.15.121:0/3502897404 wait complete. 2024-04-20T10:22:54.929 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.929+0000 7efc48200640 1 Processor -- start 2024-04-20T10:22:54.929 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.930+0000 7efc48200640 1 -- start start 2024-04-20T10:22:54.931 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.931+0000 7efc48200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc28003f80 0x7efc400b0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:54.931 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.931+0000 7efc48200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc40152ee0 0x7efc40153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:54.931 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.931+0000 7efc48200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc401538a0 0x7efc40153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:54.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.931+0000 7efc48200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7efc400ae940 con 0x7efc40152ee0 2024-04-20T10:22:54.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.931+0000 7efc48200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efc400ae660 con 0x7efc28003f80 2024-04-20T10:22:54.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.931+0000 7efc48200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7efc400aec20 con 0x7efc401538a0 2024-04-20T10:22:54.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.931+0000 7efc44e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc40152ee0 0x7efc40153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:54.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.931+0000 7efc44e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc40152ee0 0x7efc40153360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:48118/0 (socket says 172.21.15.121:48118) 2024-04-20T10:22:54.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.931+0000 7efc44e00640 1 -- 172.21.15.121:0/2789043330 learned_addr learned my addr 172.21.15.121:0/2789043330 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:54.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.931+0000 7efc45800640 1 --2- 172.21.15.121:0/2789043330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc28003f80 0x7efc400b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:54.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.931+0000 7efc46200640 1 --2- 172.21.15.121:0/2789043330 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc401538a0 0x7efc40153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:54.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.932+0000 7efc44e00640 1 -- 172.21.15.121:0/2789043330 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc401538a0 msgr2=0x7efc40153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:54.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.932+0000 7efc44e00640 1 --2- 172.21.15.121:0/2789043330 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc401538a0 0x7efc40153d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:54.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.932+0000 7efc44e00640 1 -- 172.21.15.121:0/2789043330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc28003f80 msgr2=0x7efc400b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:54.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.932+0000 7efc44e00640 1 --2- 172.21.15.121:0/2789043330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc28003f80 0x7efc400b0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:54.932 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.932+0000 7efc44e00640 1 -- 172.21.15.121:0/2789043330 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc400b0890 con 0x7efc40152ee0 2024-04-20T10:22:54.933 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.932+0000 7efc44e00640 1 --2- 172.21.15.121:0/2789043330 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc40152ee0 0x7efc40153360 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7efc2c00c330 tx=0x7efc2c02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:54.933 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.932+0000 7efc3d600640 1 -- 172.21.15.121:0/2789043330 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c045070 con 0x7efc40152ee0 2024-04-20T10:22:54.933 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.932+0000 7efc3d600640 1 -- 172.21.15.121:0/2789043330 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc2c004bf0 con 0x7efc40152ee0 2024-04-20T10:22:54.934 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.932+0000 7efc3d600640 1 -- 172.21.15.121:0/2789043330 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c007d10 con 0x7efc40152ee0 2024-04-20T10:22:54.934 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.932+0000 7efc48200640 1 -- 172.21.15.121:0/2789043330 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc400b0b70 con 0x7efc40152ee0 2024-04-20T10:22:54.934 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.933+0000 7efc48200640 1 -- 172.21.15.121:0/2789043330 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc401529c0 con 0x7efc40152ee0 2024-04-20T10:22:54.934 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.934+0000 7efc48200640 1 -- 172.21.15.121:0/2789043330 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc40068980 con 0x7efc40152ee0 2024-04-20T10:22:54.934 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.935+0000 7efc3d600640 1 -- 172.21.15.121:0/2789043330 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efc2c007070 con 0x7efc40152ee0 2024-04-20T10:22:54.936 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.936+0000 7efc3d600640 1 --2- 172.21.15.121:0/2789043330 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7efc2006c8f0 0x7efc2006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:54.936 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.936+0000 7efc3d600640 1 -- 172.21.15.121:0/2789043330 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7efc2c0acbc0 con 0x7efc40152ee0 2024-04-20T10:22:54.936 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.937+0000 7efc45800640 1 --2- 172.21.15.121:0/2789043330 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7efc2006c8f0 0x7efc2006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:54.936 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.937+0000 7efc45800640 1 --2- 172.21.15.121:0/2789043330 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7efc2006c8f0 0x7efc2006edb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7efc34009a10 tx=0x7efc34008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:54.940 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:54.941+0000 7efc3d600640 1 -- 172.21.15.121:0/2789043330 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7efc2c07ae50 con 0x7efc40152ee0 2024-04-20T10:22:55.143 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.143+0000 7efc48200640 1 -- 172.21.15.121:0/2789043330 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc400b1090 con 0x7efc40152ee0 2024-04-20T10:22:55.144 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.145+0000 7efc3d600640 1 -- 172.21.15.121:0/2789043330 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+16978 (secure 0 0 0) 0x7efc2c07a750 con 0x7efc40152ee0 2024-04-20T10:22:55.145 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:22:55.145 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":39,"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","created":"2024-04-20T10:20:38.076076+0000","modified":"2024-04-20T10:22:52.191044+0000","last_up_change":"2024-04-20T10:20:50.162700+0000","last_in_change":"2024-04-20T10:20:41.510382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:20:51.141365+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:22:48.893174+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-20T10:22:49.616949+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"629a7b8c-a40a-4a2a-a3cc-d9f357774310","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6809","nonce":1044588835}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6811","nonce":1044588835}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6815","nonce":1044588835}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":1044588835},{"type":"v1","addr":"172.21.15.121:6813","nonce":1044588835}]},"public_addr":"172.21.15.121:6809/1044588835","cluster_addr":"172.21.15.121:6811/1044588835","heartbeat_back_addr":"172.21.15.121:6815/1044588835","heartbeat_front_addr":"172.21.15.121:6813/1044588835","state":["exists","up"]},{"osd":1,"uuid":"f2570399-aa50-4ff3-bf5d-3d09207290db","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6801","nonce":3505199624}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6803","nonce":3505199624}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6807","nonce":3505199624}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":3505199624},{"type":"v1","addr":"172.21.15.121:6805","nonce":3505199624}]},"public_addr":"172.21.15.121:6801/3505199624","cluster_addr":"172.21.15.121:6803/3505199624","heartbeat_back_addr":"172.21.15.121:6807/3505199624","heartbeat_front_addr":"172.21.15.121:6805/3505199624","state":["exists","up"]},{"osd":2,"uuid":"50893807-dc30-49f4-b41f-739366626658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6817","nonce":3656724207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6819","nonce":3656724207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6827","nonce":3656724207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":3656724207},{"type":"v1","addr":"172.21.15.121:6822","nonce":3656724207}]},"public_addr":"172.21.15.121:6817/3656724207","cluster_addr":"172.21.15.121:6819/3656724207","heartbeat_back_addr":"172.21.15.121:6827/3656724207","heartbeat_front_addr":"172.21.15.121:6822/3656724207","state":["exists","up"]},{"osd":3,"uuid":"dadb01bc-5491-4ffc-8824-d48a057240b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6821","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6823","nonce":2232342832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6826","nonce":2232342832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6830","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6831","nonce":2232342832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":2232342832},{"type":"v1","addr":"172.21.15.121:6829","nonce":2232342832}]},"public_addr":"172.21.15.121:6823/2232342832","cluster_addr":"172.21.15.121:6826/2232342832","heartbeat_back_addr":"172.21.15.121:6831/2232342832","heartbeat_front_addr":"172.21.15.121:6829/2232342832","state":["exists","up"]},{"osd":4,"uuid":"b3f3b4a1-b375-4526-b69e-344ab24c2da1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6801","nonce":99353111}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6803","nonce":99353111}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6807","nonce":99353111}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":99353111},{"type":"v1","addr":"172.21.15.123:6805","nonce":99353111}]},"public_addr":"172.21.15.123:6801/99353111","cluster_addr":"172.21.15.123:6803/99353111","heartbeat_back_addr":"172.21.15.123:6807/99353111","heartbeat_front_addr":"172.21.15.123:6805/99353111","state":["exists","up"]},{"osd":5,"uuid":"2ee24a08-1624-4961-9642-d7b510282484","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6818","nonce":2504374688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6822","nonce":2504374688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6830","nonce":2504374688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":2504374688},{"type":"v1","addr":"172.21.15.123:6826","nonce":2504374688}]},"public_addr":"172.21.15.123:6818/2504374688","cluster_addr":"172.21.15.123:6822/2504374688","heartbeat_back_addr":"172.21.15.123:6830/2504374688","heartbeat_front_addr":"172.21.15.123:6826/2504374688","state":["exists","up"]},{"osd":6,"uuid":"186118cc-6128-49de-acca-55260945c1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6817","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6819","nonce":4139299599}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6821","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6823","nonce":4139299599}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6829","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6831","nonce":4139299599}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6825","nonce":4139299599},{"type":"v1","addr":"172.21.15.123:6827","nonce":4139299599}]},"public_addr":"172.21.15.123:6819/4139299599","cluster_addr":"172.21.15.123:6823/4139299599","heartbeat_back_addr":"172.21.15.123:6831/4139299599","heartbeat_front_addr":"172.21.15.123:6827/4139299599","state":["exists","up"]},{"osd":7,"uuid":"a00aac6e-4bd9-415a-a93d-ae85210865ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6809","nonce":4043981069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6811","nonce":4043981069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6815","nonce":4043981069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":4043981069},{"type":"v1","addr":"172.21.15.123:6813","nonce":4043981069}]},"public_addr":"172.21.15.123:6809/4043981069","cluster_addr":"172.21.15.123:6811/4043981069","heartbeat_back_addr":"172.21.15.123:6815/4043981069","heartbeat_front_addr":"172.21.15.123:6813/4043981069","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.904540+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.958585+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.616367+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.885244+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.103127+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:47.167248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.884348+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:20:46.862356+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:22:55.148 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.148+0000 7efc48200640 1 -- 172.21.15.121:0/2789043330 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7efc2006c8f0 msgr2=0x7efc2006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:55.148 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.148+0000 7efc48200640 1 --2- 172.21.15.121:0/2789043330 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7efc2006c8f0 0x7efc2006edb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7efc34009a10 tx=0x7efc34008040 comp rx=0 tx=0).stop 2024-04-20T10:22:55.148 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.148+0000 7efc48200640 1 -- 172.21.15.121:0/2789043330 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc40152ee0 msgr2=0x7efc40153360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:55.148 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.148+0000 7efc48200640 1 --2- 172.21.15.121:0/2789043330 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc40152ee0 0x7efc40153360 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7efc2c00c330 tx=0x7efc2c02e070 comp rx=0 tx=0).stop 2024-04-20T10:22:55.148 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.148+0000 7efc48200640 1 -- 172.21.15.121:0/2789043330 shutdown_connections 2024-04-20T10:22:55.148 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.148+0000 7efc48200640 1 --2- 172.21.15.121:0/2789043330 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7efc401538a0 0x7efc40153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:55.148 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.148+0000 7efc48200640 1 --2- 172.21.15.121:0/2789043330 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7efc40152ee0 0x7efc40153360 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:55.148 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.148+0000 7efc48200640 1 --2- 172.21.15.121:0/2789043330 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7efc2006c8f0 0x7efc2006edb0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:55.148 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.148+0000 7efc48200640 1 --2- 172.21.15.121:0/2789043330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc28003f80 0x7efc400b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:55.148 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.148+0000 7efc48200640 1 -- 172.21.15.121:0/2789043330 >> 172.21.15.121:0/2789043330 conn(0x7efc400b3350 msgr2=0x7efc40072480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:55.148 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.148+0000 7efc48200640 1 -- 172.21.15.121:0/2789043330 shutdown_connections 2024-04-20T10:22:55.148 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.148+0000 7efc48200640 1 -- 172.21.15.121:0/2789043330 wait complete. 2024-04-20T10:22:55.168 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-20T10:22:55.168 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T10:22:55.371 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.369+0000 7f0c1a400640 1 Processor -- start 2024-04-20T10:22:55.371 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.369+0000 7f0c1a400640 1 -- start start 2024-04-20T10:22:55.371 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.370+0000 7f0c1a400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f0c1415b7e0 0x7f0c1415bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:55.371 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.370+0000 7f0c1a400640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c1406ccc0 con 0x7f0c1415b400 2024-04-20T10:22:55.371 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.370+0000 7f0c1a400640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c1406cfa0 con 0x7f0c1415c190 2024-04-20T10:22:55.371 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.370+0000 7f0c1a400640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f0c1406d280 con 0x7f0c1415b7e0 2024-04-20T10:22:55.371 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.371+0000 7f0c19000640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f0c1415b400 0x7f0c14072d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:44858/0 (socket says 172.21.15.121:44858) 2024-04-20T10:22:55.371 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.371+0000 7f0c19000640 1 -- 172.21.15.121:0/1060724284 learned_addr learned my addr 172.21.15.121:0/1060724284 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:22:55.372 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.371+0000 7f0c13e00640 1 --2- 172.21.15.121:0/1060724284 >> v2:172.21.15.121:3300/0 conn(0x7f0c1415b7e0 0x7f0c1415bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:55.372 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.371+0000 7f0c13400640 1 -- 172.21.15.121:0/1060724284 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1344396099 0 0) 0x7f0c1406ccc0 con 0x7f0c1415b400 2024-04-20T10:22:55.372 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.371+0000 7f0c13400640 1 -- 172.21.15.121:0/1060724284 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0bfc003520 con 0x7f0c1415b400 2024-04-20T10:22:55.372 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.371+0000 7f0c13e00640 1 -- 172.21.15.121:0/1060724284 >> v1:172.21.15.123:6789/0 conn(0x7f0c1415c190 legacy=0x7f0c1406ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:55.372 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.371+0000 7f0c13e00640 1 -- 172.21.15.121:0/1060724284 >> v1:172.21.15.121:6789/0 conn(0x7f0c1415b400 legacy=0x7f0c14072d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:55.372 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.371+0000 7f0c13e00640 1 -- 172.21.15.121:0/1060724284 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c1406e440 con 0x7f0c1415b7e0 2024-04-20T10:22:55.373 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.372+0000 7f0c13e00640 1 --2- 172.21.15.121:0/1060724284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c1415b7e0 0x7f0c1415bc50 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0c000097a0 tx=0x7f0c0002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e525e552a15c871 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:55.373 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.373+0000 7f0c13400640 1 -- 172.21.15.121:0/1060724284 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c00035020 con 0x7f0c1415b7e0 2024-04-20T10:22:55.373 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.373+0000 7f0c13400640 1 -- 172.21.15.121:0/1060724284 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c00031830 con 0x7f0c1415b7e0 2024-04-20T10:22:55.373 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.373+0000 7f0c13400640 1 -- 172.21.15.121:0/1060724284 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c00031c00 con 0x7f0c1415b7e0 2024-04-20T10:22:55.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.373+0000 7f0c1a400640 1 -- 172.21.15.121:0/1060724284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c1415b7e0 msgr2=0x7f0c1415bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:55.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.373+0000 7f0c1a400640 1 --2- 172.21.15.121:0/1060724284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c1415b7e0 0x7f0c1415bc50 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0c000097a0 tx=0x7f0c0002b8d0 comp rx=0 tx=0).stop 2024-04-20T10:22:55.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.373+0000 7f0c1a400640 1 -- 172.21.15.121:0/1060724284 shutdown_connections 2024-04-20T10:22:55.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.373+0000 7f0c1a400640 1 --2- 172.21.15.121:0/1060724284 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c1415b7e0 0x7f0c1415bc50 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:55.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.373+0000 7f0c1a400640 1 -- 172.21.15.121:0/1060724284 >> 172.21.15.121:0/1060724284 conn(0x7f0c140d0040 msgr2=0x7f0c140d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:55.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.373+0000 7f0c1a400640 1 -- 172.21.15.121:0/1060724284 shutdown_connections 2024-04-20T10:22:55.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.373+0000 7f0c1a400640 1 -- 172.21.15.121:0/1060724284 wait complete. 2024-04-20T10:22:55.375 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.375+0000 7f0c1a400640 1 Processor -- start 2024-04-20T10:22:55.375 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.375+0000 7f0c1a400640 1 -- start start 2024-04-20T10:22:55.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.376+0000 7f0c1a400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c1415b400 0x7f0c1415e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:55.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.376+0000 7f0c1a400640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c1415b7e0 0x7f0c1414ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:55.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.376+0000 7f0c1a400640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0c1415c190 0x7f0c140ca030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:55.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.376+0000 7f0c1a400640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1415c770 con 0x7f0c1415b400 2024-04-20T10:22:55.377 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.376+0000 7f0c1a400640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1415c5f0 con 0x7f0c1415b7e0 2024-04-20T10:22:55.377 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.376+0000 7f0c1a400640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f0c1406f350 con 0x7f0c1415c190 2024-04-20T10:22:55.377 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.377+0000 7f0c19000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c1415b400 0x7f0c1415e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:55.377 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.377+0000 7f0c19000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c1415b400 0x7f0c1415e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:48132/0 (socket says 172.21.15.121:48132) 2024-04-20T10:22:55.377 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.377+0000 7f0c19000640 1 -- 172.21.15.121:0/1752051821 learned_addr learned my addr 172.21.15.121:0/1752051821 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:55.377 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.377+0000 7f0c19000640 1 -- 172.21.15.121:0/1752051821 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0c1415c190 msgr2=0x7f0c140ca030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:55.377 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.377+0000 7f0c19000640 1 --2- 172.21.15.121:0/1752051821 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0c1415c190 0x7f0c140ca030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:55.377 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.377+0000 7f0c19000640 1 -- 172.21.15.121:0/1752051821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c1415b7e0 msgr2=0x7f0c1414ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:55.377 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.377+0000 7f0c19000640 1 --2- 172.21.15.121:0/1752051821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c1415b7e0 0x7f0c1414ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:55.377 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.377+0000 7f0c19000640 1 -- 172.21.15.121:0/1752051821 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c1415ea50 con 0x7f0c1415b400 2024-04-20T10:22:55.378 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.377+0000 7f0c19000640 1 --2- 172.21.15.121:0/1752051821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c1415b400 0x7f0c1415e350 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f0c0800c430 tx=0x7f0c0800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:55.378 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.378+0000 7f0c10c00640 1 -- 172.21.15.121:0/1752051821 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c08017070 con 0x7f0c1415b400 2024-04-20T10:22:55.378 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.378+0000 7f0c10c00640 1 -- 172.21.15.121:0/1752051821 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c08005ce0 con 0x7f0c1415b400 2024-04-20T10:22:55.378 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.378+0000 7f0c10c00640 1 -- 172.21.15.121:0/1752051821 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c080072d0 con 0x7f0c1415b400 2024-04-20T10:22:55.378 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.378+0000 7f0c1a400640 1 -- 172.21.15.121:0/1752051821 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c1415ed30 con 0x7f0c1415b400 2024-04-20T10:22:55.378 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.378+0000 7f0c1a400640 1 -- 172.21.15.121:0/1752051821 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c141591f0 con 0x7f0c1415b400 2024-04-20T10:22:55.379 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.379+0000 7f0c1a400640 1 -- 172.21.15.121:0/1752051821 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c140cc410 con 0x7f0c1415b400 2024-04-20T10:22:55.379 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.380+0000 7f0c10c00640 1 -- 172.21.15.121:0/1752051821 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0c0801a070 con 0x7f0c1415b400 2024-04-20T10:22:55.381 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.381+0000 7f0c10c00640 1 --2- 172.21.15.121:0/1752051821 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f0bf406c8f0 0x7f0bf406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:55.381 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.382+0000 7f0c10c00640 1 -- 172.21.15.121:0/1752051821 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f0c0808b030 con 0x7f0c1415b400 2024-04-20T10:22:55.381 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.382+0000 7f0c13e00640 1 --2- 172.21.15.121:0/1752051821 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f0bf406c8f0 0x7f0bf406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:55.381 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.382+0000 7f0c13e00640 1 --2- 172.21.15.121:0/1752051821 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f0bf406c8f0 0x7f0bf406edb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0c00036000 tx=0x7f0c00030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:55.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.386+0000 7f0c10c00640 1 -- 172.21.15.121:0/1752051821 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0c08058c70 con 0x7f0c1415b400 2024-04-20T10:22:55.584 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.584+0000 7f0c1a400640 1 -- 172.21.15.121:0/1752051821 --> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0c140b5220 con 0x7f0bf406c8f0 2024-04-20T10:22:55.605 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.605+0000 7f0c10c00640 1 -- 172.21.15.121:0/1752051821 <== mgr.4103 v2:172.21.15.121:6832/1408233272 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246423 (secure 0 0 0) 0x7f0c140b5220 con 0x7f0bf406c8f0 2024-04-20T10:22:55.609 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:22:55.610 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-04-20T10:22:55.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.611+0000 7f0c1a400640 1 -- 172.21.15.121:0/1752051821 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f0bf406c8f0 msgr2=0x7f0bf406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:55.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.611+0000 7f0c1a400640 1 --2- 172.21.15.121:0/1752051821 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f0bf406c8f0 0x7f0bf406edb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0c00036000 tx=0x7f0c00030000 comp rx=0 tx=0).stop 2024-04-20T10:22:55.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.612+0000 7f0c1a400640 1 -- 172.21.15.121:0/1752051821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c1415b400 msgr2=0x7f0c1415e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:55.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.612+0000 7f0c1a400640 1 --2- 172.21.15.121:0/1752051821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c1415b400 0x7f0c1415e350 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f0c0800c430 tx=0x7f0c0800c8f0 comp rx=0 tx=0).stop 2024-04-20T10:22:55.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.612+0000 7f0c1a400640 1 -- 172.21.15.121:0/1752051821 shutdown_connections 2024-04-20T10:22:55.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.612+0000 7f0c1a400640 1 --2- 172.21.15.121:0/1752051821 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f0c1415c190 0x7f0c140ca030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:55.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.612+0000 7f0c1a400640 1 --2- 172.21.15.121:0/1752051821 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f0bf406c8f0 0x7f0bf406edb0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:55.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.612+0000 7f0c1a400640 1 --2- 172.21.15.121:0/1752051821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c1415b7e0 0x7f0c1414ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:55.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.612+0000 7f0c1a400640 1 --2- 172.21.15.121:0/1752051821 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f0c1415b400 0x7f0c1415e350 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:55.612 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.612+0000 7f0c1a400640 1 -- 172.21.15.121:0/1752051821 >> 172.21.15.121:0/1752051821 conn(0x7f0c140d0040 msgr2=0x7f0c140b14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:55.613 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.612+0000 7f0c1a400640 1 -- 172.21.15.121:0/1752051821 shutdown_connections 2024-04-20T10:22:55.613 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.612+0000 7f0c1a400640 1 -- 172.21.15.121:0/1752051821 wait complete. 2024-04-20T10:22:55.629 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":95,"stamp":"2024-04-20T10:22:55.147880+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":7,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220536,"kb_used_data":4372,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754184,"statfs":{"total":773094113280,"available":772868284416,"internally_reserved":0,"allocated":4476928,"data_stored":2569971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.984225"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.187152+0000","last_change":"2024-04-20T10:22:50.187152+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.187152+0000","last_undegraded":"2024-04-20T10:22:50.187152+0000","last_fullsized":"2024-04-20T10:22:50.187152+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.186798+0000","last_change":"2024-04-20T10:22:50.186798+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.186798+0000","last_undegraded":"2024-04-20T10:22:50.186798+0000","last_fullsized":"2024-04-20T10:22:50.186798+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.189601+0000","last_change":"2024-04-20T10:22:50.189601+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.189601+0000","last_undegraded":"2024-04-20T10:22:50.189601+0000","last_fullsized":"2024-04-20T10:22:50.189601+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.181311+0000","last_change":"2024-04-20T10:22:50.181311+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.181311+0000","last_undegraded":"2024-04-20T10:22:50.181311+0000","last_fullsized":"2024-04-20T10:22:50.181311+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.184441+0000","last_change":"2024-04-20T10:22:50.184441+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.184441+0000","last_undegraded":"2024-04-20T10:22:50.184441+0000","last_fullsized":"2024-04-20T10:22:50.184441+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.182950+0000","last_change":"2024-04-20T10:22:50.182950+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.182950+0000","last_undegraded":"2024-04-20T10:22:50.182950+0000","last_fullsized":"2024-04-20T10:22:50.182950+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.192894+0000","last_change":"2024-04-20T10:22:50.192894+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.192894+0000","last_undegraded":"2024-04-20T10:22:50.192894+0000","last_fullsized":"2024-04-20T10:22:50.192894+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.181902+0000","last_change":"2024-04-20T10:22:50.181902+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.181902+0000","last_undegraded":"2024-04-20T10:22:50.181902+0000","last_fullsized":"2024-04-20T10:22:50.181902+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.192533+0000","last_change":"2024-04-20T10:22:50.192533+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.192533+0000","last_undegraded":"2024-04-20T10:22:50.192533+0000","last_fullsized":"2024-04-20T10:22:50.192533+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.194398+0000","last_change":"2024-04-20T10:22:50.194398+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.194398+0000","last_undegraded":"2024-04-20T10:22:50.194398+0000","last_fullsized":"2024-04-20T10:22:50.194398+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.194904+0000","last_change":"2024-04-20T10:22:50.194904+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.194904+0000","last_undegraded":"2024-04-20T10:22:50.194904+0000","last_fullsized":"2024-04-20T10:22:50.194904+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.192601+0000","last_change":"2024-04-20T10:22:50.192601+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.192601+0000","last_undegraded":"2024-04-20T10:22:50.192601+0000","last_fullsized":"2024-04-20T10:22:50.192601+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.197754+0000","last_change":"2024-04-20T10:22:50.197754+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.197754+0000","last_undegraded":"2024-04-20T10:22:50.197754+0000","last_fullsized":"2024-04-20T10:22:50.197754+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.188557+0000","last_change":"2024-04-20T10:22:50.188557+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.188557+0000","last_undegraded":"2024-04-20T10:22:50.188557+0000","last_fullsized":"2024-04-20T10:22:50.188557+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.192196+0000","last_change":"2024-04-20T10:22:50.192196+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.192196+0000","last_undegraded":"2024-04-20T10:22:50.192196+0000","last_fullsized":"2024-04-20T10:22:50.192196+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.190134+0000","last_change":"2024-04-20T10:22:50.190134+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.190134+0000","last_undegraded":"2024-04-20T10:22:50.190134+0000","last_fullsized":"2024-04-20T10:22:50.190134+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.189486+0000","last_change":"2024-04-20T10:22:50.189486+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.189486+0000","last_undegraded":"2024-04-20T10:22:50.189486+0000","last_fullsized":"2024-04-20T10:22:50.189486+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.177303+0000","last_change":"2024-04-20T10:22:50.177303+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.177303+0000","last_undegraded":"2024-04-20T10:22:50.177303+0000","last_fullsized":"2024-04-20T10:22:50.177303+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.182415+0000","last_change":"2024-04-20T10:22:50.182415+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.182415+0000","last_undegraded":"2024-04-20T10:22:50.182415+0000","last_fullsized":"2024-04-20T10:22:50.182415+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.177312+0000","last_change":"2024-04-20T10:22:50.177312+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.177312+0000","last_undegraded":"2024-04-20T10:22:50.177312+0000","last_fullsized":"2024-04-20T10:22:50.177312+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.195671+0000","last_change":"2024-04-20T10:22:50.195671+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.195671+0000","last_undegraded":"2024-04-20T10:22:50.195671+0000","last_fullsized":"2024-04-20T10:22:50.195671+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.186598+0000","last_change":"2024-04-20T10:22:50.186598+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.186598+0000","last_undegraded":"2024-04-20T10:22:50.186598+0000","last_fullsized":"2024-04-20T10:22:50.186598+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.190727+0000","last_change":"2024-04-20T10:22:50.190727+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.190727+0000","last_undegraded":"2024-04-20T10:22:50.190727+0000","last_fullsized":"2024-04-20T10:22:50.190727+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.182533+0000","last_change":"2024-04-20T10:22:50.182533+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.182533+0000","last_undegraded":"2024-04-20T10:22:50.182533+0000","last_fullsized":"2024-04-20T10:22:50.182533+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.181275+0000","last_change":"2024-04-20T10:22:50.181275+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.181275+0000","last_undegraded":"2024-04-20T10:22:50.181275+0000","last_fullsized":"2024-04-20T10:22:50.181275+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.182908+0000","last_change":"2024-04-20T10:22:50.182908+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.182908+0000","last_undegraded":"2024-04-20T10:22:50.182908+0000","last_fullsized":"2024-04-20T10:22:50.182908+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.177500+0000","last_change":"2024-04-20T10:22:50.177500+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.177500+0000","last_undegraded":"2024-04-20T10:22:50.177500+0000","last_fullsized":"2024-04-20T10:22:50.177500+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.187253+0000","last_change":"2024-04-20T10:22:50.187253+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.187253+0000","last_undegraded":"2024-04-20T10:22:50.187253+0000","last_fullsized":"2024-04-20T10:22:50.187253+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.177359+0000","last_change":"2024-04-20T10:22:50.177359+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.177359+0000","last_undegraded":"2024-04-20T10:22:50.177359+0000","last_fullsized":"2024-04-20T10:22:50.177359+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.179150+0000","last_change":"2024-04-20T10:22:50.179150+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.179150+0000","last_undegraded":"2024-04-20T10:22:50.179150+0000","last_fullsized":"2024-04-20T10:22:50.179150+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.189823+0000","last_change":"2024-04-20T10:22:50.189823+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.189823+0000","last_undegraded":"2024-04-20T10:22:50.189823+0000","last_fullsized":"2024-04-20T10:22:50.189823+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.179894+0000","last_change":"2024-04-20T10:22:50.179894+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.179894+0000","last_undegraded":"2024-04-20T10:22:50.179894+0000","last_fullsized":"2024-04-20T10:22:50.179894+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.195627+0000","last_change":"2024-04-20T10:22:50.195627+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.195627+0000","last_undegraded":"2024-04-20T10:22:50.195627+0000","last_fullsized":"2024-04-20T10:22:50.195627+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.211706+0000","last_change":"2024-04-20T10:22:50.208086+0000","last_active":"2024-04-20T10:22:50.211706+0000","last_peered":"2024-04-20T10:22:50.211706+0000","last_clean":"2024-04-20T10:22:50.211706+0000","last_became_active":"2024-04-20T10:22:50.207645+0000","last_became_peered":"2024-04-20T10:22:50.207645+0000","last_unstale":"2024-04-20T10:22:50.211706+0000","last_undegraded":"2024-04-20T10:22:50.211706+0000","last_fullsized":"2024-04-20T10:22:50.211706+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:36:24.988490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.183462+0000","last_change":"2024-04-20T10:22:50.183462+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.183462+0000","last_undegraded":"2024-04-20T10:22:50.183462+0000","last_fullsized":"2024-04-20T10:22:50.183462+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.207581+0000","last_change":"2024-04-20T10:22:50.204532+0000","last_active":"2024-04-20T10:22:50.207581+0000","last_peered":"2024-04-20T10:22:50.207581+0000","last_clean":"2024-04-20T10:22:50.207581+0000","last_became_active":"2024-04-20T10:22:50.204230+0000","last_became_peered":"2024-04-20T10:22:50.204230+0000","last_unstale":"2024-04-20T10:22:50.207581+0000","last_undegraded":"2024-04-20T10:22:50.207581+0000","last_fullsized":"2024-04-20T10:22:50.207581+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:10:15.543826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.192569+0000","last_change":"2024-04-20T10:22:50.192569+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.192569+0000","last_undegraded":"2024-04-20T10:22:50.192569+0000","last_fullsized":"2024-04-20T10:22:50.192569+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.207873+0000","last_change":"2024-04-20T10:22:50.205333+0000","last_active":"2024-04-20T10:22:50.207873+0000","last_peered":"2024-04-20T10:22:50.207873+0000","last_clean":"2024-04-20T10:22:50.207873+0000","last_became_active":"2024-04-20T10:22:50.205029+0000","last_became_peered":"2024-04-20T10:22:50.205029+0000","last_unstale":"2024-04-20T10:22:50.207873+0000","last_undegraded":"2024-04-20T10:22:50.207873+0000","last_fullsized":"2024-04-20T10:22:50.207873+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:16:59.003634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.186710+0000","last_change":"2024-04-20T10:22:50.186710+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.186710+0000","last_undegraded":"2024-04-20T10:22:50.186710+0000","last_fullsized":"2024-04-20T10:22:50.186710+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.211920+0000","last_change":"2024-04-20T10:22:50.207305+0000","last_active":"2024-04-20T10:22:50.211920+0000","last_peered":"2024-04-20T10:22:50.211920+0000","last_clean":"2024-04-20T10:22:50.211920+0000","last_became_active":"2024-04-20T10:22:50.207018+0000","last_became_peered":"2024-04-20T10:22:50.207018+0000","last_unstale":"2024-04-20T10:22:50.211920+0000","last_undegraded":"2024-04-20T10:22:50.211920+0000","last_fullsized":"2024-04-20T10:22:50.211920+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:36:24.988490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.193141+0000","last_change":"2024-04-20T10:22:50.193141+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.193141+0000","last_undegraded":"2024-04-20T10:22:50.193141+0000","last_fullsized":"2024-04-20T10:22:50.193141+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.205759+0000","last_change":"2024-04-20T10:22:50.202177+0000","last_active":"2024-04-20T10:22:50.205759+0000","last_peered":"2024-04-20T10:22:50.205759+0000","last_clean":"2024-04-20T10:22:50.205759+0000","last_became_active":"2024-04-20T10:22:50.201840+0000","last_became_peered":"2024-04-20T10:22:50.201840+0000","last_unstale":"2024-04-20T10:22:50.205759+0000","last_undegraded":"2024-04-20T10:22:50.205759+0000","last_fullsized":"2024-04-20T10:22:50.205759+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:57:16.061872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.188832+0000","last_change":"2024-04-20T10:22:50.188832+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.188832+0000","last_undegraded":"2024-04-20T10:22:50.188832+0000","last_fullsized":"2024-04-20T10:22:50.188832+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.212443+0000","last_change":"2024-04-20T10:22:50.211068+0000","last_active":"2024-04-20T10:22:50.212443+0000","last_peered":"2024-04-20T10:22:50.212443+0000","last_clean":"2024-04-20T10:22:50.212443+0000","last_became_active":"2024-04-20T10:22:50.210709+0000","last_became_peered":"2024-04-20T10:22:50.210709+0000","last_unstale":"2024-04-20T10:22:50.212443+0000","last_undegraded":"2024-04-20T10:22:50.212443+0000","last_fullsized":"2024-04-20T10:22:50.212443+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:16:59.003634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.185316+0000","last_change":"2024-04-20T10:22:50.185316+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.185316+0000","last_undegraded":"2024-04-20T10:22:50.185316+0000","last_fullsized":"2024-04-20T10:22:50.185316+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.205815+0000","last_change":"2024-04-20T10:22:50.202082+0000","last_active":"2024-04-20T10:22:50.205815+0000","last_peered":"2024-04-20T10:22:50.205815+0000","last_clean":"2024-04-20T10:22:50.205815+0000","last_became_active":"2024-04-20T10:22:50.201785+0000","last_became_peered":"2024-04-20T10:22:50.201785+0000","last_unstale":"2024-04-20T10:22:50.205815+0000","last_undegraded":"2024-04-20T10:22:50.205815+0000","last_fullsized":"2024-04-20T10:22:50.205815+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:22:51.962842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.185906+0000","last_change":"2024-04-20T10:22:50.185906+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.185906+0000","last_undegraded":"2024-04-20T10:22:50.185906+0000","last_fullsized":"2024-04-20T10:22:50.185906+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.211884+0000","last_change":"2024-04-20T10:22:50.208162+0000","last_active":"2024-04-20T10:22:50.211884+0000","last_peered":"2024-04-20T10:22:50.211884+0000","last_clean":"2024-04-20T10:22:50.211884+0000","last_became_active":"2024-04-20T10:22:50.206236+0000","last_became_peered":"2024-04-20T10:22:50.206236+0000","last_unstale":"2024-04-20T10:22:50.211884+0000","last_undegraded":"2024-04-20T10:22:50.211884+0000","last_fullsized":"2024-04-20T10:22:50.211884+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:47:00.135513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.177757+0000","last_change":"2024-04-20T10:22:50.177757+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.177757+0000","last_undegraded":"2024-04-20T10:22:50.177757+0000","last_fullsized":"2024-04-20T10:22:50.177757+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.200401+0000","last_change":"2024-04-20T10:22:50.196202+0000","last_active":"2024-04-20T10:22:50.200401+0000","last_peered":"2024-04-20T10:22:50.200401+0000","last_clean":"2024-04-20T10:22:50.200401+0000","last_became_active":"2024-04-20T10:22:50.195933+0000","last_became_peered":"2024-04-20T10:22:50.195933+0000","last_unstale":"2024-04-20T10:22:50.200401+0000","last_undegraded":"2024-04-20T10:22:50.200401+0000","last_fullsized":"2024-04-20T10:22:50.200401+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:20:05.462979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.192693+0000","last_change":"2024-04-20T10:22:50.192693+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.192693+0000","last_undegraded":"2024-04-20T10:22:50.192693+0000","last_fullsized":"2024-04-20T10:22:50.192693+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.214543+0000","last_change":"2024-04-20T10:22:50.209570+0000","last_active":"2024-04-20T10:22:50.214543+0000","last_peered":"2024-04-20T10:22:50.214543+0000","last_clean":"2024-04-20T10:22:50.214543+0000","last_became_active":"2024-04-20T10:22:50.209281+0000","last_became_peered":"2024-04-20T10:22:50.209281+0000","last_unstale":"2024-04-20T10:22:50.214543+0000","last_undegraded":"2024-04-20T10:22:50.214543+0000","last_fullsized":"2024-04-20T10:22:50.214543+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:09:34.293198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.191996+0000","last_change":"2024-04-20T10:22:50.191996+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.191996+0000","last_undegraded":"2024-04-20T10:22:50.191996+0000","last_fullsized":"2024-04-20T10:22:50.191996+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.209812+0000","last_change":"2024-04-20T10:22:50.206722+0000","last_active":"2024-04-20T10:22:50.209812+0000","last_peered":"2024-04-20T10:22:50.209812+0000","last_clean":"2024-04-20T10:22:50.209812+0000","last_became_active":"2024-04-20T10:22:50.206395+0000","last_became_peered":"2024-04-20T10:22:50.206395+0000","last_unstale":"2024-04-20T10:22:50.209812+0000","last_undegraded":"2024-04-20T10:22:50.209812+0000","last_fullsized":"2024-04-20T10:22:50.209812+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:58:15.263333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.198022+0000","last_change":"2024-04-20T10:22:50.198022+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.198022+0000","last_undegraded":"2024-04-20T10:22:50.198022+0000","last_fullsized":"2024-04-20T10:22:50.198022+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.207715+0000","last_change":"2024-04-20T10:22:50.203958+0000","last_active":"2024-04-20T10:22:50.207715+0000","last_peered":"2024-04-20T10:22:50.207715+0000","last_clean":"2024-04-20T10:22:50.207715+0000","last_became_active":"2024-04-20T10:22:50.203499+0000","last_became_peered":"2024-04-20T10:22:50.203499+0000","last_unstale":"2024-04-20T10:22:50.207715+0000","last_undegraded":"2024-04-20T10:22:50.207715+0000","last_fullsized":"2024-04-20T10:22:50.207715+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:16:27.290919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.192536+0000","last_change":"2024-04-20T10:22:50.192536+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.192536+0000","last_undegraded":"2024-04-20T10:22:50.192536+0000","last_fullsized":"2024-04-20T10:22:50.192536+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.200458+0000","last_change":"2024-04-20T10:22:50.196624+0000","last_active":"2024-04-20T10:22:50.200458+0000","last_peered":"2024-04-20T10:22:50.200458+0000","last_clean":"2024-04-20T10:22:50.200458+0000","last_became_active":"2024-04-20T10:22:50.196295+0000","last_became_peered":"2024-04-20T10:22:50.196295+0000","last_unstale":"2024-04-20T10:22:50.200458+0000","last_undegraded":"2024-04-20T10:22:50.200458+0000","last_fullsized":"2024-04-20T10:22:50.200458+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:30:35.298814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.179317+0000","last_change":"2024-04-20T10:22:50.179317+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.179317+0000","last_undegraded":"2024-04-20T10:22:50.179317+0000","last_fullsized":"2024-04-20T10:22:50.179317+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.205746+0000","last_change":"2024-04-20T10:22:50.200183+0000","last_active":"2024-04-20T10:22:50.205746+0000","last_peered":"2024-04-20T10:22:50.205746+0000","last_clean":"2024-04-20T10:22:50.205746+0000","last_became_active":"2024-04-20T10:22:50.199884+0000","last_became_peered":"2024-04-20T10:22:50.199884+0000","last_unstale":"2024-04-20T10:22:50.205746+0000","last_undegraded":"2024-04-20T10:22:50.205746+0000","last_fullsized":"2024-04-20T10:22:50.205746+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:10:15.543826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.195676+0000","last_change":"2024-04-20T10:22:50.195676+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.195676+0000","last_undegraded":"2024-04-20T10:22:50.195676+0000","last_fullsized":"2024-04-20T10:22:50.195676+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.209575+0000","last_change":"2024-04-20T10:22:50.205765+0000","last_active":"2024-04-20T10:22:50.209575+0000","last_peered":"2024-04-20T10:22:50.209575+0000","last_clean":"2024-04-20T10:22:50.209575+0000","last_became_active":"2024-04-20T10:22:50.205454+0000","last_became_peered":"2024-04-20T10:22:50.205454+0000","last_unstale":"2024-04-20T10:22:50.209575+0000","last_undegraded":"2024-04-20T10:22:50.209575+0000","last_fullsized":"2024-04-20T10:22:50.209575+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:20:42.218198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.185184+0000","last_change":"2024-04-20T10:22:50.185184+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.185184+0000","last_undegraded":"2024-04-20T10:22:50.185184+0000","last_fullsized":"2024-04-20T10:22:50.185184+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.212433+0000","last_change":"2024-04-20T10:22:50.210848+0000","last_active":"2024-04-20T10:22:50.212433+0000","last_peered":"2024-04-20T10:22:50.212433+0000","last_clean":"2024-04-20T10:22:50.212433+0000","last_became_active":"2024-04-20T10:22:50.210382+0000","last_became_peered":"2024-04-20T10:22:50.210382+0000","last_unstale":"2024-04-20T10:22:50.212433+0000","last_undegraded":"2024-04-20T10:22:50.212433+0000","last_fullsized":"2024-04-20T10:22:50.212433+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:39:48.488461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.186954+0000","last_change":"2024-04-20T10:22:50.186954+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.186954+0000","last_undegraded":"2024-04-20T10:22:50.186954+0000","last_fullsized":"2024-04-20T10:22:50.186954+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.211775+0000","last_change":"2024-04-20T10:22:50.208371+0000","last_active":"2024-04-20T10:22:50.211775+0000","last_peered":"2024-04-20T10:22:50.211775+0000","last_clean":"2024-04-20T10:22:50.211775+0000","last_became_active":"2024-04-20T10:22:50.208102+0000","last_became_peered":"2024-04-20T10:22:50.208102+0000","last_unstale":"2024-04-20T10:22:50.211775+0000","last_undegraded":"2024-04-20T10:22:50.211775+0000","last_fullsized":"2024-04-20T10:22:50.211775+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:16:51.466273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.190353+0000","last_change":"2024-04-20T10:22:50.190353+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.190353+0000","last_undegraded":"2024-04-20T10:22:50.190353+0000","last_fullsized":"2024-04-20T10:22:50.190353+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.209177+0000","last_change":"2024-04-20T10:22:50.205722+0000","last_active":"2024-04-20T10:22:50.209177+0000","last_peered":"2024-04-20T10:22:50.209177+0000","last_clean":"2024-04-20T10:22:50.209177+0000","last_became_active":"2024-04-20T10:22:50.205402+0000","last_became_peered":"2024-04-20T10:22:50.205402+0000","last_unstale":"2024-04-20T10:22:50.209177+0000","last_undegraded":"2024-04-20T10:22:50.209177+0000","last_fullsized":"2024-04-20T10:22:50.209177+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:20:42.218198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.193228+0000","last_change":"2024-04-20T10:22:50.193228+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.193228+0000","last_undegraded":"2024-04-20T10:22:50.193228+0000","last_fullsized":"2024-04-20T10:22:50.193228+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-20T10:22:50.173967+0000","last_change":"2024-04-20T10:22:50.173967+0000","last_active":"2024-04-20T10:22:49.145911+0000","last_peered":"2024-04-20T10:22:49.145911+0000","last_clean":"2024-04-20T10:22:49.145911+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.173967+0000","last_undegraded":"2024-04-20T10:22:50.173967+0000","last_fullsized":"2024-04-20T10:22:50.173967+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":["2","7"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.180223+0000","last_change":"2024-04-20T10:22:50.180223+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.180223+0000","last_undegraded":"2024-04-20T10:22:50.180223+0000","last_fullsized":"2024-04-20T10:22:50.180223+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.207866+0000","last_change":"2024-04-20T10:22:50.205027+0000","last_active":"2024-04-20T10:22:50.207866+0000","last_peered":"2024-04-20T10:22:50.207866+0000","last_clean":"2024-04-20T10:22:50.207866+0000","last_became_active":"2024-04-20T10:22:50.204749+0000","last_became_peered":"2024-04-20T10:22:50.204749+0000","last_unstale":"2024-04-20T10:22:50.207866+0000","last_undegraded":"2024-04-20T10:22:50.207866+0000","last_fullsized":"2024-04-20T10:22:50.207866+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:39:48.488461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.188798+0000","last_change":"2024-04-20T10:22:50.188798+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.188798+0000","last_undegraded":"2024-04-20T10:22:50.188798+0000","last_fullsized":"2024-04-20T10:22:50.188798+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.200723+0000","last_change":"2024-04-20T10:22:50.194920+0000","last_active":"2024-04-20T10:22:50.200723+0000","last_peered":"2024-04-20T10:22:50.200723+0000","last_clean":"2024-04-20T10:22:50.200723+0000","last_became_active":"2024-04-20T10:22:50.194622+0000","last_became_peered":"2024-04-20T10:22:50.194622+0000","last_unstale":"2024-04-20T10:22:50.200723+0000","last_undegraded":"2024-04-20T10:22:50.200723+0000","last_fullsized":"2024-04-20T10:22:50.200723+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:30:02.849778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.181408+0000","last_change":"2024-04-20T10:22:50.181408+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.181408+0000","last_undegraded":"2024-04-20T10:22:50.181408+0000","last_fullsized":"2024-04-20T10:22:50.181408+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-20T10:22:50.173618+0000","last_change":"2024-04-20T10:22:50.173618+0000","last_active":"2024-04-20T10:22:49.145911+0000","last_peered":"2024-04-20T10:22:49.145911+0000","last_clean":"2024-04-20T10:22:49.145911+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.173618+0000","last_undegraded":"2024-04-20T10:22:50.173618+0000","last_fullsized":"2024-04-20T10:22:50.173618+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.195699+0000","last_change":"2024-04-20T10:22:50.195699+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.195699+0000","last_undegraded":"2024-04-20T10:22:50.195699+0000","last_fullsized":"2024-04-20T10:22:50.195699+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.208591+0000","last_change":"2024-04-20T10:22:50.205146+0000","last_active":"2024-04-20T10:22:50.208591+0000","last_peered":"2024-04-20T10:22:50.208591+0000","last_clean":"2024-04-20T10:22:50.208591+0000","last_became_active":"2024-04-20T10:22:50.204772+0000","last_became_peered":"2024-04-20T10:22:50.204772+0000","last_unstale":"2024-04-20T10:22:50.208591+0000","last_undegraded":"2024-04-20T10:22:50.208591+0000","last_fullsized":"2024-04-20T10:22:50.208591+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:22:51.962842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.179884+0000","last_change":"2024-04-20T10:22:50.179884+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.179884+0000","last_undegraded":"2024-04-20T10:22:50.179884+0000","last_fullsized":"2024-04-20T10:22:50.179884+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.200768+0000","last_change":"2024-04-20T10:22:50.194938+0000","last_active":"2024-04-20T10:22:50.200768+0000","last_peered":"2024-04-20T10:22:50.200768+0000","last_clean":"2024-04-20T10:22:50.200768+0000","last_became_active":"2024-04-20T10:22:50.194631+0000","last_became_peered":"2024-04-20T10:22:50.194631+0000","last_unstale":"2024-04-20T10:22:50.200768+0000","last_undegraded":"2024-04-20T10:22:50.200768+0000","last_fullsized":"2024-04-20T10:22:50.200768+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:32:32.746673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.184467+0000","last_change":"2024-04-20T10:22:50.184467+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.184467+0000","last_undegraded":"2024-04-20T10:22:50.184467+0000","last_fullsized":"2024-04-20T10:22:50.184467+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.203938+0000","last_change":"2024-04-20T10:22:50.200577+0000","last_active":"2024-04-20T10:22:50.203938+0000","last_peered":"2024-04-20T10:22:50.203938+0000","last_clean":"2024-04-20T10:22:50.203938+0000","last_became_active":"2024-04-20T10:22:50.199510+0000","last_became_peered":"2024-04-20T10:22:50.199510+0000","last_unstale":"2024-04-20T10:22:50.203938+0000","last_undegraded":"2024-04-20T10:22:50.203938+0000","last_fullsized":"2024-04-20T10:22:50.203938+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:26:16.280439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.191998+0000","last_change":"2024-04-20T10:22:50.191998+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.191998+0000","last_undegraded":"2024-04-20T10:22:50.191998+0000","last_fullsized":"2024-04-20T10:22:50.191998+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-20T10:22:50.174033+0000","last_change":"2024-04-20T10:22:50.174033+0000","last_active":"2024-04-20T10:22:49.145911+0000","last_peered":"2024-04-20T10:22:49.145911+0000","last_clean":"2024-04-20T10:22:49.145911+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.174033+0000","last_undegraded":"2024-04-20T10:22:50.174033+0000","last_fullsized":"2024-04-20T10:22:50.174033+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":["2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.183765+0000","last_change":"2024-04-20T10:22:50.183765+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.183765+0000","last_undegraded":"2024-04-20T10:22:50.183765+0000","last_fullsized":"2024-04-20T10:22:50.183765+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.200821+0000","last_change":"2024-04-20T10:22:50.196011+0000","last_active":"2024-04-20T10:22:50.200821+0000","last_peered":"2024-04-20T10:22:50.200821+0000","last_clean":"2024-04-20T10:22:50.200821+0000","last_became_active":"2024-04-20T10:22:50.195748+0000","last_became_peered":"2024-04-20T10:22:50.195748+0000","last_unstale":"2024-04-20T10:22:50.200821+0000","last_undegraded":"2024-04-20T10:22:50.200821+0000","last_fullsized":"2024-04-20T10:22:50.200821+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:36:24.988490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.184945+0000","last_change":"2024-04-20T10:22:50.184945+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.184945+0000","last_undegraded":"2024-04-20T10:22:50.184945+0000","last_fullsized":"2024-04-20T10:22:50.184945+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.205855+0000","last_change":"2024-04-20T10:22:50.203809+0000","last_active":"2024-04-20T10:22:50.205855+0000","last_peered":"2024-04-20T10:22:50.205855+0000","last_clean":"2024-04-20T10:22:50.205855+0000","last_became_active":"2024-04-20T10:22:50.203451+0000","last_became_peered":"2024-04-20T10:22:50.203451+0000","last_unstale":"2024-04-20T10:22:50.205855+0000","last_undegraded":"2024-04-20T10:22:50.205855+0000","last_fullsized":"2024-04-20T10:22:50.205855+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:47:00.135513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.180703+0000","last_change":"2024-04-20T10:22:50.180703+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.180703+0000","last_undegraded":"2024-04-20T10:22:50.180703+0000","last_fullsized":"2024-04-20T10:22:50.180703+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.203328+0000","last_change":"2024-04-20T10:22:50.197399+0000","last_active":"2024-04-20T10:22:50.203328+0000","last_peered":"2024-04-20T10:22:50.203328+0000","last_clean":"2024-04-20T10:22:50.203328+0000","last_became_active":"2024-04-20T10:22:50.197015+0000","last_became_peered":"2024-04-20T10:22:50.197015+0000","last_unstale":"2024-04-20T10:22:50.203328+0000","last_undegraded":"2024-04-20T10:22:50.203328+0000","last_fullsized":"2024-04-20T10:22:50.203328+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:41:01.514306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.187559+0000","last_change":"2024-04-20T10:22:50.187559+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.187559+0000","last_undegraded":"2024-04-20T10:22:50.187559+0000","last_fullsized":"2024-04-20T10:22:50.187559+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.208900+0000","last_change":"2024-04-20T10:22:50.203369+0000","last_active":"2024-04-20T10:22:50.208900+0000","last_peered":"2024-04-20T10:22:50.208900+0000","last_clean":"2024-04-20T10:22:50.208900+0000","last_became_active":"2024-04-20T10:22:50.203083+0000","last_became_peered":"2024-04-20T10:22:50.203083+0000","last_unstale":"2024-04-20T10:22:50.208900+0000","last_undegraded":"2024-04-20T10:22:50.208900+0000","last_fullsized":"2024-04-20T10:22:50.208900+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:41:01.514306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.189625+0000","last_change":"2024-04-20T10:22:50.189625+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.189625+0000","last_undegraded":"2024-04-20T10:22:50.189625+0000","last_fullsized":"2024-04-20T10:22:50.189625+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.205556+0000","last_change":"2024-04-20T10:22:50.198717+0000","last_active":"2024-04-20T10:22:50.205556+0000","last_peered":"2024-04-20T10:22:50.205556+0000","last_clean":"2024-04-20T10:22:50.205556+0000","last_became_active":"2024-04-20T10:22:50.197112+0000","last_became_peered":"2024-04-20T10:22:50.197112+0000","last_unstale":"2024-04-20T10:22:50.205556+0000","last_undegraded":"2024-04-20T10:22:50.205556+0000","last_fullsized":"2024-04-20T10:22:50.205556+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:36:31.551385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.189620+0000","last_change":"2024-04-20T10:22:50.189620+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.189620+0000","last_undegraded":"2024-04-20T10:22:50.189620+0000","last_fullsized":"2024-04-20T10:22:50.189620+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.206576+0000","last_change":"2024-04-20T10:22:50.203854+0000","last_active":"2024-04-20T10:22:50.206576+0000","last_peered":"2024-04-20T10:22:50.206576+0000","last_clean":"2024-04-20T10:22:50.206576+0000","last_became_active":"2024-04-20T10:22:50.203439+0000","last_became_peered":"2024-04-20T10:22:50.203439+0000","last_unstale":"2024-04-20T10:22:50.206576+0000","last_undegraded":"2024-04-20T10:22:50.206576+0000","last_fullsized":"2024-04-20T10:22:50.206576+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:10:15.543826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":198,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.195263+0000","last_change":"2024-04-20T10:20:52.916129+0000","last_active":"2024-04-20T10:22:50.195263+0000","last_peered":"2024-04-20T10:22:50.195263+0000","last_clean":"2024-04-20T10:22:50.195263+0000","last_became_active":"2024-04-20T10:20:52.913922+0000","last_became_peered":"2024-04-20T10:20:52.913922+0000","last_unstale":"2024-04-20T10:22:50.195263+0000","last_undegraded":"2024-04-20T10:22:50.195263+0000","last_fullsized":"2024-04-20T10:22:50.195263+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:20:51.267588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:20:51.267588+0000","last_clean_scrub_stamp":"2024-04-20T10:20:51.267588+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:30:44.465836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542170,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":1003520,"data_stored":763189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":409600,"data_stored":172821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542170,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":409600,"data_stored":172821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542170,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":409600,"data_stored":172821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542170,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":421888,"data_stored":179488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542170,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":409600,"data_stored":172821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542172,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":409600,"data_stored":172821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542172,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":1003520,"data_stored":763189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T10:22:55.634 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T10:22:55.696 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:22:55.722 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:22:55.874 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.874+0000 7fdfcb600640 1 Processor -- start 2024-04-20T10:22:55.995 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.875+0000 7fdfcb600640 1 -- start start 2024-04-20T10:22:56.002 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.876+0000 7fdfcb600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdfc415e8a0 0x7fdfc406e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:56.002 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.876+0000 7fdfcb600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdfc40d0a70 0x7fdfc40ad680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:56.002 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.876+0000 7fdfcb600640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdfc406f060 con 0x7fdfc406dbb0 2024-04-20T10:22:56.003 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.876+0000 7fdfcb600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fdfc406e7c0 con 0x7fdfc40d0a70 2024-04-20T10:22:56.003 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.876+0000 7fdfcb600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fdfc406eaa0 con 0x7fdfc415e8a0 2024-04-20T10:22:56.003 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.877+0000 7fdfc9600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdfc40d0a70 0x7fdfc40ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:56.003 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.877+0000 7fdfc9600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fdfc40d0a70 0x7fdfc40ad680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:48134/0 (socket says 172.21.15.121:48134) 2024-04-20T10:22:56.003 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.877+0000 7fdfc9600640 1 -- 172.21.15.121:0/3603577962 learned_addr learned my addr 172.21.15.121:0/3603577962 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:56.003 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.877+0000 7fdfc8c00640 1 --2- 172.21.15.121:0/3603577962 >> v2:172.21.15.123:3300/0 conn(0x7fdfc415e8a0 0x7fdfc406e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:56.003 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.877+0000 7fdfc9600640 1 -- 172.21.15.121:0/3603577962 >> v2:172.21.15.123:3300/0 conn(0x7fdfc415e8a0 msgr2=0x7fdfc406e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:56.003 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.877+0000 7fdfc9600640 1 --2- 172.21.15.121:0/3603577962 >> v2:172.21.15.123:3300/0 conn(0x7fdfc415e8a0 0x7fdfc406e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:56.004 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.877+0000 7fdfc9600640 1 -- 172.21.15.121:0/3603577962 >> v1:172.21.15.123:6789/0 conn(0x7fdfc406dbb0 legacy=0x7fdfc40d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:56.004 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.877+0000 7fdfc9600640 1 -- 172.21.15.121:0/3603577962 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfc406ed80 con 0x7fdfc40d0a70 2024-04-20T10:22:56.004 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.878+0000 7fdfc9600640 1 --2- 172.21.15.121:0/3603577962 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdfc40d0a70 0x7fdfc40ad680 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fdfc406d380 tx=0x7fdfbc02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b702df805f36258f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:56.004 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.879+0000 7fdfc3e00640 1 -- 172.21.15.121:0/3603577962 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc02cb20 con 0x7fdfc40d0a70 2024-04-20T10:22:56.004 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.879+0000 7fdfc3e00640 1 -- 172.21.15.121:0/3603577962 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfbc02ccc0 con 0x7fdfc40d0a70 2024-04-20T10:22:56.004 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.879+0000 7fdfc3e00640 1 -- 172.21.15.121:0/3603577962 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc034610 con 0x7fdfc40d0a70 2024-04-20T10:22:56.004 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.879+0000 7fdfcb600640 1 -- 172.21.15.121:0/3603577962 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdfc40d0a70 msgr2=0x7fdfc40ad680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:56.004 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.879+0000 7fdfcb600640 1 --2- 172.21.15.121:0/3603577962 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdfc40d0a70 0x7fdfc40ad680 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fdfc406d380 tx=0x7fdfbc02bf80 comp rx=0 tx=0).stop 2024-04-20T10:22:56.004 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.880+0000 7fdfcb600640 1 -- 172.21.15.121:0/3603577962 shutdown_connections 2024-04-20T10:22:56.004 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.880+0000 7fdfcb600640 1 --2- 172.21.15.121:0/3603577962 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdfc40d0a70 0x7fdfc40ad680 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:56.004 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.880+0000 7fdfcb600640 1 --2- 172.21.15.121:0/3603577962 >> v2:172.21.15.123:3300/0 conn(0x7fdfc415e8a0 0x7fdfc406e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:56.005 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.880+0000 7fdfcb600640 1 -- 172.21.15.121:0/3603577962 >> 172.21.15.121:0/3603577962 conn(0x7fdfc40b3350 msgr2=0x7fdfc40b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:56.005 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.880+0000 7fdfcb600640 1 -- 172.21.15.121:0/3603577962 shutdown_connections 2024-04-20T10:22:56.005 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.880+0000 7fdfcb600640 1 -- 172.21.15.121:0/3603577962 wait complete. 2024-04-20T10:22:56.005 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.882+0000 7fdfcb600640 1 Processor -- start 2024-04-20T10:22:56.005 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.882+0000 7fdfcb600640 1 -- start start 2024-04-20T10:22:56.005 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.883+0000 7fdfcb600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfc406dbb0 0x7fdfc415b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:56.005 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.883+0000 7fdfcb600640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdfc40d0a70 0x7fdfc4143610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:56.005 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.883+0000 7fdfcb600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdfc415e8a0 0x7fdfc4143b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:56.005 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.883+0000 7fdfcb600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc406fef0 con 0x7fdfc415e8a0 2024-04-20T10:22:56.005 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.883+0000 7fdfcb600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdfc406fd70 con 0x7fdfc406dbb0 2024-04-20T10:22:56.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.883+0000 7fdfcb600640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fdfc40adbc0 con 0x7fdfc40d0a70 2024-04-20T10:22:56.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.883+0000 7fdfc9600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdfc415e8a0 0x7fdfc4143b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:56.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.883+0000 7fdfc9600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdfc415e8a0 0x7fdfc4143b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:48140/0 (socket says 172.21.15.121:48140) 2024-04-20T10:22:56.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.883+0000 7fdfc9600640 1 -- 172.21.15.121:0/1899973828 learned_addr learned my addr 172.21.15.121:0/1899973828 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:56.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.884+0000 7fdfbbe00640 1 --2- 172.21.15.121:0/1899973828 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdfc40d0a70 0x7fdfc4143610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:56.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.884+0000 7fdfc8c00640 1 --2- 172.21.15.121:0/1899973828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfc406dbb0 0x7fdfc415b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:56.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.884+0000 7fdfc9600640 1 -- 172.21.15.121:0/1899973828 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdfc40d0a70 msgr2=0x7fdfc4143610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:56.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.884+0000 7fdfc9600640 1 --2- 172.21.15.121:0/1899973828 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdfc40d0a70 0x7fdfc4143610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:56.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.884+0000 7fdfc9600640 1 -- 172.21.15.121:0/1899973828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfc406dbb0 msgr2=0x7fdfc415b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:56.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.884+0000 7fdfc9600640 1 --2- 172.21.15.121:0/1899973828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfc406dbb0 0x7fdfc415b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:56.006 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.884+0000 7fdfc9600640 1 -- 172.21.15.121:0/1899973828 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfc4144220 con 0x7fdfc415e8a0 2024-04-20T10:22:56.007 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.884+0000 7fdfbbe00640 1 --2- 172.21.15.121:0/1899973828 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdfc40d0a70 0x7fdfc4143610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:56.007 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.884+0000 7fdfc9600640 1 --2- 172.21.15.121:0/1899973828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdfc415e8a0 0x7fdfc4143b50 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc036040 tx=0x7fdfbc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:56.007 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.884+0000 7fdfc1600640 1 -- 172.21.15.121:0/1899973828 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc043020 con 0x7fdfc415e8a0 2024-04-20T10:22:56.007 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.885+0000 7fdfc1600640 1 -- 172.21.15.121:0/1899973828 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdfbc03e450 con 0x7fdfc415e8a0 2024-04-20T10:22:56.007 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.885+0000 7fdfc1600640 1 -- 172.21.15.121:0/1899973828 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc009510 con 0x7fdfc415e8a0 2024-04-20T10:22:56.007 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.885+0000 7fdfcb600640 1 -- 172.21.15.121:0/1899973828 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfc4144500 con 0x7fdfc415e8a0 2024-04-20T10:22:56.007 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.885+0000 7fdfcb600640 1 -- 172.21.15.121:0/1899973828 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfc4141900 con 0x7fdfc415e8a0 2024-04-20T10:22:56.007 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.886+0000 7fdfcb600640 1 -- 172.21.15.121:0/1899973828 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfc4068980 con 0x7fdfc415e8a0 2024-04-20T10:22:56.007 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.887+0000 7fdfc1600640 1 -- 172.21.15.121:0/1899973828 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdfbc046020 con 0x7fdfc415e8a0 2024-04-20T10:22:56.007 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.888+0000 7fdfc1600640 1 --2- 172.21.15.121:0/1899973828 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fdfa006c8f0 0x7fdfa006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:56.007 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.888+0000 7fdfc1600640 1 -- 172.21.15.121:0/1899973828 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fdfbc0ae500 con 0x7fdfc415e8a0 2024-04-20T10:22:56.007 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.888+0000 7fdfc8c00640 1 --2- 172.21.15.121:0/1899973828 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fdfa006c8f0 0x7fdfa006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:56.008 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.889+0000 7fdfc8c00640 1 --2- 172.21.15.121:0/1899973828 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fdfa006c8f0 0x7fdfa006edb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4006fd0 tx=0x7fdfb4008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:56.008 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:55.892+0000 7fdfc1600640 1 -- 172.21.15.121:0/1899973828 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdfbc04d030 con 0x7fdfc415e8a0 2024-04-20T10:22:56.087 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.087+0000 7fdfcb600640 1 -- 172.21.15.121:0/1899973828 --> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdfc406d160 con 0x7fdfa006c8f0 2024-04-20T10:22:56.106 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.106+0000 7fdfc1600640 1 -- 172.21.15.121:0/1899973828 <== mgr.4103 v2:172.21.15.121:6832/1408233272 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246423 (secure 0 0 0) 0x7fdfc406d160 con 0x7fdfa006c8f0 2024-04-20T10:22:56.109 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:22:56.111 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-04-20T10:22:56.112 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.111+0000 7fdfcb600640 1 -- 172.21.15.121:0/1899973828 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fdfa006c8f0 msgr2=0x7fdfa006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:56.112 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.111+0000 7fdfcb600640 1 --2- 172.21.15.121:0/1899973828 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fdfa006c8f0 0x7fdfa006edb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4006fd0 tx=0x7fdfb4008040 comp rx=0 tx=0).stop 2024-04-20T10:22:56.112 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.111+0000 7fdfcb600640 1 -- 172.21.15.121:0/1899973828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdfc415e8a0 msgr2=0x7fdfc4143b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:56.112 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.111+0000 7fdfcb600640 1 --2- 172.21.15.121:0/1899973828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdfc415e8a0 0x7fdfc4143b50 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc036040 tx=0x7fdfbc004a40 comp rx=0 tx=0).stop 2024-04-20T10:22:56.112 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.111+0000 7fdfcb600640 1 -- 172.21.15.121:0/1899973828 shutdown_connections 2024-04-20T10:22:56.112 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.111+0000 7fdfcb600640 1 --2- 172.21.15.121:0/1899973828 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fdfc415e8a0 0x7fdfc4143b50 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:56.112 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.111+0000 7fdfcb600640 1 --2- 172.21.15.121:0/1899973828 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fdfc40d0a70 0x7fdfc4143610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:56.113 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.111+0000 7fdfcb600640 1 --2- 172.21.15.121:0/1899973828 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fdfa006c8f0 0x7fdfa006edb0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:56.113 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.111+0000 7fdfcb600640 1 --2- 172.21.15.121:0/1899973828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfc406dbb0 0x7fdfc415b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:56.113 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.111+0000 7fdfcb600640 1 -- 172.21.15.121:0/1899973828 >> 172.21.15.121:0/1899973828 conn(0x7fdfc40b3350 msgr2=0x7fdfc415db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:56.113 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.111+0000 7fdfcb600640 1 -- 172.21.15.121:0/1899973828 shutdown_connections 2024-04-20T10:22:56.113 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.111+0000 7fdfcb600640 1 -- 172.21.15.121:0/1899973828 wait complete. 2024-04-20T10:22:56.130 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":95,"stamp":"2024-04-20T10:22:55.147880+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":7,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220536,"kb_used_data":4372,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754184,"statfs":{"total":773094113280,"available":772868284416,"internally_reserved":0,"allocated":4476928,"data_stored":2569971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.984225"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.187152+0000","last_change":"2024-04-20T10:22:50.187152+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.187152+0000","last_undegraded":"2024-04-20T10:22:50.187152+0000","last_fullsized":"2024-04-20T10:22:50.187152+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.186798+0000","last_change":"2024-04-20T10:22:50.186798+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.186798+0000","last_undegraded":"2024-04-20T10:22:50.186798+0000","last_fullsized":"2024-04-20T10:22:50.186798+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.189601+0000","last_change":"2024-04-20T10:22:50.189601+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.189601+0000","last_undegraded":"2024-04-20T10:22:50.189601+0000","last_fullsized":"2024-04-20T10:22:50.189601+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.181311+0000","last_change":"2024-04-20T10:22:50.181311+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.181311+0000","last_undegraded":"2024-04-20T10:22:50.181311+0000","last_fullsized":"2024-04-20T10:22:50.181311+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.184441+0000","last_change":"2024-04-20T10:22:50.184441+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.184441+0000","last_undegraded":"2024-04-20T10:22:50.184441+0000","last_fullsized":"2024-04-20T10:22:50.184441+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.182950+0000","last_change":"2024-04-20T10:22:50.182950+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.182950+0000","last_undegraded":"2024-04-20T10:22:50.182950+0000","last_fullsized":"2024-04-20T10:22:50.182950+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.192894+0000","last_change":"2024-04-20T10:22:50.192894+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.192894+0000","last_undegraded":"2024-04-20T10:22:50.192894+0000","last_fullsized":"2024-04-20T10:22:50.192894+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.181902+0000","last_change":"2024-04-20T10:22:50.181902+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.181902+0000","last_undegraded":"2024-04-20T10:22:50.181902+0000","last_fullsized":"2024-04-20T10:22:50.181902+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.192533+0000","last_change":"2024-04-20T10:22:50.192533+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.192533+0000","last_undegraded":"2024-04-20T10:22:50.192533+0000","last_fullsized":"2024-04-20T10:22:50.192533+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.194398+0000","last_change":"2024-04-20T10:22:50.194398+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.194398+0000","last_undegraded":"2024-04-20T10:22:50.194398+0000","last_fullsized":"2024-04-20T10:22:50.194398+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.194904+0000","last_change":"2024-04-20T10:22:50.194904+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.194904+0000","last_undegraded":"2024-04-20T10:22:50.194904+0000","last_fullsized":"2024-04-20T10:22:50.194904+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.192601+0000","last_change":"2024-04-20T10:22:50.192601+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.192601+0000","last_undegraded":"2024-04-20T10:22:50.192601+0000","last_fullsized":"2024-04-20T10:22:50.192601+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.197754+0000","last_change":"2024-04-20T10:22:50.197754+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.197754+0000","last_undegraded":"2024-04-20T10:22:50.197754+0000","last_fullsized":"2024-04-20T10:22:50.197754+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.188557+0000","last_change":"2024-04-20T10:22:50.188557+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.188557+0000","last_undegraded":"2024-04-20T10:22:50.188557+0000","last_fullsized":"2024-04-20T10:22:50.188557+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.192196+0000","last_change":"2024-04-20T10:22:50.192196+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.192196+0000","last_undegraded":"2024-04-20T10:22:50.192196+0000","last_fullsized":"2024-04-20T10:22:50.192196+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.190134+0000","last_change":"2024-04-20T10:22:50.190134+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.190134+0000","last_undegraded":"2024-04-20T10:22:50.190134+0000","last_fullsized":"2024-04-20T10:22:50.190134+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.189486+0000","last_change":"2024-04-20T10:22:50.189486+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.189486+0000","last_undegraded":"2024-04-20T10:22:50.189486+0000","last_fullsized":"2024-04-20T10:22:50.189486+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.177303+0000","last_change":"2024-04-20T10:22:50.177303+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.177303+0000","last_undegraded":"2024-04-20T10:22:50.177303+0000","last_fullsized":"2024-04-20T10:22:50.177303+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.182415+0000","last_change":"2024-04-20T10:22:50.182415+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.182415+0000","last_undegraded":"2024-04-20T10:22:50.182415+0000","last_fullsized":"2024-04-20T10:22:50.182415+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.177312+0000","last_change":"2024-04-20T10:22:50.177312+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.177312+0000","last_undegraded":"2024-04-20T10:22:50.177312+0000","last_fullsized":"2024-04-20T10:22:50.177312+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.195671+0000","last_change":"2024-04-20T10:22:50.195671+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.195671+0000","last_undegraded":"2024-04-20T10:22:50.195671+0000","last_fullsized":"2024-04-20T10:22:50.195671+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.186598+0000","last_change":"2024-04-20T10:22:50.186598+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.186598+0000","last_undegraded":"2024-04-20T10:22:50.186598+0000","last_fullsized":"2024-04-20T10:22:50.186598+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.190727+0000","last_change":"2024-04-20T10:22:50.190727+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.190727+0000","last_undegraded":"2024-04-20T10:22:50.190727+0000","last_fullsized":"2024-04-20T10:22:50.190727+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.182533+0000","last_change":"2024-04-20T10:22:50.182533+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.182533+0000","last_undegraded":"2024-04-20T10:22:50.182533+0000","last_fullsized":"2024-04-20T10:22:50.182533+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.181275+0000","last_change":"2024-04-20T10:22:50.181275+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.181275+0000","last_undegraded":"2024-04-20T10:22:50.181275+0000","last_fullsized":"2024-04-20T10:22:50.181275+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.182908+0000","last_change":"2024-04-20T10:22:50.182908+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.182908+0000","last_undegraded":"2024-04-20T10:22:50.182908+0000","last_fullsized":"2024-04-20T10:22:50.182908+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.177500+0000","last_change":"2024-04-20T10:22:50.177500+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.177500+0000","last_undegraded":"2024-04-20T10:22:50.177500+0000","last_fullsized":"2024-04-20T10:22:50.177500+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.187253+0000","last_change":"2024-04-20T10:22:50.187253+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.187253+0000","last_undegraded":"2024-04-20T10:22:50.187253+0000","last_fullsized":"2024-04-20T10:22:50.187253+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.177359+0000","last_change":"2024-04-20T10:22:50.177359+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.177359+0000","last_undegraded":"2024-04-20T10:22:50.177359+0000","last_fullsized":"2024-04-20T10:22:50.177359+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.179150+0000","last_change":"2024-04-20T10:22:50.179150+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.179150+0000","last_undegraded":"2024-04-20T10:22:50.179150+0000","last_fullsized":"2024-04-20T10:22:50.179150+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.189823+0000","last_change":"2024-04-20T10:22:50.189823+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.189823+0000","last_undegraded":"2024-04-20T10:22:50.189823+0000","last_fullsized":"2024-04-20T10:22:50.189823+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.179894+0000","last_change":"2024-04-20T10:22:50.179894+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.179894+0000","last_undegraded":"2024-04-20T10:22:50.179894+0000","last_fullsized":"2024-04-20T10:22:50.179894+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.195627+0000","last_change":"2024-04-20T10:22:50.195627+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.195627+0000","last_undegraded":"2024-04-20T10:22:50.195627+0000","last_fullsized":"2024-04-20T10:22:50.195627+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.211706+0000","last_change":"2024-04-20T10:22:50.208086+0000","last_active":"2024-04-20T10:22:50.211706+0000","last_peered":"2024-04-20T10:22:50.211706+0000","last_clean":"2024-04-20T10:22:50.211706+0000","last_became_active":"2024-04-20T10:22:50.207645+0000","last_became_peered":"2024-04-20T10:22:50.207645+0000","last_unstale":"2024-04-20T10:22:50.211706+0000","last_undegraded":"2024-04-20T10:22:50.211706+0000","last_fullsized":"2024-04-20T10:22:50.211706+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:36:24.988490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.183462+0000","last_change":"2024-04-20T10:22:50.183462+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.183462+0000","last_undegraded":"2024-04-20T10:22:50.183462+0000","last_fullsized":"2024-04-20T10:22:50.183462+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.207581+0000","last_change":"2024-04-20T10:22:50.204532+0000","last_active":"2024-04-20T10:22:50.207581+0000","last_peered":"2024-04-20T10:22:50.207581+0000","last_clean":"2024-04-20T10:22:50.207581+0000","last_became_active":"2024-04-20T10:22:50.204230+0000","last_became_peered":"2024-04-20T10:22:50.204230+0000","last_unstale":"2024-04-20T10:22:50.207581+0000","last_undegraded":"2024-04-20T10:22:50.207581+0000","last_fullsized":"2024-04-20T10:22:50.207581+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:10:15.543826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.192569+0000","last_change":"2024-04-20T10:22:50.192569+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.192569+0000","last_undegraded":"2024-04-20T10:22:50.192569+0000","last_fullsized":"2024-04-20T10:22:50.192569+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.207873+0000","last_change":"2024-04-20T10:22:50.205333+0000","last_active":"2024-04-20T10:22:50.207873+0000","last_peered":"2024-04-20T10:22:50.207873+0000","last_clean":"2024-04-20T10:22:50.207873+0000","last_became_active":"2024-04-20T10:22:50.205029+0000","last_became_peered":"2024-04-20T10:22:50.205029+0000","last_unstale":"2024-04-20T10:22:50.207873+0000","last_undegraded":"2024-04-20T10:22:50.207873+0000","last_fullsized":"2024-04-20T10:22:50.207873+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:16:59.003634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.186710+0000","last_change":"2024-04-20T10:22:50.186710+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.186710+0000","last_undegraded":"2024-04-20T10:22:50.186710+0000","last_fullsized":"2024-04-20T10:22:50.186710+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.211920+0000","last_change":"2024-04-20T10:22:50.207305+0000","last_active":"2024-04-20T10:22:50.211920+0000","last_peered":"2024-04-20T10:22:50.211920+0000","last_clean":"2024-04-20T10:22:50.211920+0000","last_became_active":"2024-04-20T10:22:50.207018+0000","last_became_peered":"2024-04-20T10:22:50.207018+0000","last_unstale":"2024-04-20T10:22:50.211920+0000","last_undegraded":"2024-04-20T10:22:50.211920+0000","last_fullsized":"2024-04-20T10:22:50.211920+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:36:24.988490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.193141+0000","last_change":"2024-04-20T10:22:50.193141+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.193141+0000","last_undegraded":"2024-04-20T10:22:50.193141+0000","last_fullsized":"2024-04-20T10:22:50.193141+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.205759+0000","last_change":"2024-04-20T10:22:50.202177+0000","last_active":"2024-04-20T10:22:50.205759+0000","last_peered":"2024-04-20T10:22:50.205759+0000","last_clean":"2024-04-20T10:22:50.205759+0000","last_became_active":"2024-04-20T10:22:50.201840+0000","last_became_peered":"2024-04-20T10:22:50.201840+0000","last_unstale":"2024-04-20T10:22:50.205759+0000","last_undegraded":"2024-04-20T10:22:50.205759+0000","last_fullsized":"2024-04-20T10:22:50.205759+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:57:16.061872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.188832+0000","last_change":"2024-04-20T10:22:50.188832+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.188832+0000","last_undegraded":"2024-04-20T10:22:50.188832+0000","last_fullsized":"2024-04-20T10:22:50.188832+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.212443+0000","last_change":"2024-04-20T10:22:50.211068+0000","last_active":"2024-04-20T10:22:50.212443+0000","last_peered":"2024-04-20T10:22:50.212443+0000","last_clean":"2024-04-20T10:22:50.212443+0000","last_became_active":"2024-04-20T10:22:50.210709+0000","last_became_peered":"2024-04-20T10:22:50.210709+0000","last_unstale":"2024-04-20T10:22:50.212443+0000","last_undegraded":"2024-04-20T10:22:50.212443+0000","last_fullsized":"2024-04-20T10:22:50.212443+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:16:59.003634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.185316+0000","last_change":"2024-04-20T10:22:50.185316+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.185316+0000","last_undegraded":"2024-04-20T10:22:50.185316+0000","last_fullsized":"2024-04-20T10:22:50.185316+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.205815+0000","last_change":"2024-04-20T10:22:50.202082+0000","last_active":"2024-04-20T10:22:50.205815+0000","last_peered":"2024-04-20T10:22:50.205815+0000","last_clean":"2024-04-20T10:22:50.205815+0000","last_became_active":"2024-04-20T10:22:50.201785+0000","last_became_peered":"2024-04-20T10:22:50.201785+0000","last_unstale":"2024-04-20T10:22:50.205815+0000","last_undegraded":"2024-04-20T10:22:50.205815+0000","last_fullsized":"2024-04-20T10:22:50.205815+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:22:51.962842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.185906+0000","last_change":"2024-04-20T10:22:50.185906+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.185906+0000","last_undegraded":"2024-04-20T10:22:50.185906+0000","last_fullsized":"2024-04-20T10:22:50.185906+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.211884+0000","last_change":"2024-04-20T10:22:50.208162+0000","last_active":"2024-04-20T10:22:50.211884+0000","last_peered":"2024-04-20T10:22:50.211884+0000","last_clean":"2024-04-20T10:22:50.211884+0000","last_became_active":"2024-04-20T10:22:50.206236+0000","last_became_peered":"2024-04-20T10:22:50.206236+0000","last_unstale":"2024-04-20T10:22:50.211884+0000","last_undegraded":"2024-04-20T10:22:50.211884+0000","last_fullsized":"2024-04-20T10:22:50.211884+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:47:00.135513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.177757+0000","last_change":"2024-04-20T10:22:50.177757+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.177757+0000","last_undegraded":"2024-04-20T10:22:50.177757+0000","last_fullsized":"2024-04-20T10:22:50.177757+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.200401+0000","last_change":"2024-04-20T10:22:50.196202+0000","last_active":"2024-04-20T10:22:50.200401+0000","last_peered":"2024-04-20T10:22:50.200401+0000","last_clean":"2024-04-20T10:22:50.200401+0000","last_became_active":"2024-04-20T10:22:50.195933+0000","last_became_peered":"2024-04-20T10:22:50.195933+0000","last_unstale":"2024-04-20T10:22:50.200401+0000","last_undegraded":"2024-04-20T10:22:50.200401+0000","last_fullsized":"2024-04-20T10:22:50.200401+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:20:05.462979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.192693+0000","last_change":"2024-04-20T10:22:50.192693+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.192693+0000","last_undegraded":"2024-04-20T10:22:50.192693+0000","last_fullsized":"2024-04-20T10:22:50.192693+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.214543+0000","last_change":"2024-04-20T10:22:50.209570+0000","last_active":"2024-04-20T10:22:50.214543+0000","last_peered":"2024-04-20T10:22:50.214543+0000","last_clean":"2024-04-20T10:22:50.214543+0000","last_became_active":"2024-04-20T10:22:50.209281+0000","last_became_peered":"2024-04-20T10:22:50.209281+0000","last_unstale":"2024-04-20T10:22:50.214543+0000","last_undegraded":"2024-04-20T10:22:50.214543+0000","last_fullsized":"2024-04-20T10:22:50.214543+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:09:34.293198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.191996+0000","last_change":"2024-04-20T10:22:50.191996+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.191996+0000","last_undegraded":"2024-04-20T10:22:50.191996+0000","last_fullsized":"2024-04-20T10:22:50.191996+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.209812+0000","last_change":"2024-04-20T10:22:50.206722+0000","last_active":"2024-04-20T10:22:50.209812+0000","last_peered":"2024-04-20T10:22:50.209812+0000","last_clean":"2024-04-20T10:22:50.209812+0000","last_became_active":"2024-04-20T10:22:50.206395+0000","last_became_peered":"2024-04-20T10:22:50.206395+0000","last_unstale":"2024-04-20T10:22:50.209812+0000","last_undegraded":"2024-04-20T10:22:50.209812+0000","last_fullsized":"2024-04-20T10:22:50.209812+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:58:15.263333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.198022+0000","last_change":"2024-04-20T10:22:50.198022+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.198022+0000","last_undegraded":"2024-04-20T10:22:50.198022+0000","last_fullsized":"2024-04-20T10:22:50.198022+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.207715+0000","last_change":"2024-04-20T10:22:50.203958+0000","last_active":"2024-04-20T10:22:50.207715+0000","last_peered":"2024-04-20T10:22:50.207715+0000","last_clean":"2024-04-20T10:22:50.207715+0000","last_became_active":"2024-04-20T10:22:50.203499+0000","last_became_peered":"2024-04-20T10:22:50.203499+0000","last_unstale":"2024-04-20T10:22:50.207715+0000","last_undegraded":"2024-04-20T10:22:50.207715+0000","last_fullsized":"2024-04-20T10:22:50.207715+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:16:27.290919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.192536+0000","last_change":"2024-04-20T10:22:50.192536+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.192536+0000","last_undegraded":"2024-04-20T10:22:50.192536+0000","last_fullsized":"2024-04-20T10:22:50.192536+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.200458+0000","last_change":"2024-04-20T10:22:50.196624+0000","last_active":"2024-04-20T10:22:50.200458+0000","last_peered":"2024-04-20T10:22:50.200458+0000","last_clean":"2024-04-20T10:22:50.200458+0000","last_became_active":"2024-04-20T10:22:50.196295+0000","last_became_peered":"2024-04-20T10:22:50.196295+0000","last_unstale":"2024-04-20T10:22:50.200458+0000","last_undegraded":"2024-04-20T10:22:50.200458+0000","last_fullsized":"2024-04-20T10:22:50.200458+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:30:35.298814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.179317+0000","last_change":"2024-04-20T10:22:50.179317+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.179317+0000","last_undegraded":"2024-04-20T10:22:50.179317+0000","last_fullsized":"2024-04-20T10:22:50.179317+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.205746+0000","last_change":"2024-04-20T10:22:50.200183+0000","last_active":"2024-04-20T10:22:50.205746+0000","last_peered":"2024-04-20T10:22:50.205746+0000","last_clean":"2024-04-20T10:22:50.205746+0000","last_became_active":"2024-04-20T10:22:50.199884+0000","last_became_peered":"2024-04-20T10:22:50.199884+0000","last_unstale":"2024-04-20T10:22:50.205746+0000","last_undegraded":"2024-04-20T10:22:50.205746+0000","last_fullsized":"2024-04-20T10:22:50.205746+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:10:15.543826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.195676+0000","last_change":"2024-04-20T10:22:50.195676+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.195676+0000","last_undegraded":"2024-04-20T10:22:50.195676+0000","last_fullsized":"2024-04-20T10:22:50.195676+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.209575+0000","last_change":"2024-04-20T10:22:50.205765+0000","last_active":"2024-04-20T10:22:50.209575+0000","last_peered":"2024-04-20T10:22:50.209575+0000","last_clean":"2024-04-20T10:22:50.209575+0000","last_became_active":"2024-04-20T10:22:50.205454+0000","last_became_peered":"2024-04-20T10:22:50.205454+0000","last_unstale":"2024-04-20T10:22:50.209575+0000","last_undegraded":"2024-04-20T10:22:50.209575+0000","last_fullsized":"2024-04-20T10:22:50.209575+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:20:42.218198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.185184+0000","last_change":"2024-04-20T10:22:50.185184+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.185184+0000","last_undegraded":"2024-04-20T10:22:50.185184+0000","last_fullsized":"2024-04-20T10:22:50.185184+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.212433+0000","last_change":"2024-04-20T10:22:50.210848+0000","last_active":"2024-04-20T10:22:50.212433+0000","last_peered":"2024-04-20T10:22:50.212433+0000","last_clean":"2024-04-20T10:22:50.212433+0000","last_became_active":"2024-04-20T10:22:50.210382+0000","last_became_peered":"2024-04-20T10:22:50.210382+0000","last_unstale":"2024-04-20T10:22:50.212433+0000","last_undegraded":"2024-04-20T10:22:50.212433+0000","last_fullsized":"2024-04-20T10:22:50.212433+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:39:48.488461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.186954+0000","last_change":"2024-04-20T10:22:50.186954+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.186954+0000","last_undegraded":"2024-04-20T10:22:50.186954+0000","last_fullsized":"2024-04-20T10:22:50.186954+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.211775+0000","last_change":"2024-04-20T10:22:50.208371+0000","last_active":"2024-04-20T10:22:50.211775+0000","last_peered":"2024-04-20T10:22:50.211775+0000","last_clean":"2024-04-20T10:22:50.211775+0000","last_became_active":"2024-04-20T10:22:50.208102+0000","last_became_peered":"2024-04-20T10:22:50.208102+0000","last_unstale":"2024-04-20T10:22:50.211775+0000","last_undegraded":"2024-04-20T10:22:50.211775+0000","last_fullsized":"2024-04-20T10:22:50.211775+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:16:51.466273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.190353+0000","last_change":"2024-04-20T10:22:50.190353+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.190353+0000","last_undegraded":"2024-04-20T10:22:50.190353+0000","last_fullsized":"2024-04-20T10:22:50.190353+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.209177+0000","last_change":"2024-04-20T10:22:50.205722+0000","last_active":"2024-04-20T10:22:50.209177+0000","last_peered":"2024-04-20T10:22:50.209177+0000","last_clean":"2024-04-20T10:22:50.209177+0000","last_became_active":"2024-04-20T10:22:50.205402+0000","last_became_peered":"2024-04-20T10:22:50.205402+0000","last_unstale":"2024-04-20T10:22:50.209177+0000","last_undegraded":"2024-04-20T10:22:50.209177+0000","last_fullsized":"2024-04-20T10:22:50.209177+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:20:42.218198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.193228+0000","last_change":"2024-04-20T10:22:50.193228+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.193228+0000","last_undegraded":"2024-04-20T10:22:50.193228+0000","last_fullsized":"2024-04-20T10:22:50.193228+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-20T10:22:50.173967+0000","last_change":"2024-04-20T10:22:50.173967+0000","last_active":"2024-04-20T10:22:49.145911+0000","last_peered":"2024-04-20T10:22:49.145911+0000","last_clean":"2024-04-20T10:22:49.145911+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.173967+0000","last_undegraded":"2024-04-20T10:22:50.173967+0000","last_fullsized":"2024-04-20T10:22:50.173967+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":["2","7"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.180223+0000","last_change":"2024-04-20T10:22:50.180223+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.180223+0000","last_undegraded":"2024-04-20T10:22:50.180223+0000","last_fullsized":"2024-04-20T10:22:50.180223+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.207866+0000","last_change":"2024-04-20T10:22:50.205027+0000","last_active":"2024-04-20T10:22:50.207866+0000","last_peered":"2024-04-20T10:22:50.207866+0000","last_clean":"2024-04-20T10:22:50.207866+0000","last_became_active":"2024-04-20T10:22:50.204749+0000","last_became_peered":"2024-04-20T10:22:50.204749+0000","last_unstale":"2024-04-20T10:22:50.207866+0000","last_undegraded":"2024-04-20T10:22:50.207866+0000","last_fullsized":"2024-04-20T10:22:50.207866+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:39:48.488461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.188798+0000","last_change":"2024-04-20T10:22:50.188798+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.188798+0000","last_undegraded":"2024-04-20T10:22:50.188798+0000","last_fullsized":"2024-04-20T10:22:50.188798+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.200723+0000","last_change":"2024-04-20T10:22:50.194920+0000","last_active":"2024-04-20T10:22:50.200723+0000","last_peered":"2024-04-20T10:22:50.200723+0000","last_clean":"2024-04-20T10:22:50.200723+0000","last_became_active":"2024-04-20T10:22:50.194622+0000","last_became_peered":"2024-04-20T10:22:50.194622+0000","last_unstale":"2024-04-20T10:22:50.200723+0000","last_undegraded":"2024-04-20T10:22:50.200723+0000","last_fullsized":"2024-04-20T10:22:50.200723+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:30:02.849778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.181408+0000","last_change":"2024-04-20T10:22:50.181408+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.181408+0000","last_undegraded":"2024-04-20T10:22:50.181408+0000","last_fullsized":"2024-04-20T10:22:50.181408+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-20T10:22:50.173618+0000","last_change":"2024-04-20T10:22:50.173618+0000","last_active":"2024-04-20T10:22:49.145911+0000","last_peered":"2024-04-20T10:22:49.145911+0000","last_clean":"2024-04-20T10:22:49.145911+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.173618+0000","last_undegraded":"2024-04-20T10:22:50.173618+0000","last_fullsized":"2024-04-20T10:22:50.173618+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.195699+0000","last_change":"2024-04-20T10:22:50.195699+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.195699+0000","last_undegraded":"2024-04-20T10:22:50.195699+0000","last_fullsized":"2024-04-20T10:22:50.195699+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.208591+0000","last_change":"2024-04-20T10:22:50.205146+0000","last_active":"2024-04-20T10:22:50.208591+0000","last_peered":"2024-04-20T10:22:50.208591+0000","last_clean":"2024-04-20T10:22:50.208591+0000","last_became_active":"2024-04-20T10:22:50.204772+0000","last_became_peered":"2024-04-20T10:22:50.204772+0000","last_unstale":"2024-04-20T10:22:50.208591+0000","last_undegraded":"2024-04-20T10:22:50.208591+0000","last_fullsized":"2024-04-20T10:22:50.208591+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:22:51.962842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.179884+0000","last_change":"2024-04-20T10:22:50.179884+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.179884+0000","last_undegraded":"2024-04-20T10:22:50.179884+0000","last_fullsized":"2024-04-20T10:22:50.179884+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.200768+0000","last_change":"2024-04-20T10:22:50.194938+0000","last_active":"2024-04-20T10:22:50.200768+0000","last_peered":"2024-04-20T10:22:50.200768+0000","last_clean":"2024-04-20T10:22:50.200768+0000","last_became_active":"2024-04-20T10:22:50.194631+0000","last_became_peered":"2024-04-20T10:22:50.194631+0000","last_unstale":"2024-04-20T10:22:50.200768+0000","last_undegraded":"2024-04-20T10:22:50.200768+0000","last_fullsized":"2024-04-20T10:22:50.200768+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:32:32.746673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.184467+0000","last_change":"2024-04-20T10:22:50.184467+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.184467+0000","last_undegraded":"2024-04-20T10:22:50.184467+0000","last_fullsized":"2024-04-20T10:22:50.184467+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.203938+0000","last_change":"2024-04-20T10:22:50.200577+0000","last_active":"2024-04-20T10:22:50.203938+0000","last_peered":"2024-04-20T10:22:50.203938+0000","last_clean":"2024-04-20T10:22:50.203938+0000","last_became_active":"2024-04-20T10:22:50.199510+0000","last_became_peered":"2024-04-20T10:22:50.199510+0000","last_unstale":"2024-04-20T10:22:50.203938+0000","last_undegraded":"2024-04-20T10:22:50.203938+0000","last_fullsized":"2024-04-20T10:22:50.203938+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:26:16.280439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.191998+0000","last_change":"2024-04-20T10:22:50.191998+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.191998+0000","last_undegraded":"2024-04-20T10:22:50.191998+0000","last_fullsized":"2024-04-20T10:22:50.191998+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-20T10:22:50.174033+0000","last_change":"2024-04-20T10:22:50.174033+0000","last_active":"2024-04-20T10:22:49.145911+0000","last_peered":"2024-04-20T10:22:49.145911+0000","last_clean":"2024-04-20T10:22:49.145911+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.174033+0000","last_undegraded":"2024-04-20T10:22:50.174033+0000","last_fullsized":"2024-04-20T10:22:50.174033+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":["2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.183765+0000","last_change":"2024-04-20T10:22:50.183765+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.183765+0000","last_undegraded":"2024-04-20T10:22:50.183765+0000","last_fullsized":"2024-04-20T10:22:50.183765+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.200821+0000","last_change":"2024-04-20T10:22:50.196011+0000","last_active":"2024-04-20T10:22:50.200821+0000","last_peered":"2024-04-20T10:22:50.200821+0000","last_clean":"2024-04-20T10:22:50.200821+0000","last_became_active":"2024-04-20T10:22:50.195748+0000","last_became_peered":"2024-04-20T10:22:50.195748+0000","last_unstale":"2024-04-20T10:22:50.200821+0000","last_undegraded":"2024-04-20T10:22:50.200821+0000","last_fullsized":"2024-04-20T10:22:50.200821+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:36:24.988490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.184945+0000","last_change":"2024-04-20T10:22:50.184945+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.184945+0000","last_undegraded":"2024-04-20T10:22:50.184945+0000","last_fullsized":"2024-04-20T10:22:50.184945+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.205855+0000","last_change":"2024-04-20T10:22:50.203809+0000","last_active":"2024-04-20T10:22:50.205855+0000","last_peered":"2024-04-20T10:22:50.205855+0000","last_clean":"2024-04-20T10:22:50.205855+0000","last_became_active":"2024-04-20T10:22:50.203451+0000","last_became_peered":"2024-04-20T10:22:50.203451+0000","last_unstale":"2024-04-20T10:22:50.205855+0000","last_undegraded":"2024-04-20T10:22:50.205855+0000","last_fullsized":"2024-04-20T10:22:50.205855+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:47:00.135513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.180703+0000","last_change":"2024-04-20T10:22:50.180703+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.180703+0000","last_undegraded":"2024-04-20T10:22:50.180703+0000","last_fullsized":"2024-04-20T10:22:50.180703+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.203328+0000","last_change":"2024-04-20T10:22:50.197399+0000","last_active":"2024-04-20T10:22:50.203328+0000","last_peered":"2024-04-20T10:22:50.203328+0000","last_clean":"2024-04-20T10:22:50.203328+0000","last_became_active":"2024-04-20T10:22:50.197015+0000","last_became_peered":"2024-04-20T10:22:50.197015+0000","last_unstale":"2024-04-20T10:22:50.203328+0000","last_undegraded":"2024-04-20T10:22:50.203328+0000","last_fullsized":"2024-04-20T10:22:50.203328+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:41:01.514306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.187559+0000","last_change":"2024-04-20T10:22:50.187559+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.187559+0000","last_undegraded":"2024-04-20T10:22:50.187559+0000","last_fullsized":"2024-04-20T10:22:50.187559+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.208900+0000","last_change":"2024-04-20T10:22:50.203369+0000","last_active":"2024-04-20T10:22:50.208900+0000","last_peered":"2024-04-20T10:22:50.208900+0000","last_clean":"2024-04-20T10:22:50.208900+0000","last_became_active":"2024-04-20T10:22:50.203083+0000","last_became_peered":"2024-04-20T10:22:50.203083+0000","last_unstale":"2024-04-20T10:22:50.208900+0000","last_undegraded":"2024-04-20T10:22:50.208900+0000","last_fullsized":"2024-04-20T10:22:50.208900+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:41:01.514306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.189625+0000","last_change":"2024-04-20T10:22:50.189625+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.189625+0000","last_undegraded":"2024-04-20T10:22:50.189625+0000","last_fullsized":"2024-04-20T10:22:50.189625+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.205556+0000","last_change":"2024-04-20T10:22:50.198717+0000","last_active":"2024-04-20T10:22:50.205556+0000","last_peered":"2024-04-20T10:22:50.205556+0000","last_clean":"2024-04-20T10:22:50.205556+0000","last_became_active":"2024-04-20T10:22:50.197112+0000","last_became_peered":"2024-04-20T10:22:50.197112+0000","last_unstale":"2024-04-20T10:22:50.205556+0000","last_undegraded":"2024-04-20T10:22:50.205556+0000","last_fullsized":"2024-04-20T10:22:50.205556+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:36:31.551385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.189620+0000","last_change":"2024-04-20T10:22:50.189620+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.189620+0000","last_undegraded":"2024-04-20T10:22:50.189620+0000","last_fullsized":"2024-04-20T10:22:50.189620+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.206576+0000","last_change":"2024-04-20T10:22:50.203854+0000","last_active":"2024-04-20T10:22:50.206576+0000","last_peered":"2024-04-20T10:22:50.206576+0000","last_clean":"2024-04-20T10:22:50.206576+0000","last_became_active":"2024-04-20T10:22:50.203439+0000","last_became_peered":"2024-04-20T10:22:50.203439+0000","last_unstale":"2024-04-20T10:22:50.206576+0000","last_undegraded":"2024-04-20T10:22:50.206576+0000","last_fullsized":"2024-04-20T10:22:50.206576+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:10:15.543826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":198,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.195263+0000","last_change":"2024-04-20T10:20:52.916129+0000","last_active":"2024-04-20T10:22:50.195263+0000","last_peered":"2024-04-20T10:22:50.195263+0000","last_clean":"2024-04-20T10:22:50.195263+0000","last_became_active":"2024-04-20T10:20:52.913922+0000","last_became_peered":"2024-04-20T10:20:52.913922+0000","last_unstale":"2024-04-20T10:22:50.195263+0000","last_undegraded":"2024-04-20T10:22:50.195263+0000","last_fullsized":"2024-04-20T10:22:50.195263+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:20:51.267588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:20:51.267588+0000","last_clean_scrub_stamp":"2024-04-20T10:20:51.267588+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:30:44.465836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542170,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":1003520,"data_stored":763189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":409600,"data_stored":172821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542170,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":409600,"data_stored":172821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542170,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":409600,"data_stored":172821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542170,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":421888,"data_stored":179488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542170,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":409600,"data_stored":172821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542172,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":409600,"data_stored":172821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542172,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":1003520,"data_stored":763189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T10:22:56.134 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T10:22:56.373 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.373+0000 7fe570a00640 1 Processor -- start 2024-04-20T10:22:56.373 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.373+0000 7fe570a00640 1 -- start start 2024-04-20T10:22:56.374 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.375+0000 7fe570a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe56c156c20 0x7fe56c06b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:56.375 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.375+0000 7fe570a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe56c0d0360 0x7fe56c0d07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:56.375 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.375+0000 7fe570a00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe56c141c60 con 0x7fe56c15ca40 2024-04-20T10:22:56.375 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.375+0000 7fe570a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7fe56c0d0d10 con 0x7fe56c156c20 2024-04-20T10:22:56.375 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.375+0000 7fe570a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7fe56c0adab0 con 0x7fe56c0d0360 2024-04-20T10:22:56.375 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.375+0000 7fe569e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe56c156c20 0x7fe56c06b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:56.375 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.375+0000 7fe569e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7fe56c156c20 0x7fe56c06b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33224/0 (socket says 172.21.15.121:33224) 2024-04-20T10:22:56.375 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.375+0000 7fe569e00640 1 -- 172.21.15.121:0/144192362 learned_addr learned my addr 172.21.15.121:0/144192362 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:56.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.376+0000 7fe569e00640 1 -- 172.21.15.121:0/144192362 >> v2:172.21.15.123:3300/0 conn(0x7fe56c0d0360 msgr2=0x7fe56c0d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:56.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.376+0000 7fe56a800640 1 --2- 172.21.15.121:0/144192362 >> v2:172.21.15.123:3300/0 conn(0x7fe56c0d0360 0x7fe56c0d07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:56.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.376+0000 7fe569e00640 1 --2- 172.21.15.121:0/144192362 >> v2:172.21.15.123:3300/0 conn(0x7fe56c0d0360 0x7fe56c0d07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:56.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.376+0000 7fe569e00640 1 -- 172.21.15.121:0/144192362 >> v1:172.21.15.123:6789/0 conn(0x7fe56c15ca40 legacy=0x7fe56c0ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:56.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.376+0000 7fe569e00640 1 -- 172.21.15.121:0/144192362 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe56c15cf90 con 0x7fe56c156c20 2024-04-20T10:22:56.376 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.376+0000 7fe56a800640 1 --2- 172.21.15.121:0/144192362 >> v2:172.21.15.123:3300/0 conn(0x7fe56c0d0360 0x7fe56c0d07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:56.377 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.377+0000 7fe569e00640 1 --2- 172.21.15.121:0/144192362 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe56c156c20 0x7fe56c06b900 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe56c06ff40 tx=0x7fe55c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20f69de7ba1b995 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:56.377 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.377+0000 7fe569400640 1 -- 172.21.15.121:0/144192362 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c035020 con 0x7fe56c156c20 2024-04-20T10:22:56.378 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.377+0000 7fe569400640 1 -- 172.21.15.121:0/144192362 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe55c031870 con 0x7fe56c156c20 2024-04-20T10:22:56.378 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.377+0000 7fe569400640 1 -- 172.21.15.121:0/144192362 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c031c40 con 0x7fe56c156c20 2024-04-20T10:22:56.378 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.378+0000 7fe570a00640 1 -- 172.21.15.121:0/144192362 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe56c156c20 msgr2=0x7fe56c06b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:56.378 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.378+0000 7fe570a00640 1 --2- 172.21.15.121:0/144192362 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe56c156c20 0x7fe56c06b900 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe56c06ff40 tx=0x7fe55c02b910 comp rx=0 tx=0).stop 2024-04-20T10:22:56.378 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.378+0000 7fe570a00640 1 -- 172.21.15.121:0/144192362 shutdown_connections 2024-04-20T10:22:56.378 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.378+0000 7fe570a00640 1 --2- 172.21.15.121:0/144192362 >> v2:172.21.15.123:3300/0 conn(0x7fe56c0d0360 0x7fe56c0d07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:56.378 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.378+0000 7fe570a00640 1 --2- 172.21.15.121:0/144192362 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe56c156c20 0x7fe56c06b900 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:56.378 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.378+0000 7fe570a00640 1 -- 172.21.15.121:0/144192362 >> 172.21.15.121:0/144192362 conn(0x7fe56c0b3350 msgr2=0x7fe56c0b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:56.378 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.378+0000 7fe570a00640 1 -- 172.21.15.121:0/144192362 shutdown_connections 2024-04-20T10:22:56.378 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.378+0000 7fe570a00640 1 -- 172.21.15.121:0/144192362 wait complete. 2024-04-20T10:22:56.379 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.379+0000 7fe570a00640 1 Processor -- start 2024-04-20T10:22:56.379 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.380+0000 7fe570a00640 1 -- start start 2024-04-20T10:22:56.381 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.380+0000 7fe570a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe56c0d0360 0x7fe56c099340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:56.381 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.380+0000 7fe570a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe56c156c20 0x7fe56c0ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:56.381 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.381+0000 7fe570a00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe56c15ca40 0x7fe56c0cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:56.381 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.381+0000 7fe570a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c15d830 con 0x7fe56c156c20 2024-04-20T10:22:56.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.381+0000 7fe570a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c15d550 con 0x7fe56c0d0360 2024-04-20T10:22:56.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.381+0000 7fe570a00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7fe56c06be40 con 0x7fe56c15ca40 2024-04-20T10:22:56.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.381+0000 7fe563e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe56c156c20 0x7fe56c0ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:56.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.381+0000 7fe563e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe56c156c20 0x7fe56c0ccca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33226/0 (socket says 172.21.15.121:33226) 2024-04-20T10:22:56.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.381+0000 7fe563e00640 1 -- 172.21.15.121:0/840670155 learned_addr learned my addr 172.21.15.121:0/840670155 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:56.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.381+0000 7fe563e00640 1 -- 172.21.15.121:0/840670155 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe56c15ca40 msgr2=0x7fe56c0cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:56.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.381+0000 7fe563e00640 1 --2- 172.21.15.121:0/840670155 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe56c15ca40 0x7fe56c0cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:56.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.381+0000 7fe563e00640 1 -- 172.21.15.121:0/840670155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe56c0d0360 msgr2=0x7fe56c099340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:56.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.381+0000 7fe563e00640 1 --2- 172.21.15.121:0/840670155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe56c0d0360 0x7fe56c099340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:56.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.381+0000 7fe563e00640 1 -- 172.21.15.121:0/840670155 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe56c08c500 con 0x7fe56c156c20 2024-04-20T10:22:56.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.381+0000 7fe563e00640 1 --2- 172.21.15.121:0/840670155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe56c156c20 0x7fe56c0ccca0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe55400b3e0 tx=0x7fe55400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:56.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.382+0000 7fe562000640 1 -- 172.21.15.121:0/840670155 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe554015020 con 0x7fe56c156c20 2024-04-20T10:22:56.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.382+0000 7fe562000640 1 -- 172.21.15.121:0/840670155 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe554011850 con 0x7fe56c156c20 2024-04-20T10:22:56.382 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.382+0000 7fe570a00640 1 -- 172.21.15.121:0/840670155 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe56c08c7c0 con 0x7fe56c156c20 2024-04-20T10:22:56.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.382+0000 7fe562000640 1 -- 172.21.15.121:0/840670155 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe554011c40 con 0x7fe56c156c20 2024-04-20T10:22:56.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.382+0000 7fe570a00640 1 -- 172.21.15.121:0/840670155 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe56c0cd1e0 con 0x7fe56c156c20 2024-04-20T10:22:56.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.383+0000 7fe570a00640 1 -- 172.21.15.121:0/840670155 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe56c0cc470 con 0x7fe56c156c20 2024-04-20T10:22:56.383 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.384+0000 7fe562000640 1 -- 172.21.15.121:0/840670155 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe554018020 con 0x7fe56c156c20 2024-04-20T10:22:56.384 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.385+0000 7fe562000640 1 --2- 172.21.15.121:0/840670155 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe54806c8f0 0x7fe54806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:56.384 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.385+0000 7fe562000640 1 -- 172.21.15.121:0/840670155 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fe55408bcb0 con 0x7fe56c156c20 2024-04-20T10:22:56.384 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.385+0000 7fe569e00640 1 --2- 172.21.15.121:0/840670155 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe54806c8f0 0x7fe54806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:56.385 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.386+0000 7fe569e00640 1 --2- 172.21.15.121:0/840670155 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe54806c8f0 0x7fe54806edb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe55c036000 tx=0x7fe55c030000 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:56.388 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.389+0000 7fe562000640 1 -- 172.21.15.121:0/840670155 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe56c0cc470 con 0x7fe56c156c20 2024-04-20T10:22:56.569 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.569+0000 7fe570a00640 1 -- 172.21.15.121:0/840670155 --> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe56c15db70 con 0x7fe54806c8f0 2024-04-20T10:22:56.576 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.576+0000 7fe562000640 1 -- 172.21.15.121:0/840670155 <== mgr.4103 v2:172.21.15.121:6832/1408233272 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246423 (secure 0 0 0) 0x7fe56c15db70 con 0x7fe54806c8f0 2024-04-20T10:22:56.579 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:22:56.580 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-04-20T10:22:56.582 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.581+0000 7fe570a00640 1 -- 172.21.15.121:0/840670155 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe54806c8f0 msgr2=0x7fe54806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:56.582 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.581+0000 7fe570a00640 1 --2- 172.21.15.121:0/840670155 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe54806c8f0 0x7fe54806edb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe55c036000 tx=0x7fe55c030000 comp rx=0 tx=0).stop 2024-04-20T10:22:56.582 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.581+0000 7fe570a00640 1 -- 172.21.15.121:0/840670155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe56c156c20 msgr2=0x7fe56c0ccca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:56.582 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.581+0000 7fe570a00640 1 --2- 172.21.15.121:0/840670155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe56c156c20 0x7fe56c0ccca0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe55400b3e0 tx=0x7fe55400b8a0 comp rx=0 tx=0).stop 2024-04-20T10:22:56.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.581+0000 7fe570a00640 1 -- 172.21.15.121:0/840670155 shutdown_connections 2024-04-20T10:22:56.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.581+0000 7fe570a00640 1 --2- 172.21.15.121:0/840670155 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7fe56c15ca40 0x7fe56c0cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:56.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.581+0000 7fe570a00640 1 --2- 172.21.15.121:0/840670155 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7fe56c156c20 0x7fe56c0ccca0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:56.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.581+0000 7fe570a00640 1 --2- 172.21.15.121:0/840670155 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7fe54806c8f0 0x7fe54806edb0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:56.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.581+0000 7fe570a00640 1 --2- 172.21.15.121:0/840670155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe56c0d0360 0x7fe56c099340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:56.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.581+0000 7fe570a00640 1 -- 172.21.15.121:0/840670155 >> 172.21.15.121:0/840670155 conn(0x7fe56c0b3350 msgr2=0x7fe56c0afce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:56.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.581+0000 7fe570a00640 1 -- 172.21.15.121:0/840670155 shutdown_connections 2024-04-20T10:22:56.583 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.582+0000 7fe570a00640 1 -- 172.21.15.121:0/840670155 wait complete. 2024-04-20T10:22:56.599 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":95,"stamp":"2024-04-20T10:22:55.147880+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":7,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220536,"kb_used_data":4372,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754184,"statfs":{"total":773094113280,"available":772868284416,"internally_reserved":0,"allocated":4476928,"data_stored":2569971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.984225"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.187152+0000","last_change":"2024-04-20T10:22:50.187152+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.187152+0000","last_undegraded":"2024-04-20T10:22:50.187152+0000","last_fullsized":"2024-04-20T10:22:50.187152+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.186798+0000","last_change":"2024-04-20T10:22:50.186798+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.186798+0000","last_undegraded":"2024-04-20T10:22:50.186798+0000","last_fullsized":"2024-04-20T10:22:50.186798+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.189601+0000","last_change":"2024-04-20T10:22:50.189601+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.189601+0000","last_undegraded":"2024-04-20T10:22:50.189601+0000","last_fullsized":"2024-04-20T10:22:50.189601+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.181311+0000","last_change":"2024-04-20T10:22:50.181311+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.181311+0000","last_undegraded":"2024-04-20T10:22:50.181311+0000","last_fullsized":"2024-04-20T10:22:50.181311+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.184441+0000","last_change":"2024-04-20T10:22:50.184441+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.184441+0000","last_undegraded":"2024-04-20T10:22:50.184441+0000","last_fullsized":"2024-04-20T10:22:50.184441+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.182950+0000","last_change":"2024-04-20T10:22:50.182950+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.182950+0000","last_undegraded":"2024-04-20T10:22:50.182950+0000","last_fullsized":"2024-04-20T10:22:50.182950+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.192894+0000","last_change":"2024-04-20T10:22:50.192894+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.192894+0000","last_undegraded":"2024-04-20T10:22:50.192894+0000","last_fullsized":"2024-04-20T10:22:50.192894+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.181902+0000","last_change":"2024-04-20T10:22:50.181902+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.181902+0000","last_undegraded":"2024-04-20T10:22:50.181902+0000","last_fullsized":"2024-04-20T10:22:50.181902+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.192533+0000","last_change":"2024-04-20T10:22:50.192533+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.192533+0000","last_undegraded":"2024-04-20T10:22:50.192533+0000","last_fullsized":"2024-04-20T10:22:50.192533+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.194398+0000","last_change":"2024-04-20T10:22:50.194398+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.194398+0000","last_undegraded":"2024-04-20T10:22:50.194398+0000","last_fullsized":"2024-04-20T10:22:50.194398+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.194904+0000","last_change":"2024-04-20T10:22:50.194904+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.194904+0000","last_undegraded":"2024-04-20T10:22:50.194904+0000","last_fullsized":"2024-04-20T10:22:50.194904+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.192601+0000","last_change":"2024-04-20T10:22:50.192601+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.192601+0000","last_undegraded":"2024-04-20T10:22:50.192601+0000","last_fullsized":"2024-04-20T10:22:50.192601+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.197754+0000","last_change":"2024-04-20T10:22:50.197754+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.197754+0000","last_undegraded":"2024-04-20T10:22:50.197754+0000","last_fullsized":"2024-04-20T10:22:50.197754+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.188557+0000","last_change":"2024-04-20T10:22:50.188557+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.188557+0000","last_undegraded":"2024-04-20T10:22:50.188557+0000","last_fullsized":"2024-04-20T10:22:50.188557+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.192196+0000","last_change":"2024-04-20T10:22:50.192196+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.192196+0000","last_undegraded":"2024-04-20T10:22:50.192196+0000","last_fullsized":"2024-04-20T10:22:50.192196+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.190134+0000","last_change":"2024-04-20T10:22:50.190134+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.190134+0000","last_undegraded":"2024-04-20T10:22:50.190134+0000","last_fullsized":"2024-04-20T10:22:50.190134+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.189486+0000","last_change":"2024-04-20T10:22:50.189486+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.189486+0000","last_undegraded":"2024-04-20T10:22:50.189486+0000","last_fullsized":"2024-04-20T10:22:50.189486+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.177303+0000","last_change":"2024-04-20T10:22:50.177303+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.177303+0000","last_undegraded":"2024-04-20T10:22:50.177303+0000","last_fullsized":"2024-04-20T10:22:50.177303+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.182415+0000","last_change":"2024-04-20T10:22:50.182415+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.182415+0000","last_undegraded":"2024-04-20T10:22:50.182415+0000","last_fullsized":"2024-04-20T10:22:50.182415+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.177312+0000","last_change":"2024-04-20T10:22:50.177312+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.177312+0000","last_undegraded":"2024-04-20T10:22:50.177312+0000","last_fullsized":"2024-04-20T10:22:50.177312+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.195671+0000","last_change":"2024-04-20T10:22:50.195671+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.195671+0000","last_undegraded":"2024-04-20T10:22:50.195671+0000","last_fullsized":"2024-04-20T10:22:50.195671+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.186598+0000","last_change":"2024-04-20T10:22:50.186598+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.186598+0000","last_undegraded":"2024-04-20T10:22:50.186598+0000","last_fullsized":"2024-04-20T10:22:50.186598+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.190727+0000","last_change":"2024-04-20T10:22:50.190727+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.190727+0000","last_undegraded":"2024-04-20T10:22:50.190727+0000","last_fullsized":"2024-04-20T10:22:50.190727+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.182533+0000","last_change":"2024-04-20T10:22:50.182533+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.182533+0000","last_undegraded":"2024-04-20T10:22:50.182533+0000","last_fullsized":"2024-04-20T10:22:50.182533+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.181275+0000","last_change":"2024-04-20T10:22:50.181275+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.181275+0000","last_undegraded":"2024-04-20T10:22:50.181275+0000","last_fullsized":"2024-04-20T10:22:50.181275+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.182908+0000","last_change":"2024-04-20T10:22:50.182908+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.182908+0000","last_undegraded":"2024-04-20T10:22:50.182908+0000","last_fullsized":"2024-04-20T10:22:50.182908+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.177500+0000","last_change":"2024-04-20T10:22:50.177500+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.177500+0000","last_undegraded":"2024-04-20T10:22:50.177500+0000","last_fullsized":"2024-04-20T10:22:50.177500+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.187253+0000","last_change":"2024-04-20T10:22:50.187253+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.187253+0000","last_undegraded":"2024-04-20T10:22:50.187253+0000","last_fullsized":"2024-04-20T10:22:50.187253+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.177359+0000","last_change":"2024-04-20T10:22:50.177359+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.177359+0000","last_undegraded":"2024-04-20T10:22:50.177359+0000","last_fullsized":"2024-04-20T10:22:50.177359+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.179150+0000","last_change":"2024-04-20T10:22:50.179150+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.179150+0000","last_undegraded":"2024-04-20T10:22:50.179150+0000","last_fullsized":"2024-04-20T10:22:50.179150+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.189823+0000","last_change":"2024-04-20T10:22:50.189823+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.189823+0000","last_undegraded":"2024-04-20T10:22:50.189823+0000","last_fullsized":"2024-04-20T10:22:50.189823+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.179894+0000","last_change":"2024-04-20T10:22:50.179894+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.179894+0000","last_undegraded":"2024-04-20T10:22:50.179894+0000","last_fullsized":"2024-04-20T10:22:50.179894+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.195627+0000","last_change":"2024-04-20T10:22:50.195627+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.195627+0000","last_undegraded":"2024-04-20T10:22:50.195627+0000","last_fullsized":"2024-04-20T10:22:50.195627+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.211706+0000","last_change":"2024-04-20T10:22:50.208086+0000","last_active":"2024-04-20T10:22:50.211706+0000","last_peered":"2024-04-20T10:22:50.211706+0000","last_clean":"2024-04-20T10:22:50.211706+0000","last_became_active":"2024-04-20T10:22:50.207645+0000","last_became_peered":"2024-04-20T10:22:50.207645+0000","last_unstale":"2024-04-20T10:22:50.211706+0000","last_undegraded":"2024-04-20T10:22:50.211706+0000","last_fullsized":"2024-04-20T10:22:50.211706+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:36:24.988490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.183462+0000","last_change":"2024-04-20T10:22:50.183462+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.183462+0000","last_undegraded":"2024-04-20T10:22:50.183462+0000","last_fullsized":"2024-04-20T10:22:50.183462+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.207581+0000","last_change":"2024-04-20T10:22:50.204532+0000","last_active":"2024-04-20T10:22:50.207581+0000","last_peered":"2024-04-20T10:22:50.207581+0000","last_clean":"2024-04-20T10:22:50.207581+0000","last_became_active":"2024-04-20T10:22:50.204230+0000","last_became_peered":"2024-04-20T10:22:50.204230+0000","last_unstale":"2024-04-20T10:22:50.207581+0000","last_undegraded":"2024-04-20T10:22:50.207581+0000","last_fullsized":"2024-04-20T10:22:50.207581+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:10:15.543826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.192569+0000","last_change":"2024-04-20T10:22:50.192569+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.192569+0000","last_undegraded":"2024-04-20T10:22:50.192569+0000","last_fullsized":"2024-04-20T10:22:50.192569+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.207873+0000","last_change":"2024-04-20T10:22:50.205333+0000","last_active":"2024-04-20T10:22:50.207873+0000","last_peered":"2024-04-20T10:22:50.207873+0000","last_clean":"2024-04-20T10:22:50.207873+0000","last_became_active":"2024-04-20T10:22:50.205029+0000","last_became_peered":"2024-04-20T10:22:50.205029+0000","last_unstale":"2024-04-20T10:22:50.207873+0000","last_undegraded":"2024-04-20T10:22:50.207873+0000","last_fullsized":"2024-04-20T10:22:50.207873+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:16:59.003634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.186710+0000","last_change":"2024-04-20T10:22:50.186710+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.186710+0000","last_undegraded":"2024-04-20T10:22:50.186710+0000","last_fullsized":"2024-04-20T10:22:50.186710+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.211920+0000","last_change":"2024-04-20T10:22:50.207305+0000","last_active":"2024-04-20T10:22:50.211920+0000","last_peered":"2024-04-20T10:22:50.211920+0000","last_clean":"2024-04-20T10:22:50.211920+0000","last_became_active":"2024-04-20T10:22:50.207018+0000","last_became_peered":"2024-04-20T10:22:50.207018+0000","last_unstale":"2024-04-20T10:22:50.211920+0000","last_undegraded":"2024-04-20T10:22:50.211920+0000","last_fullsized":"2024-04-20T10:22:50.211920+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:36:24.988490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.193141+0000","last_change":"2024-04-20T10:22:50.193141+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.193141+0000","last_undegraded":"2024-04-20T10:22:50.193141+0000","last_fullsized":"2024-04-20T10:22:50.193141+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.205759+0000","last_change":"2024-04-20T10:22:50.202177+0000","last_active":"2024-04-20T10:22:50.205759+0000","last_peered":"2024-04-20T10:22:50.205759+0000","last_clean":"2024-04-20T10:22:50.205759+0000","last_became_active":"2024-04-20T10:22:50.201840+0000","last_became_peered":"2024-04-20T10:22:50.201840+0000","last_unstale":"2024-04-20T10:22:50.205759+0000","last_undegraded":"2024-04-20T10:22:50.205759+0000","last_fullsized":"2024-04-20T10:22:50.205759+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:57:16.061872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.188832+0000","last_change":"2024-04-20T10:22:50.188832+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.188832+0000","last_undegraded":"2024-04-20T10:22:50.188832+0000","last_fullsized":"2024-04-20T10:22:50.188832+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.212443+0000","last_change":"2024-04-20T10:22:50.211068+0000","last_active":"2024-04-20T10:22:50.212443+0000","last_peered":"2024-04-20T10:22:50.212443+0000","last_clean":"2024-04-20T10:22:50.212443+0000","last_became_active":"2024-04-20T10:22:50.210709+0000","last_became_peered":"2024-04-20T10:22:50.210709+0000","last_unstale":"2024-04-20T10:22:50.212443+0000","last_undegraded":"2024-04-20T10:22:50.212443+0000","last_fullsized":"2024-04-20T10:22:50.212443+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:16:59.003634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.185316+0000","last_change":"2024-04-20T10:22:50.185316+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.185316+0000","last_undegraded":"2024-04-20T10:22:50.185316+0000","last_fullsized":"2024-04-20T10:22:50.185316+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.205815+0000","last_change":"2024-04-20T10:22:50.202082+0000","last_active":"2024-04-20T10:22:50.205815+0000","last_peered":"2024-04-20T10:22:50.205815+0000","last_clean":"2024-04-20T10:22:50.205815+0000","last_became_active":"2024-04-20T10:22:50.201785+0000","last_became_peered":"2024-04-20T10:22:50.201785+0000","last_unstale":"2024-04-20T10:22:50.205815+0000","last_undegraded":"2024-04-20T10:22:50.205815+0000","last_fullsized":"2024-04-20T10:22:50.205815+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:22:51.962842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.185906+0000","last_change":"2024-04-20T10:22:50.185906+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.185906+0000","last_undegraded":"2024-04-20T10:22:50.185906+0000","last_fullsized":"2024-04-20T10:22:50.185906+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.211884+0000","last_change":"2024-04-20T10:22:50.208162+0000","last_active":"2024-04-20T10:22:50.211884+0000","last_peered":"2024-04-20T10:22:50.211884+0000","last_clean":"2024-04-20T10:22:50.211884+0000","last_became_active":"2024-04-20T10:22:50.206236+0000","last_became_peered":"2024-04-20T10:22:50.206236+0000","last_unstale":"2024-04-20T10:22:50.211884+0000","last_undegraded":"2024-04-20T10:22:50.211884+0000","last_fullsized":"2024-04-20T10:22:50.211884+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:47:00.135513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.177757+0000","last_change":"2024-04-20T10:22:50.177757+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.177757+0000","last_undegraded":"2024-04-20T10:22:50.177757+0000","last_fullsized":"2024-04-20T10:22:50.177757+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.200401+0000","last_change":"2024-04-20T10:22:50.196202+0000","last_active":"2024-04-20T10:22:50.200401+0000","last_peered":"2024-04-20T10:22:50.200401+0000","last_clean":"2024-04-20T10:22:50.200401+0000","last_became_active":"2024-04-20T10:22:50.195933+0000","last_became_peered":"2024-04-20T10:22:50.195933+0000","last_unstale":"2024-04-20T10:22:50.200401+0000","last_undegraded":"2024-04-20T10:22:50.200401+0000","last_fullsized":"2024-04-20T10:22:50.200401+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:20:05.462979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.192693+0000","last_change":"2024-04-20T10:22:50.192693+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.192693+0000","last_undegraded":"2024-04-20T10:22:50.192693+0000","last_fullsized":"2024-04-20T10:22:50.192693+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.214543+0000","last_change":"2024-04-20T10:22:50.209570+0000","last_active":"2024-04-20T10:22:50.214543+0000","last_peered":"2024-04-20T10:22:50.214543+0000","last_clean":"2024-04-20T10:22:50.214543+0000","last_became_active":"2024-04-20T10:22:50.209281+0000","last_became_peered":"2024-04-20T10:22:50.209281+0000","last_unstale":"2024-04-20T10:22:50.214543+0000","last_undegraded":"2024-04-20T10:22:50.214543+0000","last_fullsized":"2024-04-20T10:22:50.214543+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:09:34.293198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.191996+0000","last_change":"2024-04-20T10:22:50.191996+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.191996+0000","last_undegraded":"2024-04-20T10:22:50.191996+0000","last_fullsized":"2024-04-20T10:22:50.191996+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.209812+0000","last_change":"2024-04-20T10:22:50.206722+0000","last_active":"2024-04-20T10:22:50.209812+0000","last_peered":"2024-04-20T10:22:50.209812+0000","last_clean":"2024-04-20T10:22:50.209812+0000","last_became_active":"2024-04-20T10:22:50.206395+0000","last_became_peered":"2024-04-20T10:22:50.206395+0000","last_unstale":"2024-04-20T10:22:50.209812+0000","last_undegraded":"2024-04-20T10:22:50.209812+0000","last_fullsized":"2024-04-20T10:22:50.209812+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:58:15.263333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.198022+0000","last_change":"2024-04-20T10:22:50.198022+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.198022+0000","last_undegraded":"2024-04-20T10:22:50.198022+0000","last_fullsized":"2024-04-20T10:22:50.198022+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.207715+0000","last_change":"2024-04-20T10:22:50.203958+0000","last_active":"2024-04-20T10:22:50.207715+0000","last_peered":"2024-04-20T10:22:50.207715+0000","last_clean":"2024-04-20T10:22:50.207715+0000","last_became_active":"2024-04-20T10:22:50.203499+0000","last_became_peered":"2024-04-20T10:22:50.203499+0000","last_unstale":"2024-04-20T10:22:50.207715+0000","last_undegraded":"2024-04-20T10:22:50.207715+0000","last_fullsized":"2024-04-20T10:22:50.207715+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:16:27.290919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.192536+0000","last_change":"2024-04-20T10:22:50.192536+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.192536+0000","last_undegraded":"2024-04-20T10:22:50.192536+0000","last_fullsized":"2024-04-20T10:22:50.192536+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.200458+0000","last_change":"2024-04-20T10:22:50.196624+0000","last_active":"2024-04-20T10:22:50.200458+0000","last_peered":"2024-04-20T10:22:50.200458+0000","last_clean":"2024-04-20T10:22:50.200458+0000","last_became_active":"2024-04-20T10:22:50.196295+0000","last_became_peered":"2024-04-20T10:22:50.196295+0000","last_unstale":"2024-04-20T10:22:50.200458+0000","last_undegraded":"2024-04-20T10:22:50.200458+0000","last_fullsized":"2024-04-20T10:22:50.200458+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:30:35.298814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.179317+0000","last_change":"2024-04-20T10:22:50.179317+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.179317+0000","last_undegraded":"2024-04-20T10:22:50.179317+0000","last_fullsized":"2024-04-20T10:22:50.179317+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.205746+0000","last_change":"2024-04-20T10:22:50.200183+0000","last_active":"2024-04-20T10:22:50.205746+0000","last_peered":"2024-04-20T10:22:50.205746+0000","last_clean":"2024-04-20T10:22:50.205746+0000","last_became_active":"2024-04-20T10:22:50.199884+0000","last_became_peered":"2024-04-20T10:22:50.199884+0000","last_unstale":"2024-04-20T10:22:50.205746+0000","last_undegraded":"2024-04-20T10:22:50.205746+0000","last_fullsized":"2024-04-20T10:22:50.205746+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:10:15.543826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.195676+0000","last_change":"2024-04-20T10:22:50.195676+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.195676+0000","last_undegraded":"2024-04-20T10:22:50.195676+0000","last_fullsized":"2024-04-20T10:22:50.195676+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.209575+0000","last_change":"2024-04-20T10:22:50.205765+0000","last_active":"2024-04-20T10:22:50.209575+0000","last_peered":"2024-04-20T10:22:50.209575+0000","last_clean":"2024-04-20T10:22:50.209575+0000","last_became_active":"2024-04-20T10:22:50.205454+0000","last_became_peered":"2024-04-20T10:22:50.205454+0000","last_unstale":"2024-04-20T10:22:50.209575+0000","last_undegraded":"2024-04-20T10:22:50.209575+0000","last_fullsized":"2024-04-20T10:22:50.209575+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:20:42.218198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.185184+0000","last_change":"2024-04-20T10:22:50.185184+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.185184+0000","last_undegraded":"2024-04-20T10:22:50.185184+0000","last_fullsized":"2024-04-20T10:22:50.185184+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.212433+0000","last_change":"2024-04-20T10:22:50.210848+0000","last_active":"2024-04-20T10:22:50.212433+0000","last_peered":"2024-04-20T10:22:50.212433+0000","last_clean":"2024-04-20T10:22:50.212433+0000","last_became_active":"2024-04-20T10:22:50.210382+0000","last_became_peered":"2024-04-20T10:22:50.210382+0000","last_unstale":"2024-04-20T10:22:50.212433+0000","last_undegraded":"2024-04-20T10:22:50.212433+0000","last_fullsized":"2024-04-20T10:22:50.212433+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:39:48.488461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.186954+0000","last_change":"2024-04-20T10:22:50.186954+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.186954+0000","last_undegraded":"2024-04-20T10:22:50.186954+0000","last_fullsized":"2024-04-20T10:22:50.186954+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.211775+0000","last_change":"2024-04-20T10:22:50.208371+0000","last_active":"2024-04-20T10:22:50.211775+0000","last_peered":"2024-04-20T10:22:50.211775+0000","last_clean":"2024-04-20T10:22:50.211775+0000","last_became_active":"2024-04-20T10:22:50.208102+0000","last_became_peered":"2024-04-20T10:22:50.208102+0000","last_unstale":"2024-04-20T10:22:50.211775+0000","last_undegraded":"2024-04-20T10:22:50.211775+0000","last_fullsized":"2024-04-20T10:22:50.211775+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:16:51.466273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.190353+0000","last_change":"2024-04-20T10:22:50.190353+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.190353+0000","last_undegraded":"2024-04-20T10:22:50.190353+0000","last_fullsized":"2024-04-20T10:22:50.190353+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.209177+0000","last_change":"2024-04-20T10:22:50.205722+0000","last_active":"2024-04-20T10:22:50.209177+0000","last_peered":"2024-04-20T10:22:50.209177+0000","last_clean":"2024-04-20T10:22:50.209177+0000","last_became_active":"2024-04-20T10:22:50.205402+0000","last_became_peered":"2024-04-20T10:22:50.205402+0000","last_unstale":"2024-04-20T10:22:50.209177+0000","last_undegraded":"2024-04-20T10:22:50.209177+0000","last_fullsized":"2024-04-20T10:22:50.209177+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:20:42.218198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.193228+0000","last_change":"2024-04-20T10:22:50.193228+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.193228+0000","last_undegraded":"2024-04-20T10:22:50.193228+0000","last_fullsized":"2024-04-20T10:22:50.193228+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-20T10:22:50.173967+0000","last_change":"2024-04-20T10:22:50.173967+0000","last_active":"2024-04-20T10:22:49.145911+0000","last_peered":"2024-04-20T10:22:49.145911+0000","last_clean":"2024-04-20T10:22:49.145911+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.173967+0000","last_undegraded":"2024-04-20T10:22:50.173967+0000","last_fullsized":"2024-04-20T10:22:50.173967+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":["2","7"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.180223+0000","last_change":"2024-04-20T10:22:50.180223+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.180223+0000","last_undegraded":"2024-04-20T10:22:50.180223+0000","last_fullsized":"2024-04-20T10:22:50.180223+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.207866+0000","last_change":"2024-04-20T10:22:50.205027+0000","last_active":"2024-04-20T10:22:50.207866+0000","last_peered":"2024-04-20T10:22:50.207866+0000","last_clean":"2024-04-20T10:22:50.207866+0000","last_became_active":"2024-04-20T10:22:50.204749+0000","last_became_peered":"2024-04-20T10:22:50.204749+0000","last_unstale":"2024-04-20T10:22:50.207866+0000","last_undegraded":"2024-04-20T10:22:50.207866+0000","last_fullsized":"2024-04-20T10:22:50.207866+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:39:48.488461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.188798+0000","last_change":"2024-04-20T10:22:50.188798+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.188798+0000","last_undegraded":"2024-04-20T10:22:50.188798+0000","last_fullsized":"2024-04-20T10:22:50.188798+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.200723+0000","last_change":"2024-04-20T10:22:50.194920+0000","last_active":"2024-04-20T10:22:50.200723+0000","last_peered":"2024-04-20T10:22:50.200723+0000","last_clean":"2024-04-20T10:22:50.200723+0000","last_became_active":"2024-04-20T10:22:50.194622+0000","last_became_peered":"2024-04-20T10:22:50.194622+0000","last_unstale":"2024-04-20T10:22:50.200723+0000","last_undegraded":"2024-04-20T10:22:50.200723+0000","last_fullsized":"2024-04-20T10:22:50.200723+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:30:02.849778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.181408+0000","last_change":"2024-04-20T10:22:50.181408+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.181408+0000","last_undegraded":"2024-04-20T10:22:50.181408+0000","last_fullsized":"2024-04-20T10:22:50.181408+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-20T10:22:50.173618+0000","last_change":"2024-04-20T10:22:50.173618+0000","last_active":"2024-04-20T10:22:49.145911+0000","last_peered":"2024-04-20T10:22:49.145911+0000","last_clean":"2024-04-20T10:22:49.145911+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.173618+0000","last_undegraded":"2024-04-20T10:22:50.173618+0000","last_fullsized":"2024-04-20T10:22:50.173618+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.195699+0000","last_change":"2024-04-20T10:22:50.195699+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.195699+0000","last_undegraded":"2024-04-20T10:22:50.195699+0000","last_fullsized":"2024-04-20T10:22:50.195699+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.208591+0000","last_change":"2024-04-20T10:22:50.205146+0000","last_active":"2024-04-20T10:22:50.208591+0000","last_peered":"2024-04-20T10:22:50.208591+0000","last_clean":"2024-04-20T10:22:50.208591+0000","last_became_active":"2024-04-20T10:22:50.204772+0000","last_became_peered":"2024-04-20T10:22:50.204772+0000","last_unstale":"2024-04-20T10:22:50.208591+0000","last_undegraded":"2024-04-20T10:22:50.208591+0000","last_fullsized":"2024-04-20T10:22:50.208591+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:22:51.962842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.179884+0000","last_change":"2024-04-20T10:22:50.179884+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.179884+0000","last_undegraded":"2024-04-20T10:22:50.179884+0000","last_fullsized":"2024-04-20T10:22:50.179884+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.200768+0000","last_change":"2024-04-20T10:22:50.194938+0000","last_active":"2024-04-20T10:22:50.200768+0000","last_peered":"2024-04-20T10:22:50.200768+0000","last_clean":"2024-04-20T10:22:50.200768+0000","last_became_active":"2024-04-20T10:22:50.194631+0000","last_became_peered":"2024-04-20T10:22:50.194631+0000","last_unstale":"2024-04-20T10:22:50.200768+0000","last_undegraded":"2024-04-20T10:22:50.200768+0000","last_fullsized":"2024-04-20T10:22:50.200768+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:32:32.746673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.184467+0000","last_change":"2024-04-20T10:22:50.184467+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.184467+0000","last_undegraded":"2024-04-20T10:22:50.184467+0000","last_fullsized":"2024-04-20T10:22:50.184467+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.203938+0000","last_change":"2024-04-20T10:22:50.200577+0000","last_active":"2024-04-20T10:22:50.203938+0000","last_peered":"2024-04-20T10:22:50.203938+0000","last_clean":"2024-04-20T10:22:50.203938+0000","last_became_active":"2024-04-20T10:22:50.199510+0000","last_became_peered":"2024-04-20T10:22:50.199510+0000","last_unstale":"2024-04-20T10:22:50.203938+0000","last_undegraded":"2024-04-20T10:22:50.203938+0000","last_fullsized":"2024-04-20T10:22:50.203938+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:26:16.280439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.191998+0000","last_change":"2024-04-20T10:22:50.191998+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.191998+0000","last_undegraded":"2024-04-20T10:22:50.191998+0000","last_fullsized":"2024-04-20T10:22:50.191998+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":3,"reported_epoch":37,"state":"creating+activating","last_fresh":"2024-04-20T10:22:50.174033+0000","last_change":"2024-04-20T10:22:50.174033+0000","last_active":"2024-04-20T10:22:49.145911+0000","last_peered":"2024-04-20T10:22:49.145911+0000","last_clean":"2024-04-20T10:22:49.145911+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.174033+0000","last_undegraded":"2024-04-20T10:22:50.174033+0000","last_fullsized":"2024-04-20T10:22:50.174033+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":["2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.183765+0000","last_change":"2024-04-20T10:22:50.183765+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.183765+0000","last_undegraded":"2024-04-20T10:22:50.183765+0000","last_fullsized":"2024-04-20T10:22:50.183765+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.200821+0000","last_change":"2024-04-20T10:22:50.196011+0000","last_active":"2024-04-20T10:22:50.200821+0000","last_peered":"2024-04-20T10:22:50.200821+0000","last_clean":"2024-04-20T10:22:50.200821+0000","last_became_active":"2024-04-20T10:22:50.195748+0000","last_became_peered":"2024-04-20T10:22:50.195748+0000","last_unstale":"2024-04-20T10:22:50.200821+0000","last_undegraded":"2024-04-20T10:22:50.200821+0000","last_fullsized":"2024-04-20T10:22:50.200821+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:36:24.988490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.184945+0000","last_change":"2024-04-20T10:22:50.184945+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.184945+0000","last_undegraded":"2024-04-20T10:22:50.184945+0000","last_fullsized":"2024-04-20T10:22:50.184945+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.205855+0000","last_change":"2024-04-20T10:22:50.203809+0000","last_active":"2024-04-20T10:22:50.205855+0000","last_peered":"2024-04-20T10:22:50.205855+0000","last_clean":"2024-04-20T10:22:50.205855+0000","last_became_active":"2024-04-20T10:22:50.203451+0000","last_became_peered":"2024-04-20T10:22:50.203451+0000","last_unstale":"2024-04-20T10:22:50.205855+0000","last_undegraded":"2024-04-20T10:22:50.205855+0000","last_fullsized":"2024-04-20T10:22:50.205855+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:47:00.135513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.180703+0000","last_change":"2024-04-20T10:22:50.180703+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.180703+0000","last_undegraded":"2024-04-20T10:22:50.180703+0000","last_fullsized":"2024-04-20T10:22:50.180703+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.203328+0000","last_change":"2024-04-20T10:22:50.197399+0000","last_active":"2024-04-20T10:22:50.203328+0000","last_peered":"2024-04-20T10:22:50.203328+0000","last_clean":"2024-04-20T10:22:50.203328+0000","last_became_active":"2024-04-20T10:22:50.197015+0000","last_became_peered":"2024-04-20T10:22:50.197015+0000","last_unstale":"2024-04-20T10:22:50.203328+0000","last_undegraded":"2024-04-20T10:22:50.203328+0000","last_fullsized":"2024-04-20T10:22:50.203328+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:41:01.514306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.187559+0000","last_change":"2024-04-20T10:22:50.187559+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.187559+0000","last_undegraded":"2024-04-20T10:22:50.187559+0000","last_fullsized":"2024-04-20T10:22:50.187559+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.208900+0000","last_change":"2024-04-20T10:22:50.203369+0000","last_active":"2024-04-20T10:22:50.208900+0000","last_peered":"2024-04-20T10:22:50.208900+0000","last_clean":"2024-04-20T10:22:50.208900+0000","last_became_active":"2024-04-20T10:22:50.203083+0000","last_became_peered":"2024-04-20T10:22:50.203083+0000","last_unstale":"2024-04-20T10:22:50.208900+0000","last_undegraded":"2024-04-20T10:22:50.208900+0000","last_fullsized":"2024-04-20T10:22:50.208900+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:41:01.514306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.189625+0000","last_change":"2024-04-20T10:22:50.189625+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.189625+0000","last_undegraded":"2024-04-20T10:22:50.189625+0000","last_fullsized":"2024-04-20T10:22:50.189625+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.205556+0000","last_change":"2024-04-20T10:22:50.198717+0000","last_active":"2024-04-20T10:22:50.205556+0000","last_peered":"2024-04-20T10:22:50.205556+0000","last_clean":"2024-04-20T10:22:50.205556+0000","last_became_active":"2024-04-20T10:22:50.197112+0000","last_became_peered":"2024-04-20T10:22:50.197112+0000","last_unstale":"2024-04-20T10:22:50.205556+0000","last_undegraded":"2024-04-20T10:22:50.205556+0000","last_fullsized":"2024-04-20T10:22:50.205556+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:36:31.551385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-04-20T10:22:50.189620+0000","last_change":"2024-04-20T10:22:50.189620+0000","last_active":"2024-04-20T10:22:50.161787+0000","last_peered":"2024-04-20T10:22:50.161787+0000","last_clean":"2024-04-20T10:22:50.161787+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:22:50.189620+0000","last_undegraded":"2024-04-20T10:22:50.189620+0000","last_fullsized":"2024-04-20T10:22:50.189620+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.206576+0000","last_change":"2024-04-20T10:22:50.203854+0000","last_active":"2024-04-20T10:22:50.206576+0000","last_peered":"2024-04-20T10:22:50.206576+0000","last_clean":"2024-04-20T10:22:50.206576+0000","last_became_active":"2024-04-20T10:22:50.203439+0000","last_became_peered":"2024-04-20T10:22:50.203439+0000","last_unstale":"2024-04-20T10:22:50.206576+0000","last_undegraded":"2024-04-20T10:22:50.206576+0000","last_fullsized":"2024-04-20T10:22:50.206576+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:10:15.543826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":198,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-04-20T10:22:50.195263+0000","last_change":"2024-04-20T10:20:52.916129+0000","last_active":"2024-04-20T10:22:50.195263+0000","last_peered":"2024-04-20T10:22:50.195263+0000","last_clean":"2024-04-20T10:22:50.195263+0000","last_became_active":"2024-04-20T10:20:52.913922+0000","last_became_peered":"2024-04-20T10:20:52.913922+0000","last_unstale":"2024-04-20T10:22:50.195263+0000","last_undegraded":"2024-04-20T10:22:50.195263+0000","last_fullsized":"2024-04-20T10:22:50.195263+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:20:51.267588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:20:51.267588+0000","last_clean_scrub_stamp":"2024-04-20T10:20:51.267588+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:30:44.465836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542170,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":1003520,"data_stored":763189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":409600,"data_stored":172821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542170,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":409600,"data_stored":172821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542170,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":409600,"data_stored":172821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542170,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":421888,"data_stored":179488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542170,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":409600,"data_stored":172821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542172,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":409600,"data_stored":172821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542172,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":1003520,"data_stored":763189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T10:22:56.606 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3f is not active+clean 2024-04-20T10:22:56.606 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.187152+0000', 'last_change': '2024-04-20T10:22:50.187152+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.187152+0000', 'last_undegraded': '2024-04-20T10:22:50.187152+0000', 'last_fullsized': '2024-04-20T10:22:50.187152+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-20T10:22:56.606 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3e is not active+clean 2024-04-20T10:22:56.606 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.186798+0000', 'last_change': '2024-04-20T10:22:50.186798+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.186798+0000', 'last_undegraded': '2024-04-20T10:22:50.186798+0000', 'last_fullsized': '2024-04-20T10:22:50.186798+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-20T10:22:56.606 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3d is not active+clean 2024-04-20T10:22:56.607 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.189601+0000', 'last_change': '2024-04-20T10:22:50.189601+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.189601+0000', 'last_undegraded': '2024-04-20T10:22:50.189601+0000', 'last_fullsized': '2024-04-20T10:22:50.189601+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-20T10:22:56.607 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3c is not active+clean 2024-04-20T10:22:56.607 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.181311+0000', 'last_change': '2024-04-20T10:22:50.181311+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.181311+0000', 'last_undegraded': '2024-04-20T10:22:50.181311+0000', 'last_fullsized': '2024-04-20T10:22:50.181311+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-20T10:22:56.607 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3b is not active+clean 2024-04-20T10:22:56.607 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.184441+0000', 'last_change': '2024-04-20T10:22:50.184441+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.184441+0000', 'last_undegraded': '2024-04-20T10:22:50.184441+0000', 'last_fullsized': '2024-04-20T10:22:50.184441+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-20T10:22:56.607 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3a is not active+clean 2024-04-20T10:22:56.607 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.182950+0000', 'last_change': '2024-04-20T10:22:50.182950+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.182950+0000', 'last_undegraded': '2024-04-20T10:22:50.182950+0000', 'last_fullsized': '2024-04-20T10:22:50.182950+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-20T10:22:56.607 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.39 is not active+clean 2024-04-20T10:22:56.607 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.192894+0000', 'last_change': '2024-04-20T10:22:50.192894+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.192894+0000', 'last_undegraded': '2024-04-20T10:22:50.192894+0000', 'last_fullsized': '2024-04-20T10:22:50.192894+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-20T10:22:56.607 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.38 is not active+clean 2024-04-20T10:22:56.607 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.181902+0000', 'last_change': '2024-04-20T10:22:50.181902+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.181902+0000', 'last_undegraded': '2024-04-20T10:22:50.181902+0000', 'last_fullsized': '2024-04-20T10:22:50.181902+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-20T10:22:56.608 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.37 is not active+clean 2024-04-20T10:22:56.608 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.192533+0000', 'last_change': '2024-04-20T10:22:50.192533+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.192533+0000', 'last_undegraded': '2024-04-20T10:22:50.192533+0000', 'last_fullsized': '2024-04-20T10:22:50.192533+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-20T10:22:56.608 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.36 is not active+clean 2024-04-20T10:22:56.608 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.194398+0000', 'last_change': '2024-04-20T10:22:50.194398+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.194398+0000', 'last_undegraded': '2024-04-20T10:22:50.194398+0000', 'last_fullsized': '2024-04-20T10:22:50.194398+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-20T10:22:56.608 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.35 is not active+clean 2024-04-20T10:22:56.608 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.194904+0000', 'last_change': '2024-04-20T10:22:50.194904+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.194904+0000', 'last_undegraded': '2024-04-20T10:22:50.194904+0000', 'last_fullsized': '2024-04-20T10:22:50.194904+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-20T10:22:56.609 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.34 is not active+clean 2024-04-20T10:22:56.609 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.192601+0000', 'last_change': '2024-04-20T10:22:50.192601+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.192601+0000', 'last_undegraded': '2024-04-20T10:22:50.192601+0000', 'last_fullsized': '2024-04-20T10:22:50.192601+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-20T10:22:56.609 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.33 is not active+clean 2024-04-20T10:22:56.609 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.197754+0000', 'last_change': '2024-04-20T10:22:50.197754+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.197754+0000', 'last_undegraded': '2024-04-20T10:22:50.197754+0000', 'last_fullsized': '2024-04-20T10:22:50.197754+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-20T10:22:56.609 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.32 is not active+clean 2024-04-20T10:22:56.609 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.188557+0000', 'last_change': '2024-04-20T10:22:50.188557+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.188557+0000', 'last_undegraded': '2024-04-20T10:22:50.188557+0000', 'last_fullsized': '2024-04-20T10:22:50.188557+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-20T10:22:56.609 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.31 is not active+clean 2024-04-20T10:22:56.609 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.192196+0000', 'last_change': '2024-04-20T10:22:50.192196+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.192196+0000', 'last_undegraded': '2024-04-20T10:22:50.192196+0000', 'last_fullsized': '2024-04-20T10:22:50.192196+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-20T10:22:56.609 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.30 is not active+clean 2024-04-20T10:22:56.609 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.190134+0000', 'last_change': '2024-04-20T10:22:50.190134+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.190134+0000', 'last_undegraded': '2024-04-20T10:22:50.190134+0000', 'last_fullsized': '2024-04-20T10:22:50.190134+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-20T10:22:56.610 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2f is not active+clean 2024-04-20T10:22:56.610 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.189486+0000', 'last_change': '2024-04-20T10:22:50.189486+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.189486+0000', 'last_undegraded': '2024-04-20T10:22:50.189486+0000', 'last_fullsized': '2024-04-20T10:22:50.189486+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-20T10:22:56.610 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2e is not active+clean 2024-04-20T10:22:56.610 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.177303+0000', 'last_change': '2024-04-20T10:22:50.177303+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.177303+0000', 'last_undegraded': '2024-04-20T10:22:50.177303+0000', 'last_fullsized': '2024-04-20T10:22:50.177303+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-20T10:22:56.610 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2d is not active+clean 2024-04-20T10:22:56.610 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.182415+0000', 'last_change': '2024-04-20T10:22:50.182415+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.182415+0000', 'last_undegraded': '2024-04-20T10:22:50.182415+0000', 'last_fullsized': '2024-04-20T10:22:50.182415+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-20T10:22:56.610 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2c is not active+clean 2024-04-20T10:22:56.610 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.177312+0000', 'last_change': '2024-04-20T10:22:50.177312+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.177312+0000', 'last_undegraded': '2024-04-20T10:22:50.177312+0000', 'last_fullsized': '2024-04-20T10:22:50.177312+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-20T10:22:56.610 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2b is not active+clean 2024-04-20T10:22:56.610 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.195671+0000', 'last_change': '2024-04-20T10:22:50.195671+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.195671+0000', 'last_undegraded': '2024-04-20T10:22:50.195671+0000', 'last_fullsized': '2024-04-20T10:22:50.195671+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-20T10:22:56.610 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2a is not active+clean 2024-04-20T10:22:56.611 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.186598+0000', 'last_change': '2024-04-20T10:22:50.186598+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.186598+0000', 'last_undegraded': '2024-04-20T10:22:50.186598+0000', 'last_fullsized': '2024-04-20T10:22:50.186598+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-20T10:22:56.611 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.29 is not active+clean 2024-04-20T10:22:56.611 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.190727+0000', 'last_change': '2024-04-20T10:22:50.190727+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.190727+0000', 'last_undegraded': '2024-04-20T10:22:50.190727+0000', 'last_fullsized': '2024-04-20T10:22:50.190727+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-20T10:22:56.611 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.28 is not active+clean 2024-04-20T10:22:56.611 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.182533+0000', 'last_change': '2024-04-20T10:22:50.182533+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.182533+0000', 'last_undegraded': '2024-04-20T10:22:50.182533+0000', 'last_fullsized': '2024-04-20T10:22:50.182533+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-20T10:22:56.611 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.27 is not active+clean 2024-04-20T10:22:56.611 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.181275+0000', 'last_change': '2024-04-20T10:22:50.181275+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.181275+0000', 'last_undegraded': '2024-04-20T10:22:50.181275+0000', 'last_fullsized': '2024-04-20T10:22:50.181275+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-20T10:22:56.611 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.26 is not active+clean 2024-04-20T10:22:56.611 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.182908+0000', 'last_change': '2024-04-20T10:22:50.182908+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.182908+0000', 'last_undegraded': '2024-04-20T10:22:50.182908+0000', 'last_fullsized': '2024-04-20T10:22:50.182908+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-20T10:22:56.611 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.25 is not active+clean 2024-04-20T10:22:56.611 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.177500+0000', 'last_change': '2024-04-20T10:22:50.177500+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.177500+0000', 'last_undegraded': '2024-04-20T10:22:50.177500+0000', 'last_fullsized': '2024-04-20T10:22:50.177500+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-20T10:22:56.612 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.24 is not active+clean 2024-04-20T10:22:56.612 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.187253+0000', 'last_change': '2024-04-20T10:22:50.187253+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.187253+0000', 'last_undegraded': '2024-04-20T10:22:50.187253+0000', 'last_fullsized': '2024-04-20T10:22:50.187253+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-20T10:22:56.612 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.23 is not active+clean 2024-04-20T10:22:56.612 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.177359+0000', 'last_change': '2024-04-20T10:22:50.177359+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.177359+0000', 'last_undegraded': '2024-04-20T10:22:50.177359+0000', 'last_fullsized': '2024-04-20T10:22:50.177359+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-20T10:22:56.612 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.22 is not active+clean 2024-04-20T10:22:56.612 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.179150+0000', 'last_change': '2024-04-20T10:22:50.179150+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.179150+0000', 'last_undegraded': '2024-04-20T10:22:50.179150+0000', 'last_fullsized': '2024-04-20T10:22:50.179150+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-20T10:22:56.612 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.21 is not active+clean 2024-04-20T10:22:56.612 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.189823+0000', 'last_change': '2024-04-20T10:22:50.189823+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.189823+0000', 'last_undegraded': '2024-04-20T10:22:50.189823+0000', 'last_fullsized': '2024-04-20T10:22:50.189823+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-20T10:22:56.612 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.20 is not active+clean 2024-04-20T10:22:56.612 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.179894+0000', 'last_change': '2024-04-20T10:22:50.179894+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.179894+0000', 'last_undegraded': '2024-04-20T10:22:50.179894+0000', 'last_fullsized': '2024-04-20T10:22:50.179894+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-20T10:22:56.612 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1c is not active+clean 2024-04-20T10:22:56.613 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.195627+0000', 'last_change': '2024-04-20T10:22:50.195627+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.195627+0000', 'last_undegraded': '2024-04-20T10:22:50.195627+0000', 'last_fullsized': '2024-04-20T10:22:50.195627+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-20T10:22:56.613 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1d is not active+clean 2024-04-20T10:22:56.613 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.183462+0000', 'last_change': '2024-04-20T10:22:50.183462+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.183462+0000', 'last_undegraded': '2024-04-20T10:22:50.183462+0000', 'last_fullsized': '2024-04-20T10:22:50.183462+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-20T10:22:56.613 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1e is not active+clean 2024-04-20T10:22:56.613 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.192569+0000', 'last_change': '2024-04-20T10:22:50.192569+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.192569+0000', 'last_undegraded': '2024-04-20T10:22:50.192569+0000', 'last_fullsized': '2024-04-20T10:22:50.192569+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-20T10:22:56.613 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1f is not active+clean 2024-04-20T10:22:56.613 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.186710+0000', 'last_change': '2024-04-20T10:22:50.186710+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.186710+0000', 'last_undegraded': '2024-04-20T10:22:50.186710+0000', 'last_fullsized': '2024-04-20T10:22:50.186710+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-20T10:22:56.613 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.18 is not active+clean 2024-04-20T10:22:56.613 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.193141+0000', 'last_change': '2024-04-20T10:22:50.193141+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.193141+0000', 'last_undegraded': '2024-04-20T10:22:50.193141+0000', 'last_fullsized': '2024-04-20T10:22:50.193141+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-20T10:22:56.613 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.19 is not active+clean 2024-04-20T10:22:56.613 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.188832+0000', 'last_change': '2024-04-20T10:22:50.188832+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.188832+0000', 'last_undegraded': '2024-04-20T10:22:50.188832+0000', 'last_fullsized': '2024-04-20T10:22:50.188832+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-20T10:22:56.614 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1a is not active+clean 2024-04-20T10:22:56.614 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.185316+0000', 'last_change': '2024-04-20T10:22:50.185316+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.185316+0000', 'last_undegraded': '2024-04-20T10:22:50.185316+0000', 'last_fullsized': '2024-04-20T10:22:50.185316+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-20T10:22:56.614 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1b is not active+clean 2024-04-20T10:22:56.614 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.185906+0000', 'last_change': '2024-04-20T10:22:50.185906+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.185906+0000', 'last_undegraded': '2024-04-20T10:22:50.185906+0000', 'last_fullsized': '2024-04-20T10:22:50.185906+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-20T10:22:56.614 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.14 is not active+clean 2024-04-20T10:22:56.614 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.177757+0000', 'last_change': '2024-04-20T10:22:50.177757+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.177757+0000', 'last_undegraded': '2024-04-20T10:22:50.177757+0000', 'last_fullsized': '2024-04-20T10:22:50.177757+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-20T10:22:56.614 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.15 is not active+clean 2024-04-20T10:22:56.614 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.192693+0000', 'last_change': '2024-04-20T10:22:50.192693+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.192693+0000', 'last_undegraded': '2024-04-20T10:22:50.192693+0000', 'last_fullsized': '2024-04-20T10:22:50.192693+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-20T10:22:56.614 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.16 is not active+clean 2024-04-20T10:22:56.614 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.191996+0000', 'last_change': '2024-04-20T10:22:50.191996+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.191996+0000', 'last_undegraded': '2024-04-20T10:22:50.191996+0000', 'last_fullsized': '2024-04-20T10:22:50.191996+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-20T10:22:56.615 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.17 is not active+clean 2024-04-20T10:22:56.615 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.198022+0000', 'last_change': '2024-04-20T10:22:50.198022+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.198022+0000', 'last_undegraded': '2024-04-20T10:22:50.198022+0000', 'last_fullsized': '2024-04-20T10:22:50.198022+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-20T10:22:56.615 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.10 is not active+clean 2024-04-20T10:22:56.615 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.192536+0000', 'last_change': '2024-04-20T10:22:50.192536+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.192536+0000', 'last_undegraded': '2024-04-20T10:22:50.192536+0000', 'last_fullsized': '2024-04-20T10:22:50.192536+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-20T10:22:56.615 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.11 is not active+clean 2024-04-20T10:22:56.615 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.179317+0000', 'last_change': '2024-04-20T10:22:50.179317+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.179317+0000', 'last_undegraded': '2024-04-20T10:22:50.179317+0000', 'last_fullsized': '2024-04-20T10:22:50.179317+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-20T10:22:56.615 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.12 is not active+clean 2024-04-20T10:22:56.615 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.195676+0000', 'last_change': '2024-04-20T10:22:50.195676+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.195676+0000', 'last_undegraded': '2024-04-20T10:22:50.195676+0000', 'last_fullsized': '2024-04-20T10:22:50.195676+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-20T10:22:56.615 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.13 is not active+clean 2024-04-20T10:22:56.615 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.185184+0000', 'last_change': '2024-04-20T10:22:50.185184+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.185184+0000', 'last_undegraded': '2024-04-20T10:22:50.185184+0000', 'last_fullsized': '2024-04-20T10:22:50.185184+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-20T10:22:56.615 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.c is not active+clean 2024-04-20T10:22:56.615 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.186954+0000', 'last_change': '2024-04-20T10:22:50.186954+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.186954+0000', 'last_undegraded': '2024-04-20T10:22:50.186954+0000', 'last_fullsized': '2024-04-20T10:22:50.186954+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-20T10:22:56.616 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.d is not active+clean 2024-04-20T10:22:56.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.190353+0000', 'last_change': '2024-04-20T10:22:50.190353+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.190353+0000', 'last_undegraded': '2024-04-20T10:22:50.190353+0000', 'last_fullsized': '2024-04-20T10:22:50.190353+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-20T10:22:56.616 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.e is not active+clean 2024-04-20T10:22:56.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.193228+0000', 'last_change': '2024-04-20T10:22:50.193228+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.193228+0000', 'last_undegraded': '2024-04-20T10:22:50.193228+0000', 'last_fullsized': '2024-04-20T10:22:50.193228+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-20T10:22:56.616 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2024-04-20T10:22:56.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-04-20T10:22:50.173967+0000', 'last_change': '2024-04-20T10:22:50.173967+0000', 'last_active': '2024-04-20T10:22:49.145911+0000', 'last_peered': '2024-04-20T10:22:49.145911+0000', 'last_clean': '2024-04-20T10:22:49.145911+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.173967+0000', 'last_undegraded': '2024-04-20T10:22:50.173967+0000', 'last_fullsized': '2024-04-20T10:22:50.173967+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:49.145911+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:49.145911+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:49.145911+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': ['2', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-20T10:22:56.616 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.f is not active+clean 2024-04-20T10:22:56.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.180223+0000', 'last_change': '2024-04-20T10:22:50.180223+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.180223+0000', 'last_undegraded': '2024-04-20T10:22:50.180223+0000', 'last_fullsized': '2024-04-20T10:22:50.180223+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-20T10:22:56.616 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.8 is not active+clean 2024-04-20T10:22:56.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.188798+0000', 'last_change': '2024-04-20T10:22:50.188798+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.188798+0000', 'last_undegraded': '2024-04-20T10:22:50.188798+0000', 'last_fullsized': '2024-04-20T10:22:50.188798+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-20T10:22:56.616 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.9 is not active+clean 2024-04-20T10:22:56.617 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.181408+0000', 'last_change': '2024-04-20T10:22:50.181408+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.181408+0000', 'last_undegraded': '2024-04-20T10:22:50.181408+0000', 'last_fullsized': '2024-04-20T10:22:50.181408+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-20T10:22:56.617 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2024-04-20T10:22:56.617 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-04-20T10:22:50.173618+0000', 'last_change': '2024-04-20T10:22:50.173618+0000', 'last_active': '2024-04-20T10:22:49.145911+0000', 'last_peered': '2024-04-20T10:22:49.145911+0000', 'last_clean': '2024-04-20T10:22:49.145911+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.173618+0000', 'last_undegraded': '2024-04-20T10:22:50.173618+0000', 'last_fullsized': '2024-04-20T10:22:50.173618+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:49.145911+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:49.145911+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:49.145911+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': ['2', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-20T10:22:56.617 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.a is not active+clean 2024-04-20T10:22:56.617 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.195699+0000', 'last_change': '2024-04-20T10:22:50.195699+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.195699+0000', 'last_undegraded': '2024-04-20T10:22:50.195699+0000', 'last_fullsized': '2024-04-20T10:22:50.195699+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-20T10:22:56.617 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.b is not active+clean 2024-04-20T10:22:56.617 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.179884+0000', 'last_change': '2024-04-20T10:22:50.179884+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.179884+0000', 'last_undegraded': '2024-04-20T10:22:50.179884+0000', 'last_fullsized': '2024-04-20T10:22:50.179884+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-20T10:22:56.617 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.4 is not active+clean 2024-04-20T10:22:56.617 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.184467+0000', 'last_change': '2024-04-20T10:22:50.184467+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.184467+0000', 'last_undegraded': '2024-04-20T10:22:50.184467+0000', 'last_fullsized': '2024-04-20T10:22:50.184467+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-20T10:22:56.617 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.5 is not active+clean 2024-04-20T10:22:56.617 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.191998+0000', 'last_change': '2024-04-20T10:22:50.191998+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.191998+0000', 'last_undegraded': '2024-04-20T10:22:50.191998+0000', 'last_fullsized': '2024-04-20T10:22:50.191998+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-20T10:22:56.618 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2024-04-20T10:22:56.618 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 37, 'state': 'creating+activating', 'last_fresh': '2024-04-20T10:22:50.174033+0000', 'last_change': '2024-04-20T10:22:50.174033+0000', 'last_active': '2024-04-20T10:22:49.145911+0000', 'last_peered': '2024-04-20T10:22:49.145911+0000', 'last_clean': '2024-04-20T10:22:49.145911+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.174033+0000', 'last_undegraded': '2024-04-20T10:22:50.174033+0000', 'last_fullsized': '2024-04-20T10:22:50.174033+0000', 'mapping_epoch': 36, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 36, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:49.145911+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:49.145911+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:49.145911+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': ['2', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-20T10:22:56.618 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.6 is not active+clean 2024-04-20T10:22:56.618 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.183765+0000', 'last_change': '2024-04-20T10:22:50.183765+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.183765+0000', 'last_undegraded': '2024-04-20T10:22:50.183765+0000', 'last_fullsized': '2024-04-20T10:22:50.183765+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-20T10:22:56.618 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.0 is not active+clean 2024-04-20T10:22:56.618 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.184945+0000', 'last_change': '2024-04-20T10:22:50.184945+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.184945+0000', 'last_undegraded': '2024-04-20T10:22:50.184945+0000', 'last_fullsized': '2024-04-20T10:22:50.184945+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-20T10:22:56.618 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1 is not active+clean 2024-04-20T10:22:56.618 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.180703+0000', 'last_change': '2024-04-20T10:22:50.180703+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.180703+0000', 'last_undegraded': '2024-04-20T10:22:50.180703+0000', 'last_fullsized': '2024-04-20T10:22:50.180703+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-20T10:22:56.618 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2 is not active+clean 2024-04-20T10:22:56.618 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.187559+0000', 'last_change': '2024-04-20T10:22:50.187559+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.187559+0000', 'last_undegraded': '2024-04-20T10:22:50.187559+0000', 'last_fullsized': '2024-04-20T10:22:50.187559+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-20T10:22:56.619 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3 is not active+clean 2024-04-20T10:22:56.619 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.189625+0000', 'last_change': '2024-04-20T10:22:50.189625+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.189625+0000', 'last_undegraded': '2024-04-20T10:22:50.189625+0000', 'last_fullsized': '2024-04-20T10:22:50.189625+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-20T10:22:56.619 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.7 is not active+clean 2024-04-20T10:22:56.619 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-04-20T10:22:50.189620+0000', 'last_change': '2024-04-20T10:22:50.189620+0000', 'last_active': '2024-04-20T10:22:50.161787+0000', 'last_peered': '2024-04-20T10:22:50.161787+0000', 'last_clean': '2024-04-20T10:22:50.161787+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-20T10:22:50.189620+0000', 'last_undegraded': '2024-04-20T10:22:50.189620+0000', 'last_fullsized': '2024-04-20T10:22:50.189620+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'last_clean_scrub_stamp': '2024-04-20T10:22:50.161787+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-20T10:22:56.619 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-20T10:22:56.835 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.835+0000 7f10a0200640 1 Processor -- start 2024-04-20T10:22:56.835 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.835+0000 7f10a0200640 1 -- start start 2024-04-20T10:22:56.836 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.837+0000 7f10a0200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f109815b3b0 0x7f109806eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:56.837 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.837+0000 7f10a0200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f109806dcb0 0x7f1098071b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:56.837 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.837+0000 7f10a0200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10980ade40 con 0x7f1098072090 2024-04-20T10:22:56.837 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.837+0000 7f10a0200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f10980ae0e0 con 0x7f109806dcb0 2024-04-20T10:22:56.837 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.837+0000 7f10a0200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f109815b790 con 0x7f109815b3b0 2024-04-20T10:22:56.837 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.837+0000 7f109ce00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f109806dcb0 0x7f1098071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:56.837 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.837+0000 7f109ce00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f109806dcb0 0x7f1098071b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33228/0 (socket says 172.21.15.121:33228) 2024-04-20T10:22:56.837 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.837+0000 7f109ce00640 1 -- 172.21.15.121:0/1151708076 learned_addr learned my addr 172.21.15.121:0/1151708076 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:56.837 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.837+0000 7f109e200640 1 --1- 172.21.15.121:0/1151708076 >> v1:172.21.15.121:6789/0 conn(0x7f1098072090 0x7f10980d0360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:50454/0 (socket says 172.21.15.121:50454) 2024-04-20T10:22:56.838 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.838+0000 7f109ce00640 1 -- 172.21.15.121:0/1151708076 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f109815b3b0 msgr2=0x7f109806eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:56.838 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.838+0000 7f109d800640 1 --2- 172.21.15.121:0/1151708076 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f109815b3b0 0x7f109806eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:56.838 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.838+0000 7f109ce00640 1 --2- 172.21.15.121:0/1151708076 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f109815b3b0 0x7f109806eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:56.839 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.838+0000 7f109ce00640 1 -- 172.21.15.121:0/1151708076 >> v1:172.21.15.121:6789/0 conn(0x7f1098072090 legacy=0x7f10980d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:56.839 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.838+0000 7f109ce00640 1 -- 172.21.15.121:0/1151708076 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10980ae380 con 0x7f109806dcb0 2024-04-20T10:22:56.839 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.838+0000 7f1097e00640 1 -- 172.21.15.121:0/1151708076 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3804380364 0 0) 0x7f10980ade40 con 0x7f1098072090 2024-04-20T10:22:56.839 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.838+0000 7f109d800640 1 --2- 172.21.15.121:0/1151708076 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f109815b3b0 0x7f109806eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:56.839 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.839+0000 7f109ce00640 1 --2- 172.21.15.121:0/1151708076 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f109806dcb0 0x7f1098071b50 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1084009870 tx=0x7f108402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c5ec923d4560cd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:56.839 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.839+0000 7f1097e00640 1 -- 172.21.15.121:0/1151708076 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1084034070 con 0x7f109806dcb0 2024-04-20T10:22:56.839 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.839+0000 7f1097e00640 1 -- 172.21.15.121:0/1151708076 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1084030840 con 0x7f109806dcb0 2024-04-20T10:22:56.839 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.839+0000 7f1097e00640 1 -- 172.21.15.121:0/1151708076 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1084030c10 con 0x7f109806dcb0 2024-04-20T10:22:56.840 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.839+0000 7f10a0200640 1 -- 172.21.15.121:0/1151708076 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f109806dcb0 msgr2=0x7f1098071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:56.840 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.839+0000 7f10a0200640 1 --2- 172.21.15.121:0/1151708076 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f109806dcb0 0x7f1098071b50 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1084009870 tx=0x7f108402b980 comp rx=0 tx=0).stop 2024-04-20T10:22:56.840 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.840+0000 7f10a0200640 1 -- 172.21.15.121:0/1151708076 shutdown_connections 2024-04-20T10:22:56.840 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.840+0000 7f10a0200640 1 --2- 172.21.15.121:0/1151708076 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f109806dcb0 0x7f1098071b50 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:56.840 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.840+0000 7f10a0200640 1 --2- 172.21.15.121:0/1151708076 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f109815b3b0 0x7f109806eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:56.840 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.840+0000 7f10a0200640 1 -- 172.21.15.121:0/1151708076 >> 172.21.15.121:0/1151708076 conn(0x7f10980b3350 msgr2=0x7f10980b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:56.840 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.840+0000 7f10a0200640 1 -- 172.21.15.121:0/1151708076 shutdown_connections 2024-04-20T10:22:56.840 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.840+0000 7f10a0200640 1 -- 172.21.15.121:0/1151708076 wait complete. 2024-04-20T10:22:56.841 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.841+0000 7f10a0200640 1 Processor -- start 2024-04-20T10:22:56.841 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.841+0000 7f10a0200640 1 -- start start 2024-04-20T10:22:56.843 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.842+0000 7f10a0200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f109806dcb0 0x7f109806c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:56.843 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.842+0000 7f10a0200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f109815b3b0 0x7f109814dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:56.843 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.842+0000 7f10a0200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f109814e600 0x7f1098159e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:56.843 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.843+0000 7f10a0200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f10980aec20 con 0x7f109814e600 2024-04-20T10:22:56.843 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.843+0000 7f10a0200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f10980ae940 con 0x7f109806dcb0 2024-04-20T10:22:56.843 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.843+0000 7f10a0200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f109806efe0 con 0x7f109815b3b0 2024-04-20T10:22:56.843 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.843+0000 7f109e200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f109814e600 0x7f1098159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:56.843 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.843+0000 7f109e200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f109814e600 0x7f1098159e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33236/0 (socket says 172.21.15.121:33236) 2024-04-20T10:22:56.844 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.843+0000 7f109e200640 1 -- 172.21.15.121:0/2806696460 learned_addr learned my addr 172.21.15.121:0/2806696460 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:56.844 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.843+0000 7f109e200640 1 -- 172.21.15.121:0/2806696460 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f109815b3b0 msgr2=0x7f109814dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:56.844 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.843+0000 7f109e200640 1 --2- 172.21.15.121:0/2806696460 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f109815b3b0 0x7f109814dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:56.844 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.843+0000 7f109e200640 1 -- 172.21.15.121:0/2806696460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f109806dcb0 msgr2=0x7f109806c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:56.844 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.843+0000 7f109e200640 1 --2- 172.21.15.121:0/2806696460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f109806dcb0 0x7f109806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:56.844 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.843+0000 7f109e200640 1 -- 172.21.15.121:0/2806696460 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10981f2460 con 0x7f109814e600 2024-04-20T10:22:56.844 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.843+0000 7f109e200640 1 --2- 172.21.15.121:0/2806696460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f109814e600 0x7f1098159e50 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f109000c750 tx=0x7f109000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:56.844 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.844+0000 7f1095600640 1 -- 172.21.15.121:0/2806696460 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1090005da0 con 0x7f109814e600 2024-04-20T10:22:56.844 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.844+0000 7f1095600640 1 -- 172.21.15.121:0/2806696460 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f109000a5e0 con 0x7f109814e600 2024-04-20T10:22:56.844 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.844+0000 7f1095600640 1 -- 172.21.15.121:0/2806696460 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109000a9f0 con 0x7f109814e600 2024-04-20T10:22:56.844 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.844+0000 7f10a0200640 1 -- 172.21.15.121:0/2806696460 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10981f29f0 con 0x7f109814e600 2024-04-20T10:22:56.844 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.844+0000 7f10a0200640 1 -- 172.21.15.121:0/2806696460 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10981f2fb0 con 0x7f109814e600 2024-04-20T10:22:56.845 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.845+0000 7f10a0200640 1 -- 172.21.15.121:0/2806696460 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1098068980 con 0x7f109814e600 2024-04-20T10:22:56.845 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.846+0000 7f1095600640 1 -- 172.21.15.121:0/2806696460 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f109000ab90 con 0x7f109814e600 2024-04-20T10:22:56.847 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.847+0000 7f1095600640 1 --2- 172.21.15.121:0/2806696460 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f107806c8f0 0x7f107806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:56.847 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.847+0000 7f1095600640 1 -- 172.21.15.121:0/2806696460 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f109008ba00 con 0x7f109814e600 2024-04-20T10:22:56.847 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.847+0000 7f109d800640 1 --2- 172.21.15.121:0/2806696460 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f107806c8f0 0x7f107806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:56.847 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.848+0000 7f109d800640 1 --2- 172.21.15.121:0/2806696460 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f107806c8f0 0x7f107806edb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f108c006fd0 tx=0x7f108c008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:56.851 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:56.852+0000 7f1095600640 1 -- 172.21.15.121:0/2806696460 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1090059d30 con 0x7f109814e600 2024-04-20T10:22:57.121 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.121+0000 7f10a0200640 1 -- 172.21.15.121:0/2806696460 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f109806c7d0 con 0x7f109814e600 2024-04-20T10:22:57.122 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.122+0000 7f1095600640 1 -- 172.21.15.121:0/2806696460 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1081 (secure 0 0 0) 0x7f1090059630 con 0x7f109814e600 2024-04-20T10:22:57.122 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:22:57.122 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":139,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":39,"num_osds":8,"num_up_osds":8,"osd_up_since":1713608450,"num_in_osds":8,"osd_in_since":1713608441,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":30},{"state_name":"creating+activating","count":3}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":225828864,"bytes_avail":772868284416,"bytes_total":773094113280,"inactive_pgs_ratio":0.69072163105010986},"fsmap":{"epoch":17,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":5568}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-04-20T10:22:09.133129+0000","services":{}},"progress_events":{}} 2024-04-20T10:22:57.124 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.125+0000 7f10a0200640 1 -- 172.21.15.121:0/2806696460 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f107806c8f0 msgr2=0x7f107806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:57.124 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.125+0000 7f10a0200640 1 --2- 172.21.15.121:0/2806696460 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f107806c8f0 0x7f107806edb0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f108c006fd0 tx=0x7f108c008040 comp rx=0 tx=0).stop 2024-04-20T10:22:57.125 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.125+0000 7f10a0200640 1 -- 172.21.15.121:0/2806696460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f109814e600 msgr2=0x7f1098159e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:57.125 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.125+0000 7f10a0200640 1 --2- 172.21.15.121:0/2806696460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f109814e600 0x7f1098159e50 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f109000c750 tx=0x7f109000cc10 comp rx=0 tx=0).stop 2024-04-20T10:22:57.125 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.125+0000 7f10a0200640 1 -- 172.21.15.121:0/2806696460 shutdown_connections 2024-04-20T10:22:57.125 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.125+0000 7f10a0200640 1 --2- 172.21.15.121:0/2806696460 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f109814e600 0x7f1098159e50 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:57.125 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.125+0000 7f10a0200640 1 --2- 172.21.15.121:0/2806696460 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f109815b3b0 0x7f109814dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:57.125 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.125+0000 7f10a0200640 1 --2- 172.21.15.121:0/2806696460 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f107806c8f0 0x7f107806edb0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:57.125 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.125+0000 7f10a0200640 1 --2- 172.21.15.121:0/2806696460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f109806dcb0 0x7f109806c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:57.125 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.125+0000 7f10a0200640 1 -- 172.21.15.121:0/2806696460 >> 172.21.15.121:0/2806696460 conn(0x7f10980b3350 msgr2=0x7f10980d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:57.125 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.125+0000 7f10a0200640 1 -- 172.21.15.121:0/2806696460 shutdown_connections 2024-04-20T10:22:57.125 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.125+0000 7f10a0200640 1 -- 172.21.15.121:0/2806696460 wait complete. 2024-04-20T10:22:57.142 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-20T10:22:57.143 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T10:22:57.324 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.325+0000 7f6dce600640 1 Processor -- start 2024-04-20T10:22:57.325 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.325+0000 7f6dce600640 1 -- start start 2024-04-20T10:22:57.326 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.326+0000 7f6dce600640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dc80b2f70 0x7f6dc80b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:57.326 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.326+0000 7f6dce600640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6dc80adf60 0x7f6dc815f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:57.326 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.326+0000 7f6dce600640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6dc815faf0 0x7f6dc806b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:57.326 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.327+0000 7f6dce600640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f6dc8071a80 con 0x7f6dc815faf0 2024-04-20T10:22:57.327 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.327+0000 7f6dce600640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f6dc80ae3c0 con 0x7f6dc80adf60 2024-04-20T10:22:57.327 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.327+0000 7f6dce600640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f6dc8071cf0 con 0x7f6dc80b2f70 2024-04-20T10:22:57.327 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.327+0000 7f6dcdc00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6dc815faf0 0x7f6dc806b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:57.327 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.327+0000 7f6dcdc00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f6dc815faf0 0x7f6dc806b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33248/0 (socket says 172.21.15.121:33248) 2024-04-20T10:22:57.327 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.327+0000 7f6dcdc00640 1 -- 172.21.15.121:0/3625243077 learned_addr learned my addr 172.21.15.121:0/3625243077 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:57.328 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.327+0000 7f6dbfe00640 1 --2- 172.21.15.121:0/3625243077 >> v2:172.21.15.123:3300/0 conn(0x7f6dc80adf60 0x7f6dc815f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:57.328 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.327+0000 7f6dcd200640 1 --2- 172.21.15.121:0/3625243077 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dc80b2f70 0x7f6dc80b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:57.328 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.327+0000 7f6dcdc00640 1 -- 172.21.15.121:0/3625243077 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dc80b2f70 msgr2=0x7f6dc80b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:57.328 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.327+0000 7f6dcdc00640 1 --2- 172.21.15.121:0/3625243077 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dc80b2f70 0x7f6dc80b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:57.328 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.327+0000 7f6dcdc00640 1 -- 172.21.15.121:0/3625243077 >> v2:172.21.15.123:3300/0 conn(0x7f6dc80adf60 msgr2=0x7f6dc815f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:22:57.329 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.327+0000 7f6dcdc00640 1 --2- 172.21.15.121:0/3625243077 >> v2:172.21.15.123:3300/0 conn(0x7f6dc80adf60 0x7f6dc815f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:57.329 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.327+0000 7f6dcdc00640 1 -- 172.21.15.121:0/3625243077 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc8071fd0 con 0x7f6dc815faf0 2024-04-20T10:22:57.329 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.327+0000 7f6dbfe00640 1 --2- 172.21.15.121:0/3625243077 >> v2:172.21.15.123:3300/0 conn(0x7f6dc80adf60 0x7f6dc815f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:57.329 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.327+0000 7f6dcd200640 1 --2- 172.21.15.121:0/3625243077 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dc80b2f70 0x7f6dc80b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:57.329 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.329+0000 7f6dcdc00640 1 --2- 172.21.15.121:0/3625243077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6dc815faf0 0x7f6dc806b900 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8074060 tx=0x7f6dc002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82640fade4f4a1f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:57.329 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.329+0000 7f6dc7e00640 1 -- 172.21.15.121:0/3625243077 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc002ca80 con 0x7f6dc815faf0 2024-04-20T10:22:57.329 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.329+0000 7f6dc7e00640 1 -- 172.21.15.121:0/3625243077 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6dc002cc20 con 0x7f6dc815faf0 2024-04-20T10:22:57.329 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.329+0000 7f6dc7e00640 1 -- 172.21.15.121:0/3625243077 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc00343f0 con 0x7f6dc815faf0 2024-04-20T10:22:57.330 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.329+0000 7f6dce600640 1 -- 172.21.15.121:0/3625243077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6dc815faf0 msgr2=0x7f6dc806b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:57.330 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.329+0000 7f6dce600640 1 --2- 172.21.15.121:0/3625243077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6dc815faf0 0x7f6dc806b900 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8074060 tx=0x7f6dc002bee0 comp rx=0 tx=0).stop 2024-04-20T10:22:57.330 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.329+0000 7f6dce600640 1 -- 172.21.15.121:0/3625243077 shutdown_connections 2024-04-20T10:22:57.330 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.329+0000 7f6dce600640 1 --2- 172.21.15.121:0/3625243077 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6dc815faf0 0x7f6dc806b900 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:57.330 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.329+0000 7f6dce600640 1 --2- 172.21.15.121:0/3625243077 >> v2:172.21.15.123:3300/0 conn(0x7f6dc80adf60 0x7f6dc815f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:57.330 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.329+0000 7f6dce600640 1 --2- 172.21.15.121:0/3625243077 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dc80b2f70 0x7f6dc80b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:57.330 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.329+0000 7f6dce600640 1 -- 172.21.15.121:0/3625243077 >> 172.21.15.121:0/3625243077 conn(0x7f6dc80d0040 msgr2=0x7f6dc80d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:57.330 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.330+0000 7f6dce600640 1 -- 172.21.15.121:0/3625243077 shutdown_connections 2024-04-20T10:22:57.331 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.330+0000 7f6dce600640 1 -- 172.21.15.121:0/3625243077 wait complete. 2024-04-20T10:22:57.331 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.331+0000 7f6dce600640 1 Processor -- start 2024-04-20T10:22:57.331 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.331+0000 7f6dce600640 1 -- start start 2024-04-20T10:22:57.333 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.332+0000 7f6dce600640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6dc80adf60 0x7f6dc81f75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:57.333 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.332+0000 7f6dce600640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dc80b2f70 0x7f6dc809c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:57.333 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.332+0000 7f6dce600640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dc815faf0 0x7f6dc809ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:57.333 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.332+0000 7f6dce600640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc806be40 con 0x7f6dc80adf60 2024-04-20T10:22:57.333 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.332+0000 7f6dce600640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6dc80736c0 con 0x7f6dc815faf0 2024-04-20T10:22:57.334 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.332+0000 7f6dce600640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f6dc806bfc0 con 0x7f6dc80b2f70 2024-04-20T10:22:57.334 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.333+0000 7f6dcd200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6dc80adf60 0x7f6dc81f75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:57.334 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.333+0000 7f6dcd200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6dc80adf60 0x7f6dc81f75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33258/0 (socket says 172.21.15.121:33258) 2024-04-20T10:22:57.334 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.333+0000 7f6dcd200640 1 -- 172.21.15.121:0/449849218 learned_addr learned my addr 172.21.15.121:0/449849218 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:22:57.334 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.333+0000 7f6dcdc00640 1 --2- 172.21.15.121:0/449849218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dc815faf0 0x7f6dc809ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:57.334 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.333+0000 7f6dbfe00640 1 --2- 172.21.15.121:0/449849218 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dc80b2f70 0x7f6dc809c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:57.334 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.333+0000 7f6dcd200640 1 -- 172.21.15.121:0/449849218 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dc80b2f70 msgr2=0x7f6dc809c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:57.334 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.333+0000 7f6dcd200640 1 --2- 172.21.15.121:0/449849218 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dc80b2f70 0x7f6dc809c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:57.334 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.333+0000 7f6dcd200640 1 -- 172.21.15.121:0/449849218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dc815faf0 msgr2=0x7f6dc809ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:57.334 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.333+0000 7f6dcd200640 1 --2- 172.21.15.121:0/449849218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dc815faf0 0x7f6dc809ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:57.334 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.333+0000 7f6dcd200640 1 -- 172.21.15.121:0/449849218 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dc81f7b10 con 0x7f6dc80adf60 2024-04-20T10:22:57.334 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.333+0000 7f6dbfe00640 1 --2- 172.21.15.121:0/449849218 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dc80b2f70 0x7f6dc809c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:22:57.334 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.333+0000 7f6dcd200640 1 --2- 172.21.15.121:0/449849218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6dc80adf60 0x7f6dc81f75d0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f6db800c7f0 tx=0x7f6db800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:57.335 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.334+0000 7f6dc5600640 1 -- 172.21.15.121:0/449849218 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db8015070 con 0x7f6dc80adf60 2024-04-20T10:22:57.335 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.334+0000 7f6dc5600640 1 -- 172.21.15.121:0/449849218 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6db8007a30 con 0x7f6dc80adf60 2024-04-20T10:22:57.335 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.334+0000 7f6dc5600640 1 -- 172.21.15.121:0/449849218 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db8007e00 con 0x7f6dc80adf60 2024-04-20T10:22:57.335 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.334+0000 7f6dce600640 1 -- 172.21.15.121:0/449849218 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dc809b840 con 0x7f6dc80adf60 2024-04-20T10:22:57.335 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.334+0000 7f6dce600640 1 -- 172.21.15.121:0/449849218 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dc809be00 con 0x7f6dc80adf60 2024-04-20T10:22:57.335 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.335+0000 7f6dce600640 1 -- 172.21.15.121:0/449849218 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dc81f47d0 con 0x7f6dc80adf60 2024-04-20T10:22:57.335 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.336+0000 7f6dc5600640 1 -- 172.21.15.121:0/449849218 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6db8018070 con 0x7f6dc80adf60 2024-04-20T10:22:57.337 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.337+0000 7f6dc5600640 1 --2- 172.21.15.121:0/449849218 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f6da406c8f0 0x7f6da406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:22:57.337 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.337+0000 7f6dc5600640 1 -- 172.21.15.121:0/449849218 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f6db808bd60 con 0x7f6dc80adf60 2024-04-20T10:22:57.337 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.337+0000 7f6dbfe00640 1 --2- 172.21.15.121:0/449849218 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f6da406c8f0 0x7f6da406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:22:57.337 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.338+0000 7f6dbfe00640 1 --2- 172.21.15.121:0/449849218 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f6da406c8f0 0x7f6da406edb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6dc80cfa30 tx=0x7f6db0008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:22:57.341 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.342+0000 7f6dc5600640 1 -- 172.21.15.121:0/449849218 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6db8059f60 con 0x7f6dc80adf60 2024-04-20T10:22:57.539 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.539+0000 7f6dce600640 1 -- 172.21.15.121:0/449849218 --> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6dc80729d0 con 0x7f6da406c8f0 2024-04-20T10:22:57.549 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.549+0000 7f6dc5600640 1 -- 172.21.15.121:0/449849218 <== mgr.4103 v2:172.21.15.121:6832/1408233272 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251962 (secure 0 0 0) 0x7f6db00085c0 con 0x7f6da406c8f0 2024-04-20T10:22:57.552 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:22:57.554 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-04-20T10:22:57.555 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.555+0000 7f6dce600640 1 -- 172.21.15.121:0/449849218 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f6da406c8f0 msgr2=0x7f6da406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:57.556 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.555+0000 7f6dce600640 1 --2- 172.21.15.121:0/449849218 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f6da406c8f0 0x7f6da406edb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6dc80cfa30 tx=0x7f6db0008040 comp rx=0 tx=0).stop 2024-04-20T10:22:57.556 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.555+0000 7f6dce600640 1 -- 172.21.15.121:0/449849218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6dc80adf60 msgr2=0x7f6dc81f75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:22:57.556 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.555+0000 7f6dce600640 1 --2- 172.21.15.121:0/449849218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6dc80adf60 0x7f6dc81f75d0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f6db800c7f0 tx=0x7f6db800cbf0 comp rx=0 tx=0).stop 2024-04-20T10:22:57.556 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.555+0000 7f6dce600640 1 -- 172.21.15.121:0/449849218 shutdown_connections 2024-04-20T10:22:57.556 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.555+0000 7f6dce600640 1 --2- 172.21.15.121:0/449849218 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f6da406c8f0 0x7f6da406edb0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:57.556 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.555+0000 7f6dce600640 1 --2- 172.21.15.121:0/449849218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dc815faf0 0x7f6dc809ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:57.556 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.555+0000 7f6dce600640 1 --2- 172.21.15.121:0/449849218 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f6dc80b2f70 0x7f6dc809c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:57.556 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.555+0000 7f6dce600640 1 --2- 172.21.15.121:0/449849218 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f6dc80adf60 0x7f6dc81f75d0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:22:57.556 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.555+0000 7f6dce600640 1 -- 172.21.15.121:0/449849218 >> 172.21.15.121:0/449849218 conn(0x7f6dc80d0040 msgr2=0x7f6dc80afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:22:57.556 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.556+0000 7f6dce600640 1 -- 172.21.15.121:0/449849218 shutdown_connections 2024-04-20T10:22:57.556 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:22:57.556+0000 7f6dce600640 1 -- 172.21.15.121:0/449849218 wait complete. 2024-04-20T10:22:57.572 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2024-04-20T10:22:57.148218+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":168,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220872,"kb_used_data":4712,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753848,"statfs":{"total":773094113280,"available":772867940352,"internally_reserved":0,"allocated":4825088,"data_stored":2732332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.977509"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.252098+0000","last_change":"2024-04-20T10:22:51.209650+0000","last_active":"2024-04-20T10:22:52.252098+0000","last_peered":"2024-04-20T10:22:52.252098+0000","last_clean":"2024-04-20T10:22:52.252098+0000","last_became_active":"2024-04-20T10:22:51.209348+0000","last_became_peered":"2024-04-20T10:22:51.209348+0000","last_unstale":"2024-04-20T10:22:52.252098+0000","last_undegraded":"2024-04-20T10:22:52.252098+0000","last_fullsized":"2024-04-20T10:22:52.252098+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:48:19.378379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.251707+0000","last_change":"2024-04-20T10:22:51.202709+0000","last_active":"2024-04-20T10:22:52.251707+0000","last_peered":"2024-04-20T10:22:52.251707+0000","last_clean":"2024-04-20T10:22:52.251707+0000","last_became_active":"2024-04-20T10:22:51.202407+0000","last_became_peered":"2024-04-20T10:22:51.202407+0000","last_unstale":"2024-04-20T10:22:52.251707+0000","last_undegraded":"2024-04-20T10:22:52.251707+0000","last_fullsized":"2024-04-20T10:22:52.251707+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:09:44.641825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.755737+0000","last_change":"2024-04-20T10:22:51.226968+0000","last_active":"2024-04-20T10:22:52.755737+0000","last_peered":"2024-04-20T10:22:52.755737+0000","last_clean":"2024-04-20T10:22:52.755737+0000","last_became_active":"2024-04-20T10:22:51.226740+0000","last_became_peered":"2024-04-20T10:22:51.226740+0000","last_unstale":"2024-04-20T10:22:52.755737+0000","last_undegraded":"2024-04-20T10:22:52.755737+0000","last_fullsized":"2024-04-20T10:22:52.755737+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:14:15.349647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.208921+0000","last_change":"2024-04-20T10:22:51.196019+0000","last_active":"2024-04-20T10:22:52.208921+0000","last_peered":"2024-04-20T10:22:52.208921+0000","last_clean":"2024-04-20T10:22:52.208921+0000","last_became_active":"2024-04-20T10:22:51.195356+0000","last_became_peered":"2024-04-20T10:22:51.195356+0000","last_unstale":"2024-04-20T10:22:52.208921+0000","last_undegraded":"2024-04-20T10:22:52.208921+0000","last_fullsized":"2024-04-20T10:22:52.208921+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:47:01.151389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.208584+0000","last_change":"2024-04-20T10:22:51.192411+0000","last_active":"2024-04-20T10:22:52.208584+0000","last_peered":"2024-04-20T10:22:52.208584+0000","last_clean":"2024-04-20T10:22:52.208584+0000","last_became_active":"2024-04-20T10:22:51.192102+0000","last_became_peered":"2024-04-20T10:22:51.192102+0000","last_unstale":"2024-04-20T10:22:52.208584+0000","last_undegraded":"2024-04-20T10:22:52.208584+0000","last_fullsized":"2024-04-20T10:22:52.208584+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:20:43.234074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.466749+0000","last_change":"2024-04-20T10:22:51.205790+0000","last_active":"2024-04-20T10:22:52.466749+0000","last_peered":"2024-04-20T10:22:52.466749+0000","last_clean":"2024-04-20T10:22:52.466749+0000","last_became_active":"2024-04-20T10:22:51.205450+0000","last_became_peered":"2024-04-20T10:22:51.205450+0000","last_unstale":"2024-04-20T10:22:52.466749+0000","last_undegraded":"2024-04-20T10:22:52.466749+0000","last_fullsized":"2024-04-20T10:22:52.466749+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:03:26.973820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.758896+0000","last_change":"2024-04-20T10:22:51.219564+0000","last_active":"2024-04-20T10:22:52.758896+0000","last_peered":"2024-04-20T10:22:52.758896+0000","last_clean":"2024-04-20T10:22:52.758896+0000","last_became_active":"2024-04-20T10:22:51.219286+0000","last_became_peered":"2024-04-20T10:22:51.219286+0000","last_unstale":"2024-04-20T10:22:52.758896+0000","last_undegraded":"2024-04-20T10:22:52.758896+0000","last_fullsized":"2024-04-20T10:22:52.758896+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:47:56.543759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.251711+0000","last_change":"2024-04-20T10:22:51.228808+0000","last_active":"2024-04-20T10:22:52.251711+0000","last_peered":"2024-04-20T10:22:52.251711+0000","last_clean":"2024-04-20T10:22:52.251711+0000","last_became_active":"2024-04-20T10:22:51.228540+0000","last_became_peered":"2024-04-20T10:22:51.228540+0000","last_unstale":"2024-04-20T10:22:52.251711+0000","last_undegraded":"2024-04-20T10:22:52.251711+0000","last_fullsized":"2024-04-20T10:22:52.251711+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:36:13.852350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.756183+0000","last_change":"2024-04-20T10:22:51.224598+0000","last_active":"2024-04-20T10:22:52.756183+0000","last_peered":"2024-04-20T10:22:52.756183+0000","last_clean":"2024-04-20T10:22:52.756183+0000","last_became_active":"2024-04-20T10:22:51.224346+0000","last_became_peered":"2024-04-20T10:22:51.224346+0000","last_unstale":"2024-04-20T10:22:52.756183+0000","last_undegraded":"2024-04-20T10:22:52.756183+0000","last_fullsized":"2024-04-20T10:22:52.756183+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:21:24.793211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.759235+0000","last_change":"2024-04-20T10:22:51.222474+0000","last_active":"2024-04-20T10:22:52.759235+0000","last_peered":"2024-04-20T10:22:52.759235+0000","last_clean":"2024-04-20T10:22:52.759235+0000","last_became_active":"2024-04-20T10:22:51.222082+0000","last_became_peered":"2024-04-20T10:22:51.222082+0000","last_unstale":"2024-04-20T10:22:52.759235+0000","last_undegraded":"2024-04-20T10:22:52.759235+0000","last_fullsized":"2024-04-20T10:22:52.759235+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:10:55.371506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.760332+0000","last_change":"2024-04-20T10:22:51.227674+0000","last_active":"2024-04-20T10:22:52.760332+0000","last_peered":"2024-04-20T10:22:52.760332+0000","last_clean":"2024-04-20T10:22:52.760332+0000","last_became_active":"2024-04-20T10:22:51.227004+0000","last_became_peered":"2024-04-20T10:22:51.227004+0000","last_unstale":"2024-04-20T10:22:52.760332+0000","last_undegraded":"2024-04-20T10:22:52.760332+0000","last_fullsized":"2024-04-20T10:22:52.760332+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:45:01.466593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.755741+0000","last_change":"2024-04-20T10:22:51.197530+0000","last_active":"2024-04-20T10:22:52.755741+0000","last_peered":"2024-04-20T10:22:52.755741+0000","last_clean":"2024-04-20T10:22:52.755741+0000","last_became_active":"2024-04-20T10:22:51.197236+0000","last_became_peered":"2024-04-20T10:22:51.197236+0000","last_unstale":"2024-04-20T10:22:52.755741+0000","last_undegraded":"2024-04-20T10:22:52.755741+0000","last_fullsized":"2024-04-20T10:22:52.755741+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:34:34.224444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.759475+0000","last_change":"2024-04-20T10:22:51.225493+0000","last_active":"2024-04-20T10:22:52.759475+0000","last_peered":"2024-04-20T10:22:52.759475+0000","last_clean":"2024-04-20T10:22:52.759475+0000","last_became_active":"2024-04-20T10:22:51.224842+0000","last_became_peered":"2024-04-20T10:22:51.224842+0000","last_unstale":"2024-04-20T10:22:52.759475+0000","last_undegraded":"2024-04-20T10:22:52.759475+0000","last_fullsized":"2024-04-20T10:22:52.759475+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:26:31.997560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.756404+0000","last_change":"2024-04-20T10:22:51.211612+0000","last_active":"2024-04-20T10:22:52.756404+0000","last_peered":"2024-04-20T10:22:52.756404+0000","last_clean":"2024-04-20T10:22:52.756404+0000","last_became_active":"2024-04-20T10:22:51.211335+0000","last_became_peered":"2024-04-20T10:22:51.211335+0000","last_unstale":"2024-04-20T10:22:52.756404+0000","last_undegraded":"2024-04-20T10:22:52.756404+0000","last_fullsized":"2024-04-20T10:22:52.756404+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:12:23.788253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.758575+0000","last_change":"2024-04-20T10:22:51.223950+0000","last_active":"2024-04-20T10:22:52.758575+0000","last_peered":"2024-04-20T10:22:52.758575+0000","last_clean":"2024-04-20T10:22:52.758575+0000","last_became_active":"2024-04-20T10:22:51.221665+0000","last_became_peered":"2024-04-20T10:22:51.221665+0000","last_unstale":"2024-04-20T10:22:52.758575+0000","last_undegraded":"2024-04-20T10:22:52.758575+0000","last_fullsized":"2024-04-20T10:22:52.758575+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:48:00.371087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.756077+0000","last_change":"2024-04-20T10:22:51.230833+0000","last_active":"2024-04-20T10:22:52.756077+0000","last_peered":"2024-04-20T10:22:52.756077+0000","last_clean":"2024-04-20T10:22:52.756077+0000","last_became_active":"2024-04-20T10:22:51.230557+0000","last_became_peered":"2024-04-20T10:22:51.230557+0000","last_unstale":"2024-04-20T10:22:52.756077+0000","last_undegraded":"2024-04-20T10:22:52.756077+0000","last_fullsized":"2024-04-20T10:22:52.756077+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:47:34.076559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.757710+0000","last_change":"2024-04-20T10:22:51.230522+0000","last_active":"2024-04-20T10:22:52.757710+0000","last_peered":"2024-04-20T10:22:52.757710+0000","last_clean":"2024-04-20T10:22:52.757710+0000","last_became_active":"2024-04-20T10:22:51.230212+0000","last_became_peered":"2024-04-20T10:22:51.230212+0000","last_unstale":"2024-04-20T10:22:52.757710+0000","last_undegraded":"2024-04-20T10:22:52.757710+0000","last_fullsized":"2024-04-20T10:22:52.757710+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:48:56.677851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.212475+0000","last_change":"2024-04-20T10:22:51.218344+0000","last_active":"2024-04-20T10:22:52.212475+0000","last_peered":"2024-04-20T10:22:52.212475+0000","last_clean":"2024-04-20T10:22:52.212475+0000","last_became_active":"2024-04-20T10:22:51.217912+0000","last_became_peered":"2024-04-20T10:22:51.217912+0000","last_unstale":"2024-04-20T10:22:52.212475+0000","last_undegraded":"2024-04-20T10:22:52.212475+0000","last_fullsized":"2024-04-20T10:22:52.212475+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:04:29.732467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.208927+0000","last_change":"2024-04-20T10:22:51.200273+0000","last_active":"2024-04-20T10:22:52.208927+0000","last_peered":"2024-04-20T10:22:52.208927+0000","last_clean":"2024-04-20T10:22:52.208927+0000","last_became_active":"2024-04-20T10:22:51.199786+0000","last_became_peered":"2024-04-20T10:22:51.199786+0000","last_unstale":"2024-04-20T10:22:52.208927+0000","last_undegraded":"2024-04-20T10:22:52.208927+0000","last_fullsized":"2024-04-20T10:22:52.208927+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:36:26.004366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.211389+0000","last_change":"2024-04-20T10:22:51.218932+0000","last_active":"2024-04-20T10:22:52.211389+0000","last_peered":"2024-04-20T10:22:52.211389+0000","last_clean":"2024-04-20T10:22:52.211389+0000","last_became_active":"2024-04-20T10:22:51.218623+0000","last_became_peered":"2024-04-20T10:22:51.218623+0000","last_unstale":"2024-04-20T10:22:52.211389+0000","last_undegraded":"2024-04-20T10:22:52.211389+0000","last_fullsized":"2024-04-20T10:22:52.211389+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:16:28.306795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.759730+0000","last_change":"2024-04-20T10:22:51.225710+0000","last_active":"2024-04-20T10:22:52.759730+0000","last_peered":"2024-04-20T10:22:52.759730+0000","last_clean":"2024-04-20T10:22:52.759730+0000","last_became_active":"2024-04-20T10:22:51.225224+0000","last_became_peered":"2024-04-20T10:22:51.225224+0000","last_unstale":"2024-04-20T10:22:52.759730+0000","last_undegraded":"2024-04-20T10:22:52.759730+0000","last_fullsized":"2024-04-20T10:22:52.759730+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:45:39.681890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.757398+0000","last_change":"2024-04-20T10:22:51.198194+0000","last_active":"2024-04-20T10:22:52.757398+0000","last_peered":"2024-04-20T10:22:52.757398+0000","last_clean":"2024-04-20T10:22:52.757398+0000","last_became_active":"2024-04-20T10:22:51.197760+0000","last_became_peered":"2024-04-20T10:22:51.197760+0000","last_unstale":"2024-04-20T10:22:52.757398+0000","last_undegraded":"2024-04-20T10:22:52.757398+0000","last_fullsized":"2024-04-20T10:22:52.757398+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:04:29.732467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.759380+0000","last_change":"2024-04-20T10:22:51.213701+0000","last_active":"2024-04-20T10:22:52.759380+0000","last_peered":"2024-04-20T10:22:52.759380+0000","last_clean":"2024-04-20T10:22:52.759380+0000","last_became_active":"2024-04-20T10:22:51.213384+0000","last_became_peered":"2024-04-20T10:22:51.213384+0000","last_unstale":"2024-04-20T10:22:52.759380+0000","last_undegraded":"2024-04-20T10:22:52.759380+0000","last_fullsized":"2024-04-20T10:22:52.759380+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:07:09.046246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.209189+0000","last_change":"2024-04-20T10:22:51.208012+0000","last_active":"2024-04-20T10:22:52.209189+0000","last_peered":"2024-04-20T10:22:52.209189+0000","last_clean":"2024-04-20T10:22:52.209189+0000","last_became_active":"2024-04-20T10:22:51.207734+0000","last_became_peered":"2024-04-20T10:22:51.207734+0000","last_unstale":"2024-04-20T10:22:52.209189+0000","last_undegraded":"2024-04-20T10:22:52.209189+0000","last_fullsized":"2024-04-20T10:22:52.209189+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:39:49.504337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.212675+0000","last_change":"2024-04-20T10:22:51.209736+0000","last_active":"2024-04-20T10:22:52.212675+0000","last_peered":"2024-04-20T10:22:52.212675+0000","last_clean":"2024-04-20T10:22:52.212675+0000","last_became_active":"2024-04-20T10:22:51.209421+0000","last_became_peered":"2024-04-20T10:22:51.209421+0000","last_unstale":"2024-04-20T10:22:52.212675+0000","last_undegraded":"2024-04-20T10:22:52.212675+0000","last_fullsized":"2024-04-20T10:22:52.212675+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:29:08.806291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.467679+0000","last_change":"2024-04-20T10:22:51.220786+0000","last_active":"2024-04-20T10:22:52.467679+0000","last_peered":"2024-04-20T10:22:52.467679+0000","last_clean":"2024-04-20T10:22:52.467679+0000","last_became_active":"2024-04-20T10:22:51.220227+0000","last_became_peered":"2024-04-20T10:22:51.220227+0000","last_unstale":"2024-04-20T10:22:52.467679+0000","last_undegraded":"2024-04-20T10:22:52.467679+0000","last_fullsized":"2024-04-20T10:22:52.467679+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:56:23.916864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.211907+0000","last_change":"2024-04-20T10:22:51.214621+0000","last_active":"2024-04-20T10:22:52.211907+0000","last_peered":"2024-04-20T10:22:52.211907+0000","last_clean":"2024-04-20T10:22:52.211907+0000","last_became_active":"2024-04-20T10:22:51.214266+0000","last_became_peered":"2024-04-20T10:22:51.214266+0000","last_unstale":"2024-04-20T10:22:52.211907+0000","last_undegraded":"2024-04-20T10:22:52.211907+0000","last_fullsized":"2024-04-20T10:22:52.211907+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:12:23.788253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.760142+0000","last_change":"2024-04-20T10:22:51.222490+0000","last_active":"2024-04-20T10:22:52.760142+0000","last_peered":"2024-04-20T10:22:52.760142+0000","last_clean":"2024-04-20T10:22:52.760142+0000","last_became_active":"2024-04-20T10:22:51.220049+0000","last_became_peered":"2024-04-20T10:22:51.220049+0000","last_unstale":"2024-04-20T10:22:52.760142+0000","last_undegraded":"2024-04-20T10:22:52.760142+0000","last_fullsized":"2024-04-20T10:22:52.760142+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:11:59.306390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.212170+0000","last_change":"2024-04-20T10:22:51.206155+0000","last_active":"2024-04-20T10:22:52.212170+0000","last_peered":"2024-04-20T10:22:52.212170+0000","last_clean":"2024-04-20T10:22:52.212170+0000","last_became_active":"2024-04-20T10:22:51.205785+0000","last_became_peered":"2024-04-20T10:22:51.205785+0000","last_unstale":"2024-04-20T10:22:52.212170+0000","last_undegraded":"2024-04-20T10:22:52.212170+0000","last_fullsized":"2024-04-20T10:22:52.212170+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:23:42.010662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.211436+0000","last_change":"2024-04-20T10:22:51.208148+0000","last_active":"2024-04-20T10:22:52.211436+0000","last_peered":"2024-04-20T10:22:52.211436+0000","last_clean":"2024-04-20T10:22:52.211436+0000","last_became_active":"2024-04-20T10:22:51.207856+0000","last_became_peered":"2024-04-20T10:22:51.207856+0000","last_unstale":"2024-04-20T10:22:52.211436+0000","last_undegraded":"2024-04-20T10:22:52.211436+0000","last_fullsized":"2024-04-20T10:22:52.211436+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:54:46.131724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.759522+0000","last_change":"2024-04-20T10:22:51.214897+0000","last_active":"2024-04-20T10:22:52.759522+0000","last_peered":"2024-04-20T10:22:52.759522+0000","last_clean":"2024-04-20T10:22:52.759522+0000","last_became_active":"2024-04-20T10:22:51.214601+0000","last_became_peered":"2024-04-20T10:22:51.214601+0000","last_unstale":"2024-04-20T10:22:52.759522+0000","last_undegraded":"2024-04-20T10:22:52.759522+0000","last_fullsized":"2024-04-20T10:22:52.759522+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:07:37.714092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.467029+0000","last_change":"2024-04-20T10:22:51.195262+0000","last_active":"2024-04-20T10:22:52.467029+0000","last_peered":"2024-04-20T10:22:52.467029+0000","last_clean":"2024-04-20T10:22:52.467029+0000","last_became_active":"2024-04-20T10:22:51.194993+0000","last_became_peered":"2024-04-20T10:22:51.194993+0000","last_unstale":"2024-04-20T10:22:52.467029+0000","last_undegraded":"2024-04-20T10:22:52.467029+0000","last_fullsized":"2024-04-20T10:22:52.467029+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:14:15.349647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.757434+0000","last_change":"2024-04-20T10:22:51.230960+0000","last_active":"2024-04-20T10:22:52.757434+0000","last_peered":"2024-04-20T10:22:52.757434+0000","last_clean":"2024-04-20T10:22:52.757434+0000","last_became_active":"2024-04-20T10:22:51.228563+0000","last_became_peered":"2024-04-20T10:22:51.228563+0000","last_unstale":"2024-04-20T10:22:52.757434+0000","last_undegraded":"2024-04-20T10:22:52.757434+0000","last_fullsized":"2024-04-20T10:22:52.757434+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:26:44.239012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.212692+0000","last_change":"2024-04-20T10:22:50.208086+0000","last_active":"2024-04-20T10:22:52.212692+0000","last_peered":"2024-04-20T10:22:52.212692+0000","last_clean":"2024-04-20T10:22:52.212692+0000","last_became_active":"2024-04-20T10:22:50.207645+0000","last_became_peered":"2024-04-20T10:22:50.207645+0000","last_unstale":"2024-04-20T10:22:52.212692+0000","last_undegraded":"2024-04-20T10:22:52.212692+0000","last_fullsized":"2024-04-20T10:22:52.212692+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:22:51.962842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.467604+0000","last_change":"2024-04-20T10:22:51.229034+0000","last_active":"2024-04-20T10:22:52.467604+0000","last_peered":"2024-04-20T10:22:52.467604+0000","last_clean":"2024-04-20T10:22:52.467604+0000","last_became_active":"2024-04-20T10:22:51.228786+0000","last_became_peered":"2024-04-20T10:22:51.228786+0000","last_unstale":"2024-04-20T10:22:52.467604+0000","last_undegraded":"2024-04-20T10:22:52.467604+0000","last_fullsized":"2024-04-20T10:22:52.467604+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:21:24.793211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.755785+0000","last_change":"2024-04-20T10:22:50.204532+0000","last_active":"2024-04-20T10:22:52.755785+0000","last_peered":"2024-04-20T10:22:52.755785+0000","last_clean":"2024-04-20T10:22:52.755785+0000","last_became_active":"2024-04-20T10:22:50.204230+0000","last_became_peered":"2024-04-20T10:22:50.204230+0000","last_unstale":"2024-04-20T10:22:52.755785+0000","last_undegraded":"2024-04-20T10:22:52.755785+0000","last_fullsized":"2024-04-20T10:22:52.755785+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:14:18.324437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.760631+0000","last_change":"2024-04-20T10:22:51.205577+0000","last_active":"2024-04-20T10:22:52.760631+0000","last_peered":"2024-04-20T10:22:52.760631+0000","last_clean":"2024-04-20T10:22:52.760631+0000","last_became_active":"2024-04-20T10:22:51.205288+0000","last_became_peered":"2024-04-20T10:22:51.205288+0000","last_unstale":"2024-04-20T10:22:52.760631+0000","last_undegraded":"2024-04-20T10:22:52.760631+0000","last_fullsized":"2024-04-20T10:22:52.760631+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:03:26.973820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.467695+0000","last_change":"2024-04-20T10:22:50.205333+0000","last_active":"2024-04-20T10:22:52.467695+0000","last_peered":"2024-04-20T10:22:52.467695+0000","last_clean":"2024-04-20T10:22:52.467695+0000","last_became_active":"2024-04-20T10:22:50.205029+0000","last_became_peered":"2024-04-20T10:22:50.205029+0000","last_unstale":"2024-04-20T10:22:52.467695+0000","last_undegraded":"2024-04-20T10:22:52.467695+0000","last_fullsized":"2024-04-20T10:22:52.467695+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:46:38.683786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.252123+0000","last_change":"2024-04-20T10:22:51.214233+0000","last_active":"2024-04-20T10:22:52.252123+0000","last_peered":"2024-04-20T10:22:52.252123+0000","last_clean":"2024-04-20T10:22:52.252123+0000","last_became_active":"2024-04-20T10:22:51.213949+0000","last_became_peered":"2024-04-20T10:22:51.213949+0000","last_unstale":"2024-04-20T10:22:52.252123+0000","last_undegraded":"2024-04-20T10:22:52.252123+0000","last_fullsized":"2024-04-20T10:22:52.252123+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:49:13.969709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.759258+0000","last_change":"2024-04-20T10:22:50.207305+0000","last_active":"2024-04-20T10:22:52.759258+0000","last_peered":"2024-04-20T10:22:52.759258+0000","last_clean":"2024-04-20T10:22:52.759258+0000","last_became_active":"2024-04-20T10:22:50.207018+0000","last_became_peered":"2024-04-20T10:22:50.207018+0000","last_unstale":"2024-04-20T10:22:52.759258+0000","last_undegraded":"2024-04-20T10:22:52.759258+0000","last_fullsized":"2024-04-20T10:22:52.759258+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:39:48.488461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.755736+0000","last_change":"2024-04-20T10:22:51.206946+0000","last_active":"2024-04-20T10:22:52.755736+0000","last_peered":"2024-04-20T10:22:52.755736+0000","last_clean":"2024-04-20T10:22:52.755736+0000","last_became_active":"2024-04-20T10:22:51.206670+0000","last_became_peered":"2024-04-20T10:22:51.206670+0000","last_unstale":"2024-04-20T10:22:52.755736+0000","last_undegraded":"2024-04-20T10:22:52.755736+0000","last_fullsized":"2024-04-20T10:22:52.755736+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:03:26.973820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.759643+0000","last_change":"2024-04-20T10:22:50.202177+0000","last_active":"2024-04-20T10:22:52.759643+0000","last_peered":"2024-04-20T10:22:52.759643+0000","last_clean":"2024-04-20T10:22:52.759643+0000","last_became_active":"2024-04-20T10:22:50.201840+0000","last_became_peered":"2024-04-20T10:22:50.201840+0000","last_unstale":"2024-04-20T10:22:52.759643+0000","last_undegraded":"2024-04-20T10:22:52.759643+0000","last_fullsized":"2024-04-20T10:22:52.759643+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:40:01.292259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.758943+0000","last_change":"2024-04-20T10:22:51.227641+0000","last_active":"2024-04-20T10:22:52.758943+0000","last_peered":"2024-04-20T10:22:52.758943+0000","last_clean":"2024-04-20T10:22:52.758943+0000","last_became_active":"2024-04-20T10:22:51.227350+0000","last_became_peered":"2024-04-20T10:22:51.227350+0000","last_unstale":"2024-04-20T10:22:52.758943+0000","last_undegraded":"2024-04-20T10:22:52.758943+0000","last_fullsized":"2024-04-20T10:22:52.758943+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:48:31.200346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.757072+0000","last_change":"2024-04-20T10:22:50.211068+0000","last_active":"2024-04-20T10:22:52.757072+0000","last_peered":"2024-04-20T10:22:52.757072+0000","last_clean":"2024-04-20T10:22:52.757072+0000","last_became_active":"2024-04-20T10:22:50.210709+0000","last_became_peered":"2024-04-20T10:22:50.210709+0000","last_unstale":"2024-04-20T10:22:52.757072+0000","last_undegraded":"2024-04-20T10:22:52.757072+0000","last_fullsized":"2024-04-20T10:22:52.757072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:46:38.683786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.757760+0000","last_change":"2024-04-20T10:22:51.231926+0000","last_active":"2024-04-20T10:22:52.757760+0000","last_peered":"2024-04-20T10:22:52.757760+0000","last_clean":"2024-04-20T10:22:52.757760+0000","last_became_active":"2024-04-20T10:22:51.228859+0000","last_became_peered":"2024-04-20T10:22:51.228859+0000","last_unstale":"2024-04-20T10:22:52.757760+0000","last_undegraded":"2024-04-20T10:22:52.757760+0000","last_fullsized":"2024-04-20T10:22:52.757760+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:29:08.806291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.466834+0000","last_change":"2024-04-20T10:22:50.202082+0000","last_active":"2024-04-20T10:22:52.466834+0000","last_peered":"2024-04-20T10:22:52.466834+0000","last_clean":"2024-04-20T10:22:52.466834+0000","last_became_active":"2024-04-20T10:22:50.201785+0000","last_became_peered":"2024-04-20T10:22:50.201785+0000","last_unstale":"2024-04-20T10:22:52.466834+0000","last_undegraded":"2024-04-20T10:22:52.466834+0000","last_fullsized":"2024-04-20T10:22:52.466834+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:30:35.298814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.251878+0000","last_change":"2024-04-20T10:22:51.211545+0000","last_active":"2024-04-20T10:22:52.251878+0000","last_peered":"2024-04-20T10:22:52.251878+0000","last_clean":"2024-04-20T10:22:52.251878+0000","last_became_active":"2024-04-20T10:22:51.211267+0000","last_became_peered":"2024-04-20T10:22:51.211267+0000","last_unstale":"2024-04-20T10:22:52.251878+0000","last_undegraded":"2024-04-20T10:22:52.251878+0000","last_fullsized":"2024-04-20T10:22:52.251878+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:49:49.694929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.755868+0000","last_change":"2024-04-20T10:22:50.208162+0000","last_active":"2024-04-20T10:22:52.755868+0000","last_peered":"2024-04-20T10:22:52.755868+0000","last_clean":"2024-04-20T10:22:52.755868+0000","last_became_active":"2024-04-20T10:22:50.206236+0000","last_became_peered":"2024-04-20T10:22:50.206236+0000","last_unstale":"2024-04-20T10:22:52.755868+0000","last_undegraded":"2024-04-20T10:22:52.755868+0000","last_fullsized":"2024-04-20T10:22:52.755868+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:22:51.962842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.209067+0000","last_change":"2024-04-20T10:22:51.196295+0000","last_active":"2024-04-20T10:22:52.209067+0000","last_peered":"2024-04-20T10:22:52.209067+0000","last_clean":"2024-04-20T10:22:52.209067+0000","last_became_active":"2024-04-20T10:22:51.195725+0000","last_became_peered":"2024-04-20T10:22:51.195725+0000","last_unstale":"2024-04-20T10:22:52.209067+0000","last_undegraded":"2024-04-20T10:22:52.209067+0000","last_fullsized":"2024-04-20T10:22:52.209067+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:10:16.559701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.252146+0000","last_change":"2024-04-20T10:22:50.196202+0000","last_active":"2024-04-20T10:22:52.252146+0000","last_peered":"2024-04-20T10:22:52.252146+0000","last_clean":"2024-04-20T10:22:52.252146+0000","last_became_active":"2024-04-20T10:22:50.195933+0000","last_became_peered":"2024-04-20T10:22:50.195933+0000","last_unstale":"2024-04-20T10:22:52.252146+0000","last_undegraded":"2024-04-20T10:22:52.252146+0000","last_fullsized":"2024-04-20T10:22:52.252146+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:04:28.716591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.757670+0000","last_change":"2024-04-20T10:22:51.220771+0000","last_active":"2024-04-20T10:22:52.757670+0000","last_peered":"2024-04-20T10:22:52.757670+0000","last_clean":"2024-04-20T10:22:52.757670+0000","last_became_active":"2024-04-20T10:22:51.220461+0000","last_became_peered":"2024-04-20T10:22:51.220461+0000","last_unstale":"2024-04-20T10:22:52.757670+0000","last_undegraded":"2024-04-20T10:22:52.757670+0000","last_fullsized":"2024-04-20T10:22:52.757670+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:39:37.819763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.759845+0000","last_change":"2024-04-20T10:22:50.209570+0000","last_active":"2024-04-20T10:22:52.759845+0000","last_peered":"2024-04-20T10:22:52.759845+0000","last_clean":"2024-04-20T10:22:52.759845+0000","last_became_active":"2024-04-20T10:22:50.209281+0000","last_became_peered":"2024-04-20T10:22:50.209281+0000","last_unstale":"2024-04-20T10:22:52.759845+0000","last_undegraded":"2024-04-20T10:22:52.759845+0000","last_fullsized":"2024-04-20T10:22:52.759845+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:38:10.698270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.756255+0000","last_change":"2024-04-20T10:22:51.207905+0000","last_active":"2024-04-20T10:22:52.756255+0000","last_peered":"2024-04-20T10:22:52.756255+0000","last_clean":"2024-04-20T10:22:52.756255+0000","last_became_active":"2024-04-20T10:22:51.207640+0000","last_became_peered":"2024-04-20T10:22:51.207640+0000","last_unstale":"2024-04-20T10:22:52.756255+0000","last_undegraded":"2024-04-20T10:22:52.756255+0000","last_fullsized":"2024-04-20T10:22:52.756255+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:56:23.916864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.760829+0000","last_change":"2024-04-20T10:22:50.206722+0000","last_active":"2024-04-20T10:22:52.760829+0000","last_peered":"2024-04-20T10:22:52.760829+0000","last_clean":"2024-04-20T10:22:52.760829+0000","last_became_active":"2024-04-20T10:22:50.206395+0000","last_became_peered":"2024-04-20T10:22:50.206395+0000","last_unstale":"2024-04-20T10:22:52.760829+0000","last_undegraded":"2024-04-20T10:22:52.760829+0000","last_fullsized":"2024-04-20T10:22:52.760829+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:55:41.278026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.761351+0000","last_change":"2024-04-20T10:22:51.204717+0000","last_active":"2024-04-20T10:22:52.761351+0000","last_peered":"2024-04-20T10:22:52.761351+0000","last_clean":"2024-04-20T10:22:52.761351+0000","last_became_active":"2024-04-20T10:22:51.203357+0000","last_became_peered":"2024-04-20T10:22:51.203357+0000","last_unstale":"2024-04-20T10:22:52.761351+0000","last_undegraded":"2024-04-20T10:22:52.761351+0000","last_fullsized":"2024-04-20T10:22:52.761351+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:34:34.224444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.251921+0000","last_change":"2024-04-20T10:22:50.203958+0000","last_active":"2024-04-20T10:22:52.251921+0000","last_peered":"2024-04-20T10:22:52.251921+0000","last_clean":"2024-04-20T10:22:52.251921+0000","last_became_active":"2024-04-20T10:22:50.203499+0000","last_became_peered":"2024-04-20T10:22:50.203499+0000","last_unstale":"2024-04-20T10:22:52.251921+0000","last_undegraded":"2024-04-20T10:22:52.251921+0000","last_fullsized":"2024-04-20T10:22:52.251921+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:23:40.994786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.755353+0000","last_change":"2024-04-20T10:22:51.224929+0000","last_active":"2024-04-20T10:22:52.755353+0000","last_peered":"2024-04-20T10:22:52.755353+0000","last_clean":"2024-04-20T10:22:52.755353+0000","last_became_active":"2024-04-20T10:22:51.224646+0000","last_became_peered":"2024-04-20T10:22:51.224646+0000","last_unstale":"2024-04-20T10:22:52.755353+0000","last_undegraded":"2024-04-20T10:22:52.755353+0000","last_fullsized":"2024-04-20T10:22:52.755353+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:29:08.806291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.251994+0000","last_change":"2024-04-20T10:22:50.196624+0000","last_active":"2024-04-20T10:22:52.251994+0000","last_peered":"2024-04-20T10:22:52.251994+0000","last_clean":"2024-04-20T10:22:52.251994+0000","last_became_active":"2024-04-20T10:22:50.196295+0000","last_became_peered":"2024-04-20T10:22:50.196295+0000","last_unstale":"2024-04-20T10:22:52.251994+0000","last_undegraded":"2024-04-20T10:22:52.251994+0000","last_fullsized":"2024-04-20T10:22:52.251994+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:12:22.772377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.251863+0000","last_change":"2024-04-20T10:22:51.200209+0000","last_active":"2024-04-20T10:22:52.251863+0000","last_peered":"2024-04-20T10:22:52.251863+0000","last_clean":"2024-04-20T10:22:52.251863+0000","last_became_active":"2024-04-20T10:22:51.199938+0000","last_became_peered":"2024-04-20T10:22:51.199938+0000","last_unstale":"2024-04-20T10:22:52.251863+0000","last_undegraded":"2024-04-20T10:22:52.251863+0000","last_fullsized":"2024-04-20T10:22:52.251863+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:07:37.714092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.466874+0000","last_change":"2024-04-20T10:22:50.200183+0000","last_active":"2024-04-20T10:22:52.466874+0000","last_peered":"2024-04-20T10:22:52.466874+0000","last_clean":"2024-04-20T10:22:52.466874+0000","last_became_active":"2024-04-20T10:22:50.199884+0000","last_became_peered":"2024-04-20T10:22:50.199884+0000","last_unstale":"2024-04-20T10:22:52.466874+0000","last_undegraded":"2024-04-20T10:22:52.466874+0000","last_fullsized":"2024-04-20T10:22:52.466874+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:20:05.462979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.756728+0000","last_change":"2024-04-20T10:22:51.214680+0000","last_active":"2024-04-20T10:22:52.756728+0000","last_peered":"2024-04-20T10:22:52.756728+0000","last_clean":"2024-04-20T10:22:52.756728+0000","last_became_active":"2024-04-20T10:22:51.213945+0000","last_became_peered":"2024-04-20T10:22:51.213945+0000","last_unstale":"2024-04-20T10:22:52.756728+0000","last_undegraded":"2024-04-20T10:22:52.756728+0000","last_fullsized":"2024-04-20T10:22:52.756728+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:56:23.916864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.211063+0000","last_change":"2024-04-20T10:22:50.205765+0000","last_active":"2024-04-20T10:22:52.211063+0000","last_peered":"2024-04-20T10:22:52.211063+0000","last_clean":"2024-04-20T10:22:52.211063+0000","last_became_active":"2024-04-20T10:22:50.205454+0000","last_became_peered":"2024-04-20T10:22:50.205454+0000","last_unstale":"2024-04-20T10:22:52.211063+0000","last_undegraded":"2024-04-20T10:22:52.211063+0000","last_fullsized":"2024-04-20T10:22:52.211063+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:10:15.543826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.251812+0000","last_change":"2024-04-20T10:22:51.210921+0000","last_active":"2024-04-20T10:22:52.251812+0000","last_peered":"2024-04-20T10:22:52.251812+0000","last_clean":"2024-04-20T10:22:52.251812+0000","last_became_active":"2024-04-20T10:22:51.210599+0000","last_became_peered":"2024-04-20T10:22:51.210599+0000","last_unstale":"2024-04-20T10:22:52.251812+0000","last_undegraded":"2024-04-20T10:22:52.251812+0000","last_fullsized":"2024-04-20T10:22:52.251812+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:07:09.046246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":22,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.756594+0000","last_change":"2024-04-20T10:22:50.210848+0000","last_active":"2024-04-20T10:22:52.756594+0000","last_peered":"2024-04-20T10:22:52.756594+0000","last_clean":"2024-04-20T10:22:52.756594+0000","last_became_active":"2024-04-20T10:22:50.210382+0000","last_became_peered":"2024-04-20T10:22:50.210382+0000","last_unstale":"2024-04-20T10:22:52.756594+0000","last_undegraded":"2024-04-20T10:22:52.756594+0000","last_fullsized":"2024-04-20T10:22:52.756594+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:20:05.462979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.251902+0000","last_change":"2024-04-20T10:22:51.208008+0000","last_active":"2024-04-20T10:22:52.251902+0000","last_peered":"2024-04-20T10:22:52.251902+0000","last_clean":"2024-04-20T10:22:52.251902+0000","last_became_active":"2024-04-20T10:22:51.207730+0000","last_became_peered":"2024-04-20T10:22:51.207730+0000","last_unstale":"2024-04-20T10:22:52.251902+0000","last_undegraded":"2024-04-20T10:22:52.251902+0000","last_fullsized":"2024-04-20T10:22:52.251902+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:55:04.112006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.760461+0000","last_change":"2024-04-20T10:22:50.208371+0000","last_active":"2024-04-20T10:22:52.760461+0000","last_peered":"2024-04-20T10:22:52.760461+0000","last_clean":"2024-04-20T10:22:52.760461+0000","last_became_active":"2024-04-20T10:22:50.208102+0000","last_became_peered":"2024-04-20T10:22:50.208102+0000","last_unstale":"2024-04-20T10:22:52.760461+0000","last_undegraded":"2024-04-20T10:22:52.760461+0000","last_fullsized":"2024-04-20T10:22:52.760461+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:52:46.611087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.760630+0000","last_change":"2024-04-20T10:22:51.226634+0000","last_active":"2024-04-20T10:22:52.760630+0000","last_peered":"2024-04-20T10:22:52.760630+0000","last_clean":"2024-04-20T10:22:52.760630+0000","last_became_active":"2024-04-20T10:22:51.223859+0000","last_became_peered":"2024-04-20T10:22:51.223859+0000","last_unstale":"2024-04-20T10:22:52.760630+0000","last_undegraded":"2024-04-20T10:22:52.760630+0000","last_fullsized":"2024-04-20T10:22:52.760630+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:08:49.220198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.760666+0000","last_change":"2024-04-20T10:22:50.205722+0000","last_active":"2024-04-20T10:22:52.760666+0000","last_peered":"2024-04-20T10:22:52.760666+0000","last_clean":"2024-04-20T10:22:52.760666+0000","last_became_active":"2024-04-20T10:22:50.205402+0000","last_became_peered":"2024-04-20T10:22:50.205402+0000","last_unstale":"2024-04-20T10:22:52.760666+0000","last_undegraded":"2024-04-20T10:22:52.760666+0000","last_fullsized":"2024-04-20T10:22:52.760666+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:16:59.003634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.760401+0000","last_change":"2024-04-20T10:22:51.198910+0000","last_active":"2024-04-20T10:22:52.760401+0000","last_peered":"2024-04-20T10:22:52.760401+0000","last_clean":"2024-04-20T10:22:52.760401+0000","last_became_active":"2024-04-20T10:22:51.198452+0000","last_became_peered":"2024-04-20T10:22:51.198452+0000","last_unstale":"2024-04-20T10:22:52.760401+0000","last_undegraded":"2024-04-20T10:22:52.760401+0000","last_fullsized":"2024-04-20T10:22:52.760401+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:32:30.754118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.209015+0000","last_change":"2024-04-20T10:22:50.206721+0000","last_active":"2024-04-20T10:22:52.209015+0000","last_peered":"2024-04-20T10:22:52.209015+0000","last_clean":"2024-04-20T10:22:52.209015+0000","last_became_active":"2024-04-20T10:22:50.206370+0000","last_became_peered":"2024-04-20T10:22:50.206370+0000","last_unstale":"2024-04-20T10:22:52.209015+0000","last_undegraded":"2024-04-20T10:22:52.209015+0000","last_fullsized":"2024-04-20T10:22:52.209015+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:04:57.713148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.209138+0000","last_change":"2024-04-20T10:22:51.196095+0000","last_active":"2024-04-20T10:22:52.209138+0000","last_peered":"2024-04-20T10:22:52.209138+0000","last_clean":"2024-04-20T10:22:52.209138+0000","last_became_active":"2024-04-20T10:22:51.195784+0000","last_became_peered":"2024-04-20T10:22:51.195784+0000","last_unstale":"2024-04-20T10:22:52.209138+0000","last_undegraded":"2024-04-20T10:22:52.209138+0000","last_fullsized":"2024-04-20T10:22:52.209138+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:22:52.978718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.467282+0000","last_change":"2024-04-20T10:22:50.205027+0000","last_active":"2024-04-20T10:22:52.467282+0000","last_peered":"2024-04-20T10:22:52.467282+0000","last_clean":"2024-04-20T10:22:52.467282+0000","last_became_active":"2024-04-20T10:22:50.204749+0000","last_became_peered":"2024-04-20T10:22:50.204749+0000","last_unstale":"2024-04-20T10:22:52.467282+0000","last_undegraded":"2024-04-20T10:22:52.467282+0000","last_fullsized":"2024-04-20T10:22:52.467282+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:14:18.324437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.756642+0000","last_change":"2024-04-20T10:22:51.222374+0000","last_active":"2024-04-20T10:22:52.756642+0000","last_peered":"2024-04-20T10:22:52.756642+0000","last_clean":"2024-04-20T10:22:52.756642+0000","last_became_active":"2024-04-20T10:22:51.221790+0000","last_became_peered":"2024-04-20T10:22:51.221790+0000","last_unstale":"2024-04-20T10:22:52.756642+0000","last_undegraded":"2024-04-20T10:22:52.756642+0000","last_fullsized":"2024-04-20T10:22:52.756642+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:21:24.793211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.208680+0000","last_change":"2024-04-20T10:22:50.194920+0000","last_active":"2024-04-20T10:22:52.208680+0000","last_peered":"2024-04-20T10:22:52.208680+0000","last_clean":"2024-04-20T10:22:52.208680+0000","last_became_active":"2024-04-20T10:22:50.194622+0000","last_became_peered":"2024-04-20T10:22:50.194622+0000","last_unstale":"2024-04-20T10:22:52.208680+0000","last_undegraded":"2024-04-20T10:22:52.208680+0000","last_fullsized":"2024-04-20T10:22:52.208680+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:06:56.248670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.755381+0000","last_change":"2024-04-20T10:22:51.200318+0000","last_active":"2024-04-20T10:22:52.755381+0000","last_peered":"2024-04-20T10:22:52.755381+0000","last_clean":"2024-04-20T10:22:52.755381+0000","last_became_active":"2024-04-20T10:22:51.200056+0000","last_became_peered":"2024-04-20T10:22:51.200056+0000","last_unstale":"2024-04-20T10:22:52.755381+0000","last_undegraded":"2024-04-20T10:22:52.755381+0000","last_fullsized":"2024-04-20T10:22:52.755381+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:32:30.754118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.208847+0000","last_change":"2024-04-20T10:22:50.202646+0000","last_active":"2024-04-20T10:22:52.208847+0000","last_peered":"2024-04-20T10:22:52.208847+0000","last_clean":"2024-04-20T10:22:52.208847+0000","last_became_active":"2024-04-20T10:22:50.202266+0000","last_became_peered":"2024-04-20T10:22:50.202266+0000","last_unstale":"2024-04-20T10:22:52.208847+0000","last_undegraded":"2024-04-20T10:22:52.208847+0000","last_fullsized":"2024-04-20T10:22:52.208847+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:04:33.741440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.758860+0000","last_change":"2024-04-20T10:22:51.217439+0000","last_active":"2024-04-20T10:22:52.758860+0000","last_peered":"2024-04-20T10:22:52.758860+0000","last_clean":"2024-04-20T10:22:52.758860+0000","last_became_active":"2024-04-20T10:22:51.216616+0000","last_became_peered":"2024-04-20T10:22:51.216616+0000","last_unstale":"2024-04-20T10:22:52.758860+0000","last_undegraded":"2024-04-20T10:22:52.758860+0000","last_fullsized":"2024-04-20T10:22:52.758860+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:04:16.514782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.759616+0000","last_change":"2024-04-20T10:22:50.205146+0000","last_active":"2024-04-20T10:22:52.759616+0000","last_peered":"2024-04-20T10:22:52.759616+0000","last_clean":"2024-04-20T10:22:52.759616+0000","last_became_active":"2024-04-20T10:22:50.204772+0000","last_became_peered":"2024-04-20T10:22:50.204772+0000","last_unstale":"2024-04-20T10:22:52.759616+0000","last_undegraded":"2024-04-20T10:22:52.759616+0000","last_fullsized":"2024-04-20T10:22:52.759616+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:14:18.324437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.251959+0000","last_change":"2024-04-20T10:22:51.200462+0000","last_active":"2024-04-20T10:22:52.251959+0000","last_peered":"2024-04-20T10:22:52.251959+0000","last_clean":"2024-04-20T10:22:52.251959+0000","last_became_active":"2024-04-20T10:22:51.200148+0000","last_became_peered":"2024-04-20T10:22:51.200148+0000","last_unstale":"2024-04-20T10:22:52.251959+0000","last_undegraded":"2024-04-20T10:22:52.251959+0000","last_fullsized":"2024-04-20T10:22:52.251959+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:49:52.511861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.209268+0000","last_change":"2024-04-20T10:22:50.194938+0000","last_active":"2024-04-20T10:22:52.209268+0000","last_peered":"2024-04-20T10:22:52.209268+0000","last_clean":"2024-04-20T10:22:52.209268+0000","last_became_active":"2024-04-20T10:22:50.194631+0000","last_became_peered":"2024-04-20T10:22:50.194631+0000","last_unstale":"2024-04-20T10:22:52.209268+0000","last_undegraded":"2024-04-20T10:22:52.209268+0000","last_fullsized":"2024-04-20T10:22:52.209268+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:28:14.564063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.467289+0000","last_change":"2024-04-20T10:22:51.230038+0000","last_active":"2024-04-20T10:22:52.467289+0000","last_peered":"2024-04-20T10:22:52.467289+0000","last_clean":"2024-04-20T10:22:52.467289+0000","last_became_active":"2024-04-20T10:22:51.229752+0000","last_became_peered":"2024-04-20T10:22:51.229752+0000","last_unstale":"2024-04-20T10:22:52.467289+0000","last_undegraded":"2024-04-20T10:22:52.467289+0000","last_fullsized":"2024-04-20T10:22:52.467289+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:47:34.076559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.760724+0000","last_change":"2024-04-20T10:22:50.200577+0000","last_active":"2024-04-20T10:22:52.760724+0000","last_peered":"2024-04-20T10:22:52.760724+0000","last_clean":"2024-04-20T10:22:52.760724+0000","last_became_active":"2024-04-20T10:22:50.199510+0000","last_became_peered":"2024-04-20T10:22:50.199510+0000","last_unstale":"2024-04-20T10:22:52.760724+0000","last_undegraded":"2024-04-20T10:22:52.760724+0000","last_fullsized":"2024-04-20T10:22:52.760724+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:32:22.264498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.758090+0000","last_change":"2024-04-20T10:22:51.224530+0000","last_active":"2024-04-20T10:22:52.758090+0000","last_peered":"2024-04-20T10:22:52.758090+0000","last_clean":"2024-04-20T10:22:52.758090+0000","last_became_active":"2024-04-20T10:22:51.224210+0000","last_became_peered":"2024-04-20T10:22:51.224210+0000","last_unstale":"2024-04-20T10:22:52.758090+0000","last_undegraded":"2024-04-20T10:22:52.758090+0000","last_fullsized":"2024-04-20T10:22:52.758090+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:36:13.852350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"39'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.209010+0000","last_change":"2024-04-20T10:22:50.202138+0000","last_active":"2024-04-20T10:22:52.209010+0000","last_peered":"2024-04-20T10:22:52.209010+0000","last_clean":"2024-04-20T10:22:52.209010+0000","last_became_active":"2024-04-20T10:22:50.201808+0000","last_became_peered":"2024-04-20T10:22:50.201808+0000","last_unstale":"2024-04-20T10:22:52.209010+0000","last_undegraded":"2024-04-20T10:22:52.209010+0000","last_fullsized":"2024-04-20T10:22:52.209010+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:09:39.714678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.251864+0000","last_change":"2024-04-20T10:22:51.209475+0000","last_active":"2024-04-20T10:22:52.251864+0000","last_peered":"2024-04-20T10:22:52.251864+0000","last_clean":"2024-04-20T10:22:52.251864+0000","last_became_active":"2024-04-20T10:22:51.208015+0000","last_became_peered":"2024-04-20T10:22:51.208015+0000","last_unstale":"2024-04-20T10:22:52.251864+0000","last_undegraded":"2024-04-20T10:22:52.251864+0000","last_fullsized":"2024-04-20T10:22:52.251864+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:31:20.035828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.755614+0000","last_change":"2024-04-20T10:22:50.196011+0000","last_active":"2024-04-20T10:22:52.755614+0000","last_peered":"2024-04-20T10:22:52.755614+0000","last_clean":"2024-04-20T10:22:52.755614+0000","last_became_active":"2024-04-20T10:22:50.195748+0000","last_became_peered":"2024-04-20T10:22:50.195748+0000","last_unstale":"2024-04-20T10:22:52.755614+0000","last_undegraded":"2024-04-20T10:22:52.755614+0000","last_fullsized":"2024-04-20T10:22:52.755614+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:16:59.003634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.755796+0000","last_change":"2024-04-20T10:22:51.209020+0000","last_active":"2024-04-20T10:22:52.755796+0000","last_peered":"2024-04-20T10:22:52.755796+0000","last_clean":"2024-04-20T10:22:52.755796+0000","last_became_active":"2024-04-20T10:22:51.208794+0000","last_became_peered":"2024-04-20T10:22:51.208794+0000","last_unstale":"2024-04-20T10:22:52.755796+0000","last_undegraded":"2024-04-20T10:22:52.755796+0000","last_fullsized":"2024-04-20T10:22:52.755796+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:47:34.076559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.755851+0000","last_change":"2024-04-20T10:22:50.203809+0000","last_active":"2024-04-20T10:22:52.755851+0000","last_peered":"2024-04-20T10:22:52.755851+0000","last_clean":"2024-04-20T10:22:52.755851+0000","last_became_active":"2024-04-20T10:22:50.203451+0000","last_became_peered":"2024-04-20T10:22:50.203451+0000","last_unstale":"2024-04-20T10:22:52.755851+0000","last_undegraded":"2024-04-20T10:22:52.755851+0000","last_fullsized":"2024-04-20T10:22:52.755851+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:39:48.488461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.466825+0000","last_change":"2024-04-20T10:22:51.208821+0000","last_active":"2024-04-20T10:22:52.466825+0000","last_peered":"2024-04-20T10:22:52.466825+0000","last_clean":"2024-04-20T10:22:52.466825+0000","last_became_active":"2024-04-20T10:22:51.208525+0000","last_became_peered":"2024-04-20T10:22:51.208525+0000","last_unstale":"2024-04-20T10:22:52.466825+0000","last_undegraded":"2024-04-20T10:22:52.466825+0000","last_fullsized":"2024-04-20T10:22:52.466825+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:34:34.224444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.211375+0000","last_change":"2024-04-20T10:22:50.197399+0000","last_active":"2024-04-20T10:22:52.211375+0000","last_peered":"2024-04-20T10:22:52.211375+0000","last_clean":"2024-04-20T10:22:52.211375+0000","last_became_active":"2024-04-20T10:22:50.197015+0000","last_became_peered":"2024-04-20T10:22:50.197015+0000","last_unstale":"2024-04-20T10:22:52.211375+0000","last_undegraded":"2024-04-20T10:22:52.211375+0000","last_fullsized":"2024-04-20T10:22:52.211375+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:47:00.135513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.756824+0000","last_change":"2024-04-20T10:22:51.212368+0000","last_active":"2024-04-20T10:22:52.756824+0000","last_peered":"2024-04-20T10:22:52.756824+0000","last_clean":"2024-04-20T10:22:52.756824+0000","last_became_active":"2024-04-20T10:22:51.208131+0000","last_became_peered":"2024-04-20T10:22:51.208131+0000","last_unstale":"2024-04-20T10:22:52.756824+0000","last_undegraded":"2024-04-20T10:22:52.756824+0000","last_fullsized":"2024-04-20T10:22:52.756824+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:23:42.010662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.756079+0000","last_change":"2024-04-20T10:22:50.203369+0000","last_active":"2024-04-20T10:22:52.756079+0000","last_peered":"2024-04-20T10:22:52.756079+0000","last_clean":"2024-04-20T10:22:52.756079+0000","last_became_active":"2024-04-20T10:22:50.203083+0000","last_became_peered":"2024-04-20T10:22:50.203083+0000","last_unstale":"2024-04-20T10:22:52.756079+0000","last_undegraded":"2024-04-20T10:22:52.756079+0000","last_fullsized":"2024-04-20T10:22:52.756079+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:10:15.543826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.757719+0000","last_change":"2024-04-20T10:22:51.219412+0000","last_active":"2024-04-20T10:22:52.757719+0000","last_peered":"2024-04-20T10:22:52.757719+0000","last_clean":"2024-04-20T10:22:52.757719+0000","last_became_active":"2024-04-20T10:22:51.218872+0000","last_became_peered":"2024-04-20T10:22:51.218872+0000","last_unstale":"2024-04-20T10:22:52.757719+0000","last_undegraded":"2024-04-20T10:22:52.757719+0000","last_fullsized":"2024-04-20T10:22:52.757719+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:09:44.641825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.251590+0000","last_change":"2024-04-20T10:22:50.198717+0000","last_active":"2024-04-20T10:22:52.251590+0000","last_peered":"2024-04-20T10:22:52.251590+0000","last_clean":"2024-04-20T10:22:52.251590+0000","last_became_active":"2024-04-20T10:22:50.197112+0000","last_became_peered":"2024-04-20T10:22:50.197112+0000","last_unstale":"2024-04-20T10:22:52.251590+0000","last_undegraded":"2024-04-20T10:22:52.251590+0000","last_fullsized":"2024-04-20T10:22:52.251590+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:54:45.115849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.760929+0000","last_change":"2024-04-20T10:22:51.226656+0000","last_active":"2024-04-20T10:22:52.760929+0000","last_peered":"2024-04-20T10:22:52.760929+0000","last_clean":"2024-04-20T10:22:52.760929+0000","last_became_active":"2024-04-20T10:22:51.226325+0000","last_became_peered":"2024-04-20T10:22:51.226325+0000","last_unstale":"2024-04-20T10:22:52.760929+0000","last_undegraded":"2024-04-20T10:22:52.760929+0000","last_fullsized":"2024-04-20T10:22:52.760929+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:15:26.440918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.757755+0000","last_change":"2024-04-20T10:22:50.203854+0000","last_active":"2024-04-20T10:22:52.757755+0000","last_peered":"2024-04-20T10:22:52.757755+0000","last_clean":"2024-04-20T10:22:52.757755+0000","last_became_active":"2024-04-20T10:22:50.203439+0000","last_became_peered":"2024-04-20T10:22:50.203439+0000","last_unstale":"2024-04-20T10:22:52.757755+0000","last_undegraded":"2024-04-20T10:22:52.757755+0000","last_fullsized":"2024-04-20T10:22:52.757755+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:30:35.298814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":203,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.757056+0000","last_change":"2024-04-20T10:20:52.916129+0000","last_active":"2024-04-20T10:22:52.757056+0000","last_peered":"2024-04-20T10:22:52.757056+0000","last_clean":"2024-04-20T10:22:52.757056+0000","last_became_active":"2024-04-20T10:20:52.913922+0000","last_became_peered":"2024-04-20T10:20:52.913922+0000","last_unstale":"2024-04-20T10:22:52.757056+0000","last_undegraded":"2024-04-20T10:22:52.757056+0000","last_fullsized":"2024-04-20T10:22:52.757056+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:20:51.267588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:20:51.267588+0000","last_clean_scrub_stamp":"2024-04-20T10:20:51.267588+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:30:44.465836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542171,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1056768,"data_stored":785397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542172,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":193929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542171,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":193459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542171,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":193951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542171,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":462848,"data_stored":194993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542171,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":193417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542173,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":193437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542173,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":1044480,"data_stored":783749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T10:23:00.577 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T10:23:00.789 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.790+0000 7f1d63800640 1 Processor -- start 2024-04-20T10:23:00.790 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.790+0000 7f1d63800640 1 -- start start 2024-04-20T10:23:00.791 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.791+0000 7f1d63800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d5c156c20 0x7f1d5c06b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:00.791 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.791+0000 7f1d63800640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1d5c15ca40 0x7f1d5c0ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:00.791 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.791+0000 7f1d63800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d5c141c60 con 0x7f1d5c0ad8e0 2024-04-20T10:23:00.791 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.791+0000 7f1d63800640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f1d5c15cf90 con 0x7f1d5c15ca40 2024-04-20T10:23:00.791 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.791+0000 7f1d63800640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f1d5c15d270 con 0x7f1d5c156c20 2024-04-20T10:23:00.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.792+0000 7f1d5be00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1d5c15ca40 0x7f1d5c0ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:00.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.792+0000 7f1d60e00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d5c156c20 0x7f1d5c06b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:00.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.792+0000 7f1d5be00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1d5c15ca40 0x7f1d5c0ad3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.121:44168/0 (socket says 172.21.15.121:44168) 2024-04-20T10:23:00.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.792+0000 7f1d5be00640 1 -- 172.21.15.121:0/1608748640 learned_addr learned my addr 172.21.15.121:0/1608748640 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:23:00.792 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.792+0000 7f1d61800640 1 --1- 172.21.15.121:0/1608748640 >> v1:172.21.15.123:6789/0 conn(0x7f1d5c0ad8e0 0x7f1d5c0d0360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.121:57756/0 (socket says 172.21.15.121:57756) 2024-04-20T10:23:00.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.793+0000 7f1d5be00640 1 -- 172.21.15.121:0/1608748640 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d5c156c20 msgr2=0x7f1d5c06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:23:00.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.793+0000 7f1d5be00640 1 --2- 172.21.15.121:0/1608748640 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d5c156c20 0x7f1d5c06b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:00.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.793+0000 7f1d5be00640 1 -- 172.21.15.121:0/1608748640 >> v1:172.21.15.123:6789/0 conn(0x7f1d5c0ad8e0 legacy=0x7f1d5c0d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:00.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.793+0000 7f1d5be00640 1 -- 172.21.15.121:0/1608748640 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d5c15d550 con 0x7f1d5c15ca40 2024-04-20T10:23:00.793 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.793+0000 7f1d5b400640 1 -- 172.21.15.121:0/1608748640 <== mon.1 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3917337418 0 0) 0x7f1d5c141c60 con 0x7f1d5c0ad8e0 2024-04-20T10:23:00.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.794+0000 7f1d5be00640 1 --2- 172.21.15.121:0/1608748640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d5c15ca40 0x7f1d5c0ad3a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1d48009870 tx=0x7f1d4802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0d666de14a0bce0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:00.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.795+0000 7f1d5b400640 1 -- 172.21.15.121:0/1608748640 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d48034070 con 0x7f1d5c15ca40 2024-04-20T10:23:00.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.795+0000 7f1d5b400640 1 -- 172.21.15.121:0/1608748640 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d48030840 con 0x7f1d5c15ca40 2024-04-20T10:23:00.794 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.795+0000 7f1d5b400640 1 -- 172.21.15.121:0/1608748640 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d48030c10 con 0x7f1d5c15ca40 2024-04-20T10:23:00.795 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.795+0000 7f1d63800640 1 -- 172.21.15.121:0/1608748640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d5c15ca40 msgr2=0x7f1d5c0ad3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:00.795 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.795+0000 7f1d63800640 1 --2- 172.21.15.121:0/1608748640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d5c15ca40 0x7f1d5c0ad3a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1d48009870 tx=0x7f1d4802b980 comp rx=0 tx=0).stop 2024-04-20T10:23:00.795 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.795+0000 7f1d63800640 1 -- 172.21.15.121:0/1608748640 shutdown_connections 2024-04-20T10:23:00.795 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.796+0000 7f1d63800640 1 --2- 172.21.15.121:0/1608748640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d5c15ca40 0x7f1d5c0ad3a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:00.795 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.796+0000 7f1d63800640 1 --2- 172.21.15.121:0/1608748640 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d5c156c20 0x7f1d5c06b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:00.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.796+0000 7f1d63800640 1 -- 172.21.15.121:0/1608748640 >> 172.21.15.121:0/1608748640 conn(0x7f1d5c0b3350 msgr2=0x7f1d5c0b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:00.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.796+0000 7f1d63800640 1 -- 172.21.15.121:0/1608748640 shutdown_connections 2024-04-20T10:23:00.796 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.796+0000 7f1d63800640 1 -- 172.21.15.121:0/1608748640 wait complete. 2024-04-20T10:23:00.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.797+0000 7f1d63800640 1 Processor -- start 2024-04-20T10:23:00.797 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.798+0000 7f1d63800640 1 -- start start 2024-04-20T10:23:00.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.798+0000 7f1d63800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d5c0ad8e0 0x7f1d5c06dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:00.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.798+0000 7f1d63800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d5c156c20 0x7f1d5c0b0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:00.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.798+0000 7f1d63800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d5c15ca40 0x7f1d5c0b1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:00.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.798+0000 7f1d63800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5c06bfc0 con 0x7f1d5c156c20 2024-04-20T10:23:00.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.798+0000 7f1d63800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5c06be40 con 0x7f1d5c0ad8e0 2024-04-20T10:23:00.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.798+0000 7f1d63800640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f1d5c0d0a70 con 0x7f1d5c15ca40 2024-04-20T10:23:00.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.799+0000 7f1d5be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d5c156c20 0x7f1d5c0b0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:00.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.799+0000 7f1d5be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d5c156c20 0x7f1d5c0b0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33262/0 (socket says 172.21.15.121:33262) 2024-04-20T10:23:00.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.799+0000 7f1d5be00640 1 -- 172.21.15.121:0/4084445826 learned_addr learned my addr 172.21.15.121:0/4084445826 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:23:00.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.799+0000 7f1d60e00640 1 --2- 172.21.15.121:0/4084445826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d5c0ad8e0 0x7f1d5c06dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:00.799 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.799+0000 7f1d61800640 1 --2- 172.21.15.121:0/4084445826 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d5c15ca40 0x7f1d5c0b1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:00.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.799+0000 7f1d5be00640 1 -- 172.21.15.121:0/4084445826 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d5c15ca40 msgr2=0x7f1d5c0b1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:00.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.799+0000 7f1d5be00640 1 --2- 172.21.15.121:0/4084445826 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d5c15ca40 0x7f1d5c0b1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:00.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.799+0000 7f1d5be00640 1 -- 172.21.15.121:0/4084445826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d5c0ad8e0 msgr2=0x7f1d5c06dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:00.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.799+0000 7f1d5be00640 1 --2- 172.21.15.121:0/4084445826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d5c0ad8e0 0x7f1d5c06dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:00.800 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.799+0000 7f1d5be00640 1 -- 172.21.15.121:0/4084445826 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d5c06f110 con 0x7f1d5c156c20 2024-04-20T10:23:00.801 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.800+0000 7f1d5be00640 1 --2- 172.21.15.121:0/4084445826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d5c156c20 0x7f1d5c0b0e00 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f1d480090b0 tx=0x7f1d480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:00.801 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.800+0000 7f1d58c00640 1 -- 172.21.15.121:0/4084445826 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d48034040 con 0x7f1d5c156c20 2024-04-20T10:23:00.801 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.800+0000 7f1d58c00640 1 -- 172.21.15.121:0/4084445826 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d4803ab90 con 0x7f1d5c156c20 2024-04-20T10:23:00.801 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.800+0000 7f1d58c00640 1 -- 172.21.15.121:0/4084445826 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d48004030 con 0x7f1d5c156c20 2024-04-20T10:23:00.801 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.800+0000 7f1d63800640 1 -- 172.21.15.121:0/4084445826 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d5c06e500 con 0x7f1d5c156c20 2024-04-20T10:23:00.801 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.800+0000 7f1d63800640 1 -- 172.21.15.121:0/4084445826 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d5c06eac0 con 0x7f1d5c156c20 2024-04-20T10:23:00.801 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.801+0000 7f1d63800640 1 -- 172.21.15.121:0/4084445826 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d5c068980 con 0x7f1d5c156c20 2024-04-20T10:23:00.801 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.802+0000 7f1d58c00640 1 -- 172.21.15.121:0/4084445826 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1d4804b060 con 0x7f1d5c156c20 2024-04-20T10:23:00.803 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.803+0000 7f1d58c00640 1 --2- 172.21.15.121:0/4084445826 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f1d3c06c8f0 0x7f1d3c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:00.803 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.803+0000 7f1d58c00640 1 -- 172.21.15.121:0/4084445826 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f1d480adb30 con 0x7f1d5c156c20 2024-04-20T10:23:00.803 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.803+0000 7f1d60e00640 1 --2- 172.21.15.121:0/4084445826 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f1d3c06c8f0 0x7f1d3c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:00.803 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.804+0000 7f1d60e00640 1 --2- 172.21.15.121:0/4084445826 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f1d3c06c8f0 0x7f1d3c06edb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009ad0 tx=0x7f1d50008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:00.807 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:00.808+0000 7f1d58c00640 1 -- 172.21.15.121:0/4084445826 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1d4803f070 con 0x7f1d5c156c20 2024-04-20T10:23:01.008 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.009+0000 7f1d63800640 1 -- 172.21.15.121:0/4084445826 --> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1d5c0cfd90 con 0x7f1d3c06c8f0 2024-04-20T10:23:01.016 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.017+0000 7f1d58c00640 1 -- 172.21.15.121:0/4084445826 <== mgr.4103 v2:172.21.15.121:6832/1408233272 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251962 (secure 0 0 0) 0x7f1d5c0cfd90 con 0x7f1d3c06c8f0 2024-04-20T10:23:01.021 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:23:01.023 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2024-04-20T10:23:01.026 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.022+0000 7f1d63800640 1 -- 172.21.15.121:0/4084445826 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f1d3c06c8f0 msgr2=0x7f1d3c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:01.026 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.022+0000 7f1d63800640 1 --2- 172.21.15.121:0/4084445826 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f1d3c06c8f0 0x7f1d3c06edb0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009ad0 tx=0x7f1d50008040 comp rx=0 tx=0).stop 2024-04-20T10:23:01.026 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.022+0000 7f1d63800640 1 -- 172.21.15.121:0/4084445826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d5c156c20 msgr2=0x7f1d5c0b0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:01.026 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.022+0000 7f1d63800640 1 --2- 172.21.15.121:0/4084445826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d5c156c20 0x7f1d5c0b0e00 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f1d480090b0 tx=0x7f1d480046c0 comp rx=0 tx=0).stop 2024-04-20T10:23:01.026 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.022+0000 7f1d63800640 1 -- 172.21.15.121:0/4084445826 shutdown_connections 2024-04-20T10:23:01.027 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.022+0000 7f1d63800640 1 --2- 172.21.15.121:0/4084445826 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f1d5c15ca40 0x7f1d5c0b1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:01.027 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.022+0000 7f1d63800640 1 --2- 172.21.15.121:0/4084445826 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f1d5c156c20 0x7f1d5c0b0e00 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:01.027 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.022+0000 7f1d63800640 1 --2- 172.21.15.121:0/4084445826 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f1d3c06c8f0 0x7f1d3c06edb0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:01.027 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.022+0000 7f1d63800640 1 --2- 172.21.15.121:0/4084445826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d5c0ad8e0 0x7f1d5c06dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:01.027 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.022+0000 7f1d63800640 1 -- 172.21.15.121:0/4084445826 >> 172.21.15.121:0/4084445826 conn(0x7f1d5c0b3350 msgr2=0x7f1d5c15e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:01.027 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.022+0000 7f1d63800640 1 -- 172.21.15.121:0/4084445826 shutdown_connections 2024-04-20T10:23:01.028 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.022+0000 7f1d63800640 1 -- 172.21.15.121:0/4084445826 wait complete. 2024-04-20T10:23:01.041 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2024-04-20T10:22:59.148974+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":168,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220872,"kb_used_data":4712,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753848,"statfs":{"total":773094113280,"available":772867940352,"internally_reserved":0,"allocated":4825088,"data_stored":2732332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002815"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.252098+0000","last_change":"2024-04-20T10:22:51.209650+0000","last_active":"2024-04-20T10:22:52.252098+0000","last_peered":"2024-04-20T10:22:52.252098+0000","last_clean":"2024-04-20T10:22:52.252098+0000","last_became_active":"2024-04-20T10:22:51.209348+0000","last_became_peered":"2024-04-20T10:22:51.209348+0000","last_unstale":"2024-04-20T10:22:52.252098+0000","last_undegraded":"2024-04-20T10:22:52.252098+0000","last_fullsized":"2024-04-20T10:22:52.252098+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:48:19.378379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.251707+0000","last_change":"2024-04-20T10:22:51.202709+0000","last_active":"2024-04-20T10:22:52.251707+0000","last_peered":"2024-04-20T10:22:52.251707+0000","last_clean":"2024-04-20T10:22:52.251707+0000","last_became_active":"2024-04-20T10:22:51.202407+0000","last_became_peered":"2024-04-20T10:22:51.202407+0000","last_unstale":"2024-04-20T10:22:52.251707+0000","last_undegraded":"2024-04-20T10:22:52.251707+0000","last_fullsized":"2024-04-20T10:22:52.251707+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:09:44.641825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.755737+0000","last_change":"2024-04-20T10:22:51.226968+0000","last_active":"2024-04-20T10:22:52.755737+0000","last_peered":"2024-04-20T10:22:52.755737+0000","last_clean":"2024-04-20T10:22:52.755737+0000","last_became_active":"2024-04-20T10:22:51.226740+0000","last_became_peered":"2024-04-20T10:22:51.226740+0000","last_unstale":"2024-04-20T10:22:52.755737+0000","last_undegraded":"2024-04-20T10:22:52.755737+0000","last_fullsized":"2024-04-20T10:22:52.755737+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:14:15.349647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.208921+0000","last_change":"2024-04-20T10:22:51.196019+0000","last_active":"2024-04-20T10:22:52.208921+0000","last_peered":"2024-04-20T10:22:52.208921+0000","last_clean":"2024-04-20T10:22:52.208921+0000","last_became_active":"2024-04-20T10:22:51.195356+0000","last_became_peered":"2024-04-20T10:22:51.195356+0000","last_unstale":"2024-04-20T10:22:52.208921+0000","last_undegraded":"2024-04-20T10:22:52.208921+0000","last_fullsized":"2024-04-20T10:22:52.208921+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:47:01.151389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.208584+0000","last_change":"2024-04-20T10:22:51.192411+0000","last_active":"2024-04-20T10:22:52.208584+0000","last_peered":"2024-04-20T10:22:52.208584+0000","last_clean":"2024-04-20T10:22:52.208584+0000","last_became_active":"2024-04-20T10:22:51.192102+0000","last_became_peered":"2024-04-20T10:22:51.192102+0000","last_unstale":"2024-04-20T10:22:52.208584+0000","last_undegraded":"2024-04-20T10:22:52.208584+0000","last_fullsized":"2024-04-20T10:22:52.208584+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:20:43.234074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.466749+0000","last_change":"2024-04-20T10:22:51.205790+0000","last_active":"2024-04-20T10:22:52.466749+0000","last_peered":"2024-04-20T10:22:52.466749+0000","last_clean":"2024-04-20T10:22:52.466749+0000","last_became_active":"2024-04-20T10:22:51.205450+0000","last_became_peered":"2024-04-20T10:22:51.205450+0000","last_unstale":"2024-04-20T10:22:52.466749+0000","last_undegraded":"2024-04-20T10:22:52.466749+0000","last_fullsized":"2024-04-20T10:22:52.466749+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:03:26.973820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.758896+0000","last_change":"2024-04-20T10:22:51.219564+0000","last_active":"2024-04-20T10:22:52.758896+0000","last_peered":"2024-04-20T10:22:52.758896+0000","last_clean":"2024-04-20T10:22:52.758896+0000","last_became_active":"2024-04-20T10:22:51.219286+0000","last_became_peered":"2024-04-20T10:22:51.219286+0000","last_unstale":"2024-04-20T10:22:52.758896+0000","last_undegraded":"2024-04-20T10:22:52.758896+0000","last_fullsized":"2024-04-20T10:22:52.758896+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:47:56.543759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.251711+0000","last_change":"2024-04-20T10:22:51.228808+0000","last_active":"2024-04-20T10:22:52.251711+0000","last_peered":"2024-04-20T10:22:52.251711+0000","last_clean":"2024-04-20T10:22:52.251711+0000","last_became_active":"2024-04-20T10:22:51.228540+0000","last_became_peered":"2024-04-20T10:22:51.228540+0000","last_unstale":"2024-04-20T10:22:52.251711+0000","last_undegraded":"2024-04-20T10:22:52.251711+0000","last_fullsized":"2024-04-20T10:22:52.251711+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:36:13.852350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.756183+0000","last_change":"2024-04-20T10:22:51.224598+0000","last_active":"2024-04-20T10:22:52.756183+0000","last_peered":"2024-04-20T10:22:52.756183+0000","last_clean":"2024-04-20T10:22:52.756183+0000","last_became_active":"2024-04-20T10:22:51.224346+0000","last_became_peered":"2024-04-20T10:22:51.224346+0000","last_unstale":"2024-04-20T10:22:52.756183+0000","last_undegraded":"2024-04-20T10:22:52.756183+0000","last_fullsized":"2024-04-20T10:22:52.756183+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:21:24.793211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.759235+0000","last_change":"2024-04-20T10:22:51.222474+0000","last_active":"2024-04-20T10:22:52.759235+0000","last_peered":"2024-04-20T10:22:52.759235+0000","last_clean":"2024-04-20T10:22:52.759235+0000","last_became_active":"2024-04-20T10:22:51.222082+0000","last_became_peered":"2024-04-20T10:22:51.222082+0000","last_unstale":"2024-04-20T10:22:52.759235+0000","last_undegraded":"2024-04-20T10:22:52.759235+0000","last_fullsized":"2024-04-20T10:22:52.759235+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:10:55.371506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.760332+0000","last_change":"2024-04-20T10:22:51.227674+0000","last_active":"2024-04-20T10:22:52.760332+0000","last_peered":"2024-04-20T10:22:52.760332+0000","last_clean":"2024-04-20T10:22:52.760332+0000","last_became_active":"2024-04-20T10:22:51.227004+0000","last_became_peered":"2024-04-20T10:22:51.227004+0000","last_unstale":"2024-04-20T10:22:52.760332+0000","last_undegraded":"2024-04-20T10:22:52.760332+0000","last_fullsized":"2024-04-20T10:22:52.760332+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:45:01.466593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.755741+0000","last_change":"2024-04-20T10:22:51.197530+0000","last_active":"2024-04-20T10:22:52.755741+0000","last_peered":"2024-04-20T10:22:52.755741+0000","last_clean":"2024-04-20T10:22:52.755741+0000","last_became_active":"2024-04-20T10:22:51.197236+0000","last_became_peered":"2024-04-20T10:22:51.197236+0000","last_unstale":"2024-04-20T10:22:52.755741+0000","last_undegraded":"2024-04-20T10:22:52.755741+0000","last_fullsized":"2024-04-20T10:22:52.755741+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:34:34.224444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.759475+0000","last_change":"2024-04-20T10:22:51.225493+0000","last_active":"2024-04-20T10:22:52.759475+0000","last_peered":"2024-04-20T10:22:52.759475+0000","last_clean":"2024-04-20T10:22:52.759475+0000","last_became_active":"2024-04-20T10:22:51.224842+0000","last_became_peered":"2024-04-20T10:22:51.224842+0000","last_unstale":"2024-04-20T10:22:52.759475+0000","last_undegraded":"2024-04-20T10:22:52.759475+0000","last_fullsized":"2024-04-20T10:22:52.759475+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:26:31.997560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.756404+0000","last_change":"2024-04-20T10:22:51.211612+0000","last_active":"2024-04-20T10:22:52.756404+0000","last_peered":"2024-04-20T10:22:52.756404+0000","last_clean":"2024-04-20T10:22:52.756404+0000","last_became_active":"2024-04-20T10:22:51.211335+0000","last_became_peered":"2024-04-20T10:22:51.211335+0000","last_unstale":"2024-04-20T10:22:52.756404+0000","last_undegraded":"2024-04-20T10:22:52.756404+0000","last_fullsized":"2024-04-20T10:22:52.756404+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:12:23.788253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.758575+0000","last_change":"2024-04-20T10:22:51.223950+0000","last_active":"2024-04-20T10:22:52.758575+0000","last_peered":"2024-04-20T10:22:52.758575+0000","last_clean":"2024-04-20T10:22:52.758575+0000","last_became_active":"2024-04-20T10:22:51.221665+0000","last_became_peered":"2024-04-20T10:22:51.221665+0000","last_unstale":"2024-04-20T10:22:52.758575+0000","last_undegraded":"2024-04-20T10:22:52.758575+0000","last_fullsized":"2024-04-20T10:22:52.758575+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:48:00.371087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.756077+0000","last_change":"2024-04-20T10:22:51.230833+0000","last_active":"2024-04-20T10:22:52.756077+0000","last_peered":"2024-04-20T10:22:52.756077+0000","last_clean":"2024-04-20T10:22:52.756077+0000","last_became_active":"2024-04-20T10:22:51.230557+0000","last_became_peered":"2024-04-20T10:22:51.230557+0000","last_unstale":"2024-04-20T10:22:52.756077+0000","last_undegraded":"2024-04-20T10:22:52.756077+0000","last_fullsized":"2024-04-20T10:22:52.756077+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:47:34.076559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.757710+0000","last_change":"2024-04-20T10:22:51.230522+0000","last_active":"2024-04-20T10:22:52.757710+0000","last_peered":"2024-04-20T10:22:52.757710+0000","last_clean":"2024-04-20T10:22:52.757710+0000","last_became_active":"2024-04-20T10:22:51.230212+0000","last_became_peered":"2024-04-20T10:22:51.230212+0000","last_unstale":"2024-04-20T10:22:52.757710+0000","last_undegraded":"2024-04-20T10:22:52.757710+0000","last_fullsized":"2024-04-20T10:22:52.757710+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:48:56.677851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.212475+0000","last_change":"2024-04-20T10:22:51.218344+0000","last_active":"2024-04-20T10:22:52.212475+0000","last_peered":"2024-04-20T10:22:52.212475+0000","last_clean":"2024-04-20T10:22:52.212475+0000","last_became_active":"2024-04-20T10:22:51.217912+0000","last_became_peered":"2024-04-20T10:22:51.217912+0000","last_unstale":"2024-04-20T10:22:52.212475+0000","last_undegraded":"2024-04-20T10:22:52.212475+0000","last_fullsized":"2024-04-20T10:22:52.212475+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:04:29.732467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.208927+0000","last_change":"2024-04-20T10:22:51.200273+0000","last_active":"2024-04-20T10:22:52.208927+0000","last_peered":"2024-04-20T10:22:52.208927+0000","last_clean":"2024-04-20T10:22:52.208927+0000","last_became_active":"2024-04-20T10:22:51.199786+0000","last_became_peered":"2024-04-20T10:22:51.199786+0000","last_unstale":"2024-04-20T10:22:52.208927+0000","last_undegraded":"2024-04-20T10:22:52.208927+0000","last_fullsized":"2024-04-20T10:22:52.208927+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:36:26.004366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.211389+0000","last_change":"2024-04-20T10:22:51.218932+0000","last_active":"2024-04-20T10:22:52.211389+0000","last_peered":"2024-04-20T10:22:52.211389+0000","last_clean":"2024-04-20T10:22:52.211389+0000","last_became_active":"2024-04-20T10:22:51.218623+0000","last_became_peered":"2024-04-20T10:22:51.218623+0000","last_unstale":"2024-04-20T10:22:52.211389+0000","last_undegraded":"2024-04-20T10:22:52.211389+0000","last_fullsized":"2024-04-20T10:22:52.211389+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:16:28.306795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.759730+0000","last_change":"2024-04-20T10:22:51.225710+0000","last_active":"2024-04-20T10:22:52.759730+0000","last_peered":"2024-04-20T10:22:52.759730+0000","last_clean":"2024-04-20T10:22:52.759730+0000","last_became_active":"2024-04-20T10:22:51.225224+0000","last_became_peered":"2024-04-20T10:22:51.225224+0000","last_unstale":"2024-04-20T10:22:52.759730+0000","last_undegraded":"2024-04-20T10:22:52.759730+0000","last_fullsized":"2024-04-20T10:22:52.759730+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:45:39.681890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.757398+0000","last_change":"2024-04-20T10:22:51.198194+0000","last_active":"2024-04-20T10:22:52.757398+0000","last_peered":"2024-04-20T10:22:52.757398+0000","last_clean":"2024-04-20T10:22:52.757398+0000","last_became_active":"2024-04-20T10:22:51.197760+0000","last_became_peered":"2024-04-20T10:22:51.197760+0000","last_unstale":"2024-04-20T10:22:52.757398+0000","last_undegraded":"2024-04-20T10:22:52.757398+0000","last_fullsized":"2024-04-20T10:22:52.757398+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:04:29.732467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.759380+0000","last_change":"2024-04-20T10:22:51.213701+0000","last_active":"2024-04-20T10:22:52.759380+0000","last_peered":"2024-04-20T10:22:52.759380+0000","last_clean":"2024-04-20T10:22:52.759380+0000","last_became_active":"2024-04-20T10:22:51.213384+0000","last_became_peered":"2024-04-20T10:22:51.213384+0000","last_unstale":"2024-04-20T10:22:52.759380+0000","last_undegraded":"2024-04-20T10:22:52.759380+0000","last_fullsized":"2024-04-20T10:22:52.759380+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:07:09.046246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.209189+0000","last_change":"2024-04-20T10:22:51.208012+0000","last_active":"2024-04-20T10:22:52.209189+0000","last_peered":"2024-04-20T10:22:52.209189+0000","last_clean":"2024-04-20T10:22:52.209189+0000","last_became_active":"2024-04-20T10:22:51.207734+0000","last_became_peered":"2024-04-20T10:22:51.207734+0000","last_unstale":"2024-04-20T10:22:52.209189+0000","last_undegraded":"2024-04-20T10:22:52.209189+0000","last_fullsized":"2024-04-20T10:22:52.209189+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:39:49.504337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.212675+0000","last_change":"2024-04-20T10:22:51.209736+0000","last_active":"2024-04-20T10:22:52.212675+0000","last_peered":"2024-04-20T10:22:52.212675+0000","last_clean":"2024-04-20T10:22:52.212675+0000","last_became_active":"2024-04-20T10:22:51.209421+0000","last_became_peered":"2024-04-20T10:22:51.209421+0000","last_unstale":"2024-04-20T10:22:52.212675+0000","last_undegraded":"2024-04-20T10:22:52.212675+0000","last_fullsized":"2024-04-20T10:22:52.212675+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:29:08.806291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.467679+0000","last_change":"2024-04-20T10:22:51.220786+0000","last_active":"2024-04-20T10:22:52.467679+0000","last_peered":"2024-04-20T10:22:52.467679+0000","last_clean":"2024-04-20T10:22:52.467679+0000","last_became_active":"2024-04-20T10:22:51.220227+0000","last_became_peered":"2024-04-20T10:22:51.220227+0000","last_unstale":"2024-04-20T10:22:52.467679+0000","last_undegraded":"2024-04-20T10:22:52.467679+0000","last_fullsized":"2024-04-20T10:22:52.467679+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:56:23.916864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.211907+0000","last_change":"2024-04-20T10:22:51.214621+0000","last_active":"2024-04-20T10:22:52.211907+0000","last_peered":"2024-04-20T10:22:52.211907+0000","last_clean":"2024-04-20T10:22:52.211907+0000","last_became_active":"2024-04-20T10:22:51.214266+0000","last_became_peered":"2024-04-20T10:22:51.214266+0000","last_unstale":"2024-04-20T10:22:52.211907+0000","last_undegraded":"2024-04-20T10:22:52.211907+0000","last_fullsized":"2024-04-20T10:22:52.211907+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:12:23.788253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.760142+0000","last_change":"2024-04-20T10:22:51.222490+0000","last_active":"2024-04-20T10:22:52.760142+0000","last_peered":"2024-04-20T10:22:52.760142+0000","last_clean":"2024-04-20T10:22:52.760142+0000","last_became_active":"2024-04-20T10:22:51.220049+0000","last_became_peered":"2024-04-20T10:22:51.220049+0000","last_unstale":"2024-04-20T10:22:52.760142+0000","last_undegraded":"2024-04-20T10:22:52.760142+0000","last_fullsized":"2024-04-20T10:22:52.760142+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:11:59.306390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.212170+0000","last_change":"2024-04-20T10:22:51.206155+0000","last_active":"2024-04-20T10:22:52.212170+0000","last_peered":"2024-04-20T10:22:52.212170+0000","last_clean":"2024-04-20T10:22:52.212170+0000","last_became_active":"2024-04-20T10:22:51.205785+0000","last_became_peered":"2024-04-20T10:22:51.205785+0000","last_unstale":"2024-04-20T10:22:52.212170+0000","last_undegraded":"2024-04-20T10:22:52.212170+0000","last_fullsized":"2024-04-20T10:22:52.212170+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:23:42.010662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.211436+0000","last_change":"2024-04-20T10:22:51.208148+0000","last_active":"2024-04-20T10:22:52.211436+0000","last_peered":"2024-04-20T10:22:52.211436+0000","last_clean":"2024-04-20T10:22:52.211436+0000","last_became_active":"2024-04-20T10:22:51.207856+0000","last_became_peered":"2024-04-20T10:22:51.207856+0000","last_unstale":"2024-04-20T10:22:52.211436+0000","last_undegraded":"2024-04-20T10:22:52.211436+0000","last_fullsized":"2024-04-20T10:22:52.211436+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:54:46.131724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.759522+0000","last_change":"2024-04-20T10:22:51.214897+0000","last_active":"2024-04-20T10:22:52.759522+0000","last_peered":"2024-04-20T10:22:52.759522+0000","last_clean":"2024-04-20T10:22:52.759522+0000","last_became_active":"2024-04-20T10:22:51.214601+0000","last_became_peered":"2024-04-20T10:22:51.214601+0000","last_unstale":"2024-04-20T10:22:52.759522+0000","last_undegraded":"2024-04-20T10:22:52.759522+0000","last_fullsized":"2024-04-20T10:22:52.759522+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:07:37.714092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.467029+0000","last_change":"2024-04-20T10:22:51.195262+0000","last_active":"2024-04-20T10:22:52.467029+0000","last_peered":"2024-04-20T10:22:52.467029+0000","last_clean":"2024-04-20T10:22:52.467029+0000","last_became_active":"2024-04-20T10:22:51.194993+0000","last_became_peered":"2024-04-20T10:22:51.194993+0000","last_unstale":"2024-04-20T10:22:52.467029+0000","last_undegraded":"2024-04-20T10:22:52.467029+0000","last_fullsized":"2024-04-20T10:22:52.467029+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:14:15.349647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.757434+0000","last_change":"2024-04-20T10:22:51.230960+0000","last_active":"2024-04-20T10:22:52.757434+0000","last_peered":"2024-04-20T10:22:52.757434+0000","last_clean":"2024-04-20T10:22:52.757434+0000","last_became_active":"2024-04-20T10:22:51.228563+0000","last_became_peered":"2024-04-20T10:22:51.228563+0000","last_unstale":"2024-04-20T10:22:52.757434+0000","last_undegraded":"2024-04-20T10:22:52.757434+0000","last_fullsized":"2024-04-20T10:22:52.757434+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:26:44.239012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.212692+0000","last_change":"2024-04-20T10:22:50.208086+0000","last_active":"2024-04-20T10:22:52.212692+0000","last_peered":"2024-04-20T10:22:52.212692+0000","last_clean":"2024-04-20T10:22:52.212692+0000","last_became_active":"2024-04-20T10:22:50.207645+0000","last_became_peered":"2024-04-20T10:22:50.207645+0000","last_unstale":"2024-04-20T10:22:52.212692+0000","last_undegraded":"2024-04-20T10:22:52.212692+0000","last_fullsized":"2024-04-20T10:22:52.212692+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:22:51.962842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.467604+0000","last_change":"2024-04-20T10:22:51.229034+0000","last_active":"2024-04-20T10:22:52.467604+0000","last_peered":"2024-04-20T10:22:52.467604+0000","last_clean":"2024-04-20T10:22:52.467604+0000","last_became_active":"2024-04-20T10:22:51.228786+0000","last_became_peered":"2024-04-20T10:22:51.228786+0000","last_unstale":"2024-04-20T10:22:52.467604+0000","last_undegraded":"2024-04-20T10:22:52.467604+0000","last_fullsized":"2024-04-20T10:22:52.467604+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:21:24.793211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.755785+0000","last_change":"2024-04-20T10:22:50.204532+0000","last_active":"2024-04-20T10:22:52.755785+0000","last_peered":"2024-04-20T10:22:52.755785+0000","last_clean":"2024-04-20T10:22:52.755785+0000","last_became_active":"2024-04-20T10:22:50.204230+0000","last_became_peered":"2024-04-20T10:22:50.204230+0000","last_unstale":"2024-04-20T10:22:52.755785+0000","last_undegraded":"2024-04-20T10:22:52.755785+0000","last_fullsized":"2024-04-20T10:22:52.755785+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:14:18.324437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.760631+0000","last_change":"2024-04-20T10:22:51.205577+0000","last_active":"2024-04-20T10:22:52.760631+0000","last_peered":"2024-04-20T10:22:52.760631+0000","last_clean":"2024-04-20T10:22:52.760631+0000","last_became_active":"2024-04-20T10:22:51.205288+0000","last_became_peered":"2024-04-20T10:22:51.205288+0000","last_unstale":"2024-04-20T10:22:52.760631+0000","last_undegraded":"2024-04-20T10:22:52.760631+0000","last_fullsized":"2024-04-20T10:22:52.760631+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:03:26.973820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.467695+0000","last_change":"2024-04-20T10:22:50.205333+0000","last_active":"2024-04-20T10:22:52.467695+0000","last_peered":"2024-04-20T10:22:52.467695+0000","last_clean":"2024-04-20T10:22:52.467695+0000","last_became_active":"2024-04-20T10:22:50.205029+0000","last_became_peered":"2024-04-20T10:22:50.205029+0000","last_unstale":"2024-04-20T10:22:52.467695+0000","last_undegraded":"2024-04-20T10:22:52.467695+0000","last_fullsized":"2024-04-20T10:22:52.467695+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:46:38.683786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.252123+0000","last_change":"2024-04-20T10:22:51.214233+0000","last_active":"2024-04-20T10:22:52.252123+0000","last_peered":"2024-04-20T10:22:52.252123+0000","last_clean":"2024-04-20T10:22:52.252123+0000","last_became_active":"2024-04-20T10:22:51.213949+0000","last_became_peered":"2024-04-20T10:22:51.213949+0000","last_unstale":"2024-04-20T10:22:52.252123+0000","last_undegraded":"2024-04-20T10:22:52.252123+0000","last_fullsized":"2024-04-20T10:22:52.252123+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:49:13.969709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.759258+0000","last_change":"2024-04-20T10:22:50.207305+0000","last_active":"2024-04-20T10:22:52.759258+0000","last_peered":"2024-04-20T10:22:52.759258+0000","last_clean":"2024-04-20T10:22:52.759258+0000","last_became_active":"2024-04-20T10:22:50.207018+0000","last_became_peered":"2024-04-20T10:22:50.207018+0000","last_unstale":"2024-04-20T10:22:52.759258+0000","last_undegraded":"2024-04-20T10:22:52.759258+0000","last_fullsized":"2024-04-20T10:22:52.759258+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:39:48.488461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.755736+0000","last_change":"2024-04-20T10:22:51.206946+0000","last_active":"2024-04-20T10:22:52.755736+0000","last_peered":"2024-04-20T10:22:52.755736+0000","last_clean":"2024-04-20T10:22:52.755736+0000","last_became_active":"2024-04-20T10:22:51.206670+0000","last_became_peered":"2024-04-20T10:22:51.206670+0000","last_unstale":"2024-04-20T10:22:52.755736+0000","last_undegraded":"2024-04-20T10:22:52.755736+0000","last_fullsized":"2024-04-20T10:22:52.755736+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:03:26.973820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.759643+0000","last_change":"2024-04-20T10:22:50.202177+0000","last_active":"2024-04-20T10:22:52.759643+0000","last_peered":"2024-04-20T10:22:52.759643+0000","last_clean":"2024-04-20T10:22:52.759643+0000","last_became_active":"2024-04-20T10:22:50.201840+0000","last_became_peered":"2024-04-20T10:22:50.201840+0000","last_unstale":"2024-04-20T10:22:52.759643+0000","last_undegraded":"2024-04-20T10:22:52.759643+0000","last_fullsized":"2024-04-20T10:22:52.759643+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:40:01.292259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.758943+0000","last_change":"2024-04-20T10:22:51.227641+0000","last_active":"2024-04-20T10:22:52.758943+0000","last_peered":"2024-04-20T10:22:52.758943+0000","last_clean":"2024-04-20T10:22:52.758943+0000","last_became_active":"2024-04-20T10:22:51.227350+0000","last_became_peered":"2024-04-20T10:22:51.227350+0000","last_unstale":"2024-04-20T10:22:52.758943+0000","last_undegraded":"2024-04-20T10:22:52.758943+0000","last_fullsized":"2024-04-20T10:22:52.758943+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:48:31.200346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.757072+0000","last_change":"2024-04-20T10:22:50.211068+0000","last_active":"2024-04-20T10:22:52.757072+0000","last_peered":"2024-04-20T10:22:52.757072+0000","last_clean":"2024-04-20T10:22:52.757072+0000","last_became_active":"2024-04-20T10:22:50.210709+0000","last_became_peered":"2024-04-20T10:22:50.210709+0000","last_unstale":"2024-04-20T10:22:52.757072+0000","last_undegraded":"2024-04-20T10:22:52.757072+0000","last_fullsized":"2024-04-20T10:22:52.757072+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:46:38.683786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.757760+0000","last_change":"2024-04-20T10:22:51.231926+0000","last_active":"2024-04-20T10:22:52.757760+0000","last_peered":"2024-04-20T10:22:52.757760+0000","last_clean":"2024-04-20T10:22:52.757760+0000","last_became_active":"2024-04-20T10:22:51.228859+0000","last_became_peered":"2024-04-20T10:22:51.228859+0000","last_unstale":"2024-04-20T10:22:52.757760+0000","last_undegraded":"2024-04-20T10:22:52.757760+0000","last_fullsized":"2024-04-20T10:22:52.757760+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:29:08.806291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.466834+0000","last_change":"2024-04-20T10:22:50.202082+0000","last_active":"2024-04-20T10:22:52.466834+0000","last_peered":"2024-04-20T10:22:52.466834+0000","last_clean":"2024-04-20T10:22:52.466834+0000","last_became_active":"2024-04-20T10:22:50.201785+0000","last_became_peered":"2024-04-20T10:22:50.201785+0000","last_unstale":"2024-04-20T10:22:52.466834+0000","last_undegraded":"2024-04-20T10:22:52.466834+0000","last_fullsized":"2024-04-20T10:22:52.466834+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:30:35.298814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.251878+0000","last_change":"2024-04-20T10:22:51.211545+0000","last_active":"2024-04-20T10:22:52.251878+0000","last_peered":"2024-04-20T10:22:52.251878+0000","last_clean":"2024-04-20T10:22:52.251878+0000","last_became_active":"2024-04-20T10:22:51.211267+0000","last_became_peered":"2024-04-20T10:22:51.211267+0000","last_unstale":"2024-04-20T10:22:52.251878+0000","last_undegraded":"2024-04-20T10:22:52.251878+0000","last_fullsized":"2024-04-20T10:22:52.251878+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:49:49.694929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.755868+0000","last_change":"2024-04-20T10:22:50.208162+0000","last_active":"2024-04-20T10:22:52.755868+0000","last_peered":"2024-04-20T10:22:52.755868+0000","last_clean":"2024-04-20T10:22:52.755868+0000","last_became_active":"2024-04-20T10:22:50.206236+0000","last_became_peered":"2024-04-20T10:22:50.206236+0000","last_unstale":"2024-04-20T10:22:52.755868+0000","last_undegraded":"2024-04-20T10:22:52.755868+0000","last_fullsized":"2024-04-20T10:22:52.755868+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:22:51.962842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.209067+0000","last_change":"2024-04-20T10:22:51.196295+0000","last_active":"2024-04-20T10:22:52.209067+0000","last_peered":"2024-04-20T10:22:52.209067+0000","last_clean":"2024-04-20T10:22:52.209067+0000","last_became_active":"2024-04-20T10:22:51.195725+0000","last_became_peered":"2024-04-20T10:22:51.195725+0000","last_unstale":"2024-04-20T10:22:52.209067+0000","last_undegraded":"2024-04-20T10:22:52.209067+0000","last_fullsized":"2024-04-20T10:22:52.209067+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:10:16.559701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.252146+0000","last_change":"2024-04-20T10:22:50.196202+0000","last_active":"2024-04-20T10:22:52.252146+0000","last_peered":"2024-04-20T10:22:52.252146+0000","last_clean":"2024-04-20T10:22:52.252146+0000","last_became_active":"2024-04-20T10:22:50.195933+0000","last_became_peered":"2024-04-20T10:22:50.195933+0000","last_unstale":"2024-04-20T10:22:52.252146+0000","last_undegraded":"2024-04-20T10:22:52.252146+0000","last_fullsized":"2024-04-20T10:22:52.252146+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:04:28.716591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.757670+0000","last_change":"2024-04-20T10:22:51.220771+0000","last_active":"2024-04-20T10:22:52.757670+0000","last_peered":"2024-04-20T10:22:52.757670+0000","last_clean":"2024-04-20T10:22:52.757670+0000","last_became_active":"2024-04-20T10:22:51.220461+0000","last_became_peered":"2024-04-20T10:22:51.220461+0000","last_unstale":"2024-04-20T10:22:52.757670+0000","last_undegraded":"2024-04-20T10:22:52.757670+0000","last_fullsized":"2024-04-20T10:22:52.757670+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:39:37.819763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.759845+0000","last_change":"2024-04-20T10:22:50.209570+0000","last_active":"2024-04-20T10:22:52.759845+0000","last_peered":"2024-04-20T10:22:52.759845+0000","last_clean":"2024-04-20T10:22:52.759845+0000","last_became_active":"2024-04-20T10:22:50.209281+0000","last_became_peered":"2024-04-20T10:22:50.209281+0000","last_unstale":"2024-04-20T10:22:52.759845+0000","last_undegraded":"2024-04-20T10:22:52.759845+0000","last_fullsized":"2024-04-20T10:22:52.759845+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:38:10.698270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.756255+0000","last_change":"2024-04-20T10:22:51.207905+0000","last_active":"2024-04-20T10:22:52.756255+0000","last_peered":"2024-04-20T10:22:52.756255+0000","last_clean":"2024-04-20T10:22:52.756255+0000","last_became_active":"2024-04-20T10:22:51.207640+0000","last_became_peered":"2024-04-20T10:22:51.207640+0000","last_unstale":"2024-04-20T10:22:52.756255+0000","last_undegraded":"2024-04-20T10:22:52.756255+0000","last_fullsized":"2024-04-20T10:22:52.756255+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:56:23.916864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.760829+0000","last_change":"2024-04-20T10:22:50.206722+0000","last_active":"2024-04-20T10:22:52.760829+0000","last_peered":"2024-04-20T10:22:52.760829+0000","last_clean":"2024-04-20T10:22:52.760829+0000","last_became_active":"2024-04-20T10:22:50.206395+0000","last_became_peered":"2024-04-20T10:22:50.206395+0000","last_unstale":"2024-04-20T10:22:52.760829+0000","last_undegraded":"2024-04-20T10:22:52.760829+0000","last_fullsized":"2024-04-20T10:22:52.760829+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:55:41.278026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.761351+0000","last_change":"2024-04-20T10:22:51.204717+0000","last_active":"2024-04-20T10:22:52.761351+0000","last_peered":"2024-04-20T10:22:52.761351+0000","last_clean":"2024-04-20T10:22:52.761351+0000","last_became_active":"2024-04-20T10:22:51.203357+0000","last_became_peered":"2024-04-20T10:22:51.203357+0000","last_unstale":"2024-04-20T10:22:52.761351+0000","last_undegraded":"2024-04-20T10:22:52.761351+0000","last_fullsized":"2024-04-20T10:22:52.761351+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:34:34.224444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.251921+0000","last_change":"2024-04-20T10:22:50.203958+0000","last_active":"2024-04-20T10:22:52.251921+0000","last_peered":"2024-04-20T10:22:52.251921+0000","last_clean":"2024-04-20T10:22:52.251921+0000","last_became_active":"2024-04-20T10:22:50.203499+0000","last_became_peered":"2024-04-20T10:22:50.203499+0000","last_unstale":"2024-04-20T10:22:52.251921+0000","last_undegraded":"2024-04-20T10:22:52.251921+0000","last_fullsized":"2024-04-20T10:22:52.251921+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:23:40.994786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.755353+0000","last_change":"2024-04-20T10:22:51.224929+0000","last_active":"2024-04-20T10:22:52.755353+0000","last_peered":"2024-04-20T10:22:52.755353+0000","last_clean":"2024-04-20T10:22:52.755353+0000","last_became_active":"2024-04-20T10:22:51.224646+0000","last_became_peered":"2024-04-20T10:22:51.224646+0000","last_unstale":"2024-04-20T10:22:52.755353+0000","last_undegraded":"2024-04-20T10:22:52.755353+0000","last_fullsized":"2024-04-20T10:22:52.755353+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:29:08.806291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.251994+0000","last_change":"2024-04-20T10:22:50.196624+0000","last_active":"2024-04-20T10:22:52.251994+0000","last_peered":"2024-04-20T10:22:52.251994+0000","last_clean":"2024-04-20T10:22:52.251994+0000","last_became_active":"2024-04-20T10:22:50.196295+0000","last_became_peered":"2024-04-20T10:22:50.196295+0000","last_unstale":"2024-04-20T10:22:52.251994+0000","last_undegraded":"2024-04-20T10:22:52.251994+0000","last_fullsized":"2024-04-20T10:22:52.251994+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:12:22.772377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.251863+0000","last_change":"2024-04-20T10:22:51.200209+0000","last_active":"2024-04-20T10:22:52.251863+0000","last_peered":"2024-04-20T10:22:52.251863+0000","last_clean":"2024-04-20T10:22:52.251863+0000","last_became_active":"2024-04-20T10:22:51.199938+0000","last_became_peered":"2024-04-20T10:22:51.199938+0000","last_unstale":"2024-04-20T10:22:52.251863+0000","last_undegraded":"2024-04-20T10:22:52.251863+0000","last_fullsized":"2024-04-20T10:22:52.251863+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:07:37.714092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.466874+0000","last_change":"2024-04-20T10:22:50.200183+0000","last_active":"2024-04-20T10:22:52.466874+0000","last_peered":"2024-04-20T10:22:52.466874+0000","last_clean":"2024-04-20T10:22:52.466874+0000","last_became_active":"2024-04-20T10:22:50.199884+0000","last_became_peered":"2024-04-20T10:22:50.199884+0000","last_unstale":"2024-04-20T10:22:52.466874+0000","last_undegraded":"2024-04-20T10:22:52.466874+0000","last_fullsized":"2024-04-20T10:22:52.466874+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:20:05.462979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.756728+0000","last_change":"2024-04-20T10:22:51.214680+0000","last_active":"2024-04-20T10:22:52.756728+0000","last_peered":"2024-04-20T10:22:52.756728+0000","last_clean":"2024-04-20T10:22:52.756728+0000","last_became_active":"2024-04-20T10:22:51.213945+0000","last_became_peered":"2024-04-20T10:22:51.213945+0000","last_unstale":"2024-04-20T10:22:52.756728+0000","last_undegraded":"2024-04-20T10:22:52.756728+0000","last_fullsized":"2024-04-20T10:22:52.756728+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:56:23.916864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.211063+0000","last_change":"2024-04-20T10:22:50.205765+0000","last_active":"2024-04-20T10:22:52.211063+0000","last_peered":"2024-04-20T10:22:52.211063+0000","last_clean":"2024-04-20T10:22:52.211063+0000","last_became_active":"2024-04-20T10:22:50.205454+0000","last_became_peered":"2024-04-20T10:22:50.205454+0000","last_unstale":"2024-04-20T10:22:52.211063+0000","last_undegraded":"2024-04-20T10:22:52.211063+0000","last_fullsized":"2024-04-20T10:22:52.211063+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:10:15.543826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.251812+0000","last_change":"2024-04-20T10:22:51.210921+0000","last_active":"2024-04-20T10:22:52.251812+0000","last_peered":"2024-04-20T10:22:52.251812+0000","last_clean":"2024-04-20T10:22:52.251812+0000","last_became_active":"2024-04-20T10:22:51.210599+0000","last_became_peered":"2024-04-20T10:22:51.210599+0000","last_unstale":"2024-04-20T10:22:52.251812+0000","last_undegraded":"2024-04-20T10:22:52.251812+0000","last_fullsized":"2024-04-20T10:22:52.251812+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:07:09.046246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":22,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.756594+0000","last_change":"2024-04-20T10:22:50.210848+0000","last_active":"2024-04-20T10:22:52.756594+0000","last_peered":"2024-04-20T10:22:52.756594+0000","last_clean":"2024-04-20T10:22:52.756594+0000","last_became_active":"2024-04-20T10:22:50.210382+0000","last_became_peered":"2024-04-20T10:22:50.210382+0000","last_unstale":"2024-04-20T10:22:52.756594+0000","last_undegraded":"2024-04-20T10:22:52.756594+0000","last_fullsized":"2024-04-20T10:22:52.756594+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:20:05.462979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.251902+0000","last_change":"2024-04-20T10:22:51.208008+0000","last_active":"2024-04-20T10:22:52.251902+0000","last_peered":"2024-04-20T10:22:52.251902+0000","last_clean":"2024-04-20T10:22:52.251902+0000","last_became_active":"2024-04-20T10:22:51.207730+0000","last_became_peered":"2024-04-20T10:22:51.207730+0000","last_unstale":"2024-04-20T10:22:52.251902+0000","last_undegraded":"2024-04-20T10:22:52.251902+0000","last_fullsized":"2024-04-20T10:22:52.251902+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:55:04.112006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.760461+0000","last_change":"2024-04-20T10:22:50.208371+0000","last_active":"2024-04-20T10:22:52.760461+0000","last_peered":"2024-04-20T10:22:52.760461+0000","last_clean":"2024-04-20T10:22:52.760461+0000","last_became_active":"2024-04-20T10:22:50.208102+0000","last_became_peered":"2024-04-20T10:22:50.208102+0000","last_unstale":"2024-04-20T10:22:52.760461+0000","last_undegraded":"2024-04-20T10:22:52.760461+0000","last_fullsized":"2024-04-20T10:22:52.760461+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:52:46.611087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.760630+0000","last_change":"2024-04-20T10:22:51.226634+0000","last_active":"2024-04-20T10:22:52.760630+0000","last_peered":"2024-04-20T10:22:52.760630+0000","last_clean":"2024-04-20T10:22:52.760630+0000","last_became_active":"2024-04-20T10:22:51.223859+0000","last_became_peered":"2024-04-20T10:22:51.223859+0000","last_unstale":"2024-04-20T10:22:52.760630+0000","last_undegraded":"2024-04-20T10:22:52.760630+0000","last_fullsized":"2024-04-20T10:22:52.760630+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:08:49.220198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.760666+0000","last_change":"2024-04-20T10:22:50.205722+0000","last_active":"2024-04-20T10:22:52.760666+0000","last_peered":"2024-04-20T10:22:52.760666+0000","last_clean":"2024-04-20T10:22:52.760666+0000","last_became_active":"2024-04-20T10:22:50.205402+0000","last_became_peered":"2024-04-20T10:22:50.205402+0000","last_unstale":"2024-04-20T10:22:52.760666+0000","last_undegraded":"2024-04-20T10:22:52.760666+0000","last_fullsized":"2024-04-20T10:22:52.760666+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:16:59.003634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.760401+0000","last_change":"2024-04-20T10:22:51.198910+0000","last_active":"2024-04-20T10:22:52.760401+0000","last_peered":"2024-04-20T10:22:52.760401+0000","last_clean":"2024-04-20T10:22:52.760401+0000","last_became_active":"2024-04-20T10:22:51.198452+0000","last_became_peered":"2024-04-20T10:22:51.198452+0000","last_unstale":"2024-04-20T10:22:52.760401+0000","last_undegraded":"2024-04-20T10:22:52.760401+0000","last_fullsized":"2024-04-20T10:22:52.760401+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:32:30.754118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.209015+0000","last_change":"2024-04-20T10:22:50.206721+0000","last_active":"2024-04-20T10:22:52.209015+0000","last_peered":"2024-04-20T10:22:52.209015+0000","last_clean":"2024-04-20T10:22:52.209015+0000","last_became_active":"2024-04-20T10:22:50.206370+0000","last_became_peered":"2024-04-20T10:22:50.206370+0000","last_unstale":"2024-04-20T10:22:52.209015+0000","last_undegraded":"2024-04-20T10:22:52.209015+0000","last_fullsized":"2024-04-20T10:22:52.209015+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:04:57.713148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.209138+0000","last_change":"2024-04-20T10:22:51.196095+0000","last_active":"2024-04-20T10:22:52.209138+0000","last_peered":"2024-04-20T10:22:52.209138+0000","last_clean":"2024-04-20T10:22:52.209138+0000","last_became_active":"2024-04-20T10:22:51.195784+0000","last_became_peered":"2024-04-20T10:22:51.195784+0000","last_unstale":"2024-04-20T10:22:52.209138+0000","last_undegraded":"2024-04-20T10:22:52.209138+0000","last_fullsized":"2024-04-20T10:22:52.209138+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:22:52.978718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.467282+0000","last_change":"2024-04-20T10:22:50.205027+0000","last_active":"2024-04-20T10:22:52.467282+0000","last_peered":"2024-04-20T10:22:52.467282+0000","last_clean":"2024-04-20T10:22:52.467282+0000","last_became_active":"2024-04-20T10:22:50.204749+0000","last_became_peered":"2024-04-20T10:22:50.204749+0000","last_unstale":"2024-04-20T10:22:52.467282+0000","last_undegraded":"2024-04-20T10:22:52.467282+0000","last_fullsized":"2024-04-20T10:22:52.467282+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:14:18.324437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.756642+0000","last_change":"2024-04-20T10:22:51.222374+0000","last_active":"2024-04-20T10:22:52.756642+0000","last_peered":"2024-04-20T10:22:52.756642+0000","last_clean":"2024-04-20T10:22:52.756642+0000","last_became_active":"2024-04-20T10:22:51.221790+0000","last_became_peered":"2024-04-20T10:22:51.221790+0000","last_unstale":"2024-04-20T10:22:52.756642+0000","last_undegraded":"2024-04-20T10:22:52.756642+0000","last_fullsized":"2024-04-20T10:22:52.756642+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:21:24.793211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.208680+0000","last_change":"2024-04-20T10:22:50.194920+0000","last_active":"2024-04-20T10:22:52.208680+0000","last_peered":"2024-04-20T10:22:52.208680+0000","last_clean":"2024-04-20T10:22:52.208680+0000","last_became_active":"2024-04-20T10:22:50.194622+0000","last_became_peered":"2024-04-20T10:22:50.194622+0000","last_unstale":"2024-04-20T10:22:52.208680+0000","last_undegraded":"2024-04-20T10:22:52.208680+0000","last_fullsized":"2024-04-20T10:22:52.208680+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:06:56.248670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.755381+0000","last_change":"2024-04-20T10:22:51.200318+0000","last_active":"2024-04-20T10:22:52.755381+0000","last_peered":"2024-04-20T10:22:52.755381+0000","last_clean":"2024-04-20T10:22:52.755381+0000","last_became_active":"2024-04-20T10:22:51.200056+0000","last_became_peered":"2024-04-20T10:22:51.200056+0000","last_unstale":"2024-04-20T10:22:52.755381+0000","last_undegraded":"2024-04-20T10:22:52.755381+0000","last_fullsized":"2024-04-20T10:22:52.755381+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:32:30.754118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.208847+0000","last_change":"2024-04-20T10:22:50.202646+0000","last_active":"2024-04-20T10:22:52.208847+0000","last_peered":"2024-04-20T10:22:52.208847+0000","last_clean":"2024-04-20T10:22:52.208847+0000","last_became_active":"2024-04-20T10:22:50.202266+0000","last_became_peered":"2024-04-20T10:22:50.202266+0000","last_unstale":"2024-04-20T10:22:52.208847+0000","last_undegraded":"2024-04-20T10:22:52.208847+0000","last_fullsized":"2024-04-20T10:22:52.208847+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:04:33.741440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.758860+0000","last_change":"2024-04-20T10:22:51.217439+0000","last_active":"2024-04-20T10:22:52.758860+0000","last_peered":"2024-04-20T10:22:52.758860+0000","last_clean":"2024-04-20T10:22:52.758860+0000","last_became_active":"2024-04-20T10:22:51.216616+0000","last_became_peered":"2024-04-20T10:22:51.216616+0000","last_unstale":"2024-04-20T10:22:52.758860+0000","last_undegraded":"2024-04-20T10:22:52.758860+0000","last_fullsized":"2024-04-20T10:22:52.758860+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:04:16.514782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.759616+0000","last_change":"2024-04-20T10:22:50.205146+0000","last_active":"2024-04-20T10:22:52.759616+0000","last_peered":"2024-04-20T10:22:52.759616+0000","last_clean":"2024-04-20T10:22:52.759616+0000","last_became_active":"2024-04-20T10:22:50.204772+0000","last_became_peered":"2024-04-20T10:22:50.204772+0000","last_unstale":"2024-04-20T10:22:52.759616+0000","last_undegraded":"2024-04-20T10:22:52.759616+0000","last_fullsized":"2024-04-20T10:22:52.759616+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:14:18.324437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.251959+0000","last_change":"2024-04-20T10:22:51.200462+0000","last_active":"2024-04-20T10:22:52.251959+0000","last_peered":"2024-04-20T10:22:52.251959+0000","last_clean":"2024-04-20T10:22:52.251959+0000","last_became_active":"2024-04-20T10:22:51.200148+0000","last_became_peered":"2024-04-20T10:22:51.200148+0000","last_unstale":"2024-04-20T10:22:52.251959+0000","last_undegraded":"2024-04-20T10:22:52.251959+0000","last_fullsized":"2024-04-20T10:22:52.251959+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:49:52.511861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.209268+0000","last_change":"2024-04-20T10:22:50.194938+0000","last_active":"2024-04-20T10:22:52.209268+0000","last_peered":"2024-04-20T10:22:52.209268+0000","last_clean":"2024-04-20T10:22:52.209268+0000","last_became_active":"2024-04-20T10:22:50.194631+0000","last_became_peered":"2024-04-20T10:22:50.194631+0000","last_unstale":"2024-04-20T10:22:52.209268+0000","last_undegraded":"2024-04-20T10:22:52.209268+0000","last_fullsized":"2024-04-20T10:22:52.209268+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:28:14.564063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.467289+0000","last_change":"2024-04-20T10:22:51.230038+0000","last_active":"2024-04-20T10:22:52.467289+0000","last_peered":"2024-04-20T10:22:52.467289+0000","last_clean":"2024-04-20T10:22:52.467289+0000","last_became_active":"2024-04-20T10:22:51.229752+0000","last_became_peered":"2024-04-20T10:22:51.229752+0000","last_unstale":"2024-04-20T10:22:52.467289+0000","last_undegraded":"2024-04-20T10:22:52.467289+0000","last_fullsized":"2024-04-20T10:22:52.467289+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:47:34.076559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.760724+0000","last_change":"2024-04-20T10:22:50.200577+0000","last_active":"2024-04-20T10:22:52.760724+0000","last_peered":"2024-04-20T10:22:52.760724+0000","last_clean":"2024-04-20T10:22:52.760724+0000","last_became_active":"2024-04-20T10:22:50.199510+0000","last_became_peered":"2024-04-20T10:22:50.199510+0000","last_unstale":"2024-04-20T10:22:52.760724+0000","last_undegraded":"2024-04-20T10:22:52.760724+0000","last_fullsized":"2024-04-20T10:22:52.760724+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:32:22.264498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.758090+0000","last_change":"2024-04-20T10:22:51.224530+0000","last_active":"2024-04-20T10:22:52.758090+0000","last_peered":"2024-04-20T10:22:52.758090+0000","last_clean":"2024-04-20T10:22:52.758090+0000","last_became_active":"2024-04-20T10:22:51.224210+0000","last_became_peered":"2024-04-20T10:22:51.224210+0000","last_unstale":"2024-04-20T10:22:52.758090+0000","last_undegraded":"2024-04-20T10:22:52.758090+0000","last_fullsized":"2024-04-20T10:22:52.758090+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:36:13.852350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"39'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.209010+0000","last_change":"2024-04-20T10:22:50.202138+0000","last_active":"2024-04-20T10:22:52.209010+0000","last_peered":"2024-04-20T10:22:52.209010+0000","last_clean":"2024-04-20T10:22:52.209010+0000","last_became_active":"2024-04-20T10:22:50.201808+0000","last_became_peered":"2024-04-20T10:22:50.201808+0000","last_unstale":"2024-04-20T10:22:52.209010+0000","last_undegraded":"2024-04-20T10:22:52.209010+0000","last_fullsized":"2024-04-20T10:22:52.209010+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:09:39.714678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.251864+0000","last_change":"2024-04-20T10:22:51.209475+0000","last_active":"2024-04-20T10:22:52.251864+0000","last_peered":"2024-04-20T10:22:52.251864+0000","last_clean":"2024-04-20T10:22:52.251864+0000","last_became_active":"2024-04-20T10:22:51.208015+0000","last_became_peered":"2024-04-20T10:22:51.208015+0000","last_unstale":"2024-04-20T10:22:52.251864+0000","last_undegraded":"2024-04-20T10:22:52.251864+0000","last_fullsized":"2024-04-20T10:22:52.251864+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:31:20.035828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.755614+0000","last_change":"2024-04-20T10:22:50.196011+0000","last_active":"2024-04-20T10:22:52.755614+0000","last_peered":"2024-04-20T10:22:52.755614+0000","last_clean":"2024-04-20T10:22:52.755614+0000","last_became_active":"2024-04-20T10:22:50.195748+0000","last_became_peered":"2024-04-20T10:22:50.195748+0000","last_unstale":"2024-04-20T10:22:52.755614+0000","last_undegraded":"2024-04-20T10:22:52.755614+0000","last_fullsized":"2024-04-20T10:22:52.755614+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:16:59.003634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.755796+0000","last_change":"2024-04-20T10:22:51.209020+0000","last_active":"2024-04-20T10:22:52.755796+0000","last_peered":"2024-04-20T10:22:52.755796+0000","last_clean":"2024-04-20T10:22:52.755796+0000","last_became_active":"2024-04-20T10:22:51.208794+0000","last_became_peered":"2024-04-20T10:22:51.208794+0000","last_unstale":"2024-04-20T10:22:52.755796+0000","last_undegraded":"2024-04-20T10:22:52.755796+0000","last_fullsized":"2024-04-20T10:22:52.755796+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:47:34.076559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.755851+0000","last_change":"2024-04-20T10:22:50.203809+0000","last_active":"2024-04-20T10:22:52.755851+0000","last_peered":"2024-04-20T10:22:52.755851+0000","last_clean":"2024-04-20T10:22:52.755851+0000","last_became_active":"2024-04-20T10:22:50.203451+0000","last_became_peered":"2024-04-20T10:22:50.203451+0000","last_unstale":"2024-04-20T10:22:52.755851+0000","last_undegraded":"2024-04-20T10:22:52.755851+0000","last_fullsized":"2024-04-20T10:22:52.755851+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:39:48.488461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.466825+0000","last_change":"2024-04-20T10:22:51.208821+0000","last_active":"2024-04-20T10:22:52.466825+0000","last_peered":"2024-04-20T10:22:52.466825+0000","last_clean":"2024-04-20T10:22:52.466825+0000","last_became_active":"2024-04-20T10:22:51.208525+0000","last_became_peered":"2024-04-20T10:22:51.208525+0000","last_unstale":"2024-04-20T10:22:52.466825+0000","last_undegraded":"2024-04-20T10:22:52.466825+0000","last_fullsized":"2024-04-20T10:22:52.466825+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:34:34.224444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.211375+0000","last_change":"2024-04-20T10:22:50.197399+0000","last_active":"2024-04-20T10:22:52.211375+0000","last_peered":"2024-04-20T10:22:52.211375+0000","last_clean":"2024-04-20T10:22:52.211375+0000","last_became_active":"2024-04-20T10:22:50.197015+0000","last_became_peered":"2024-04-20T10:22:50.197015+0000","last_unstale":"2024-04-20T10:22:52.211375+0000","last_undegraded":"2024-04-20T10:22:52.211375+0000","last_fullsized":"2024-04-20T10:22:52.211375+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:47:00.135513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.756824+0000","last_change":"2024-04-20T10:22:51.212368+0000","last_active":"2024-04-20T10:22:52.756824+0000","last_peered":"2024-04-20T10:22:52.756824+0000","last_clean":"2024-04-20T10:22:52.756824+0000","last_became_active":"2024-04-20T10:22:51.208131+0000","last_became_peered":"2024-04-20T10:22:51.208131+0000","last_unstale":"2024-04-20T10:22:52.756824+0000","last_undegraded":"2024-04-20T10:22:52.756824+0000","last_fullsized":"2024-04-20T10:22:52.756824+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:23:42.010662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.756079+0000","last_change":"2024-04-20T10:22:50.203369+0000","last_active":"2024-04-20T10:22:52.756079+0000","last_peered":"2024-04-20T10:22:52.756079+0000","last_clean":"2024-04-20T10:22:52.756079+0000","last_became_active":"2024-04-20T10:22:50.203083+0000","last_became_peered":"2024-04-20T10:22:50.203083+0000","last_unstale":"2024-04-20T10:22:52.756079+0000","last_undegraded":"2024-04-20T10:22:52.756079+0000","last_fullsized":"2024-04-20T10:22:52.756079+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:10:15.543826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.757719+0000","last_change":"2024-04-20T10:22:51.219412+0000","last_active":"2024-04-20T10:22:52.757719+0000","last_peered":"2024-04-20T10:22:52.757719+0000","last_clean":"2024-04-20T10:22:52.757719+0000","last_became_active":"2024-04-20T10:22:51.218872+0000","last_became_peered":"2024-04-20T10:22:51.218872+0000","last_unstale":"2024-04-20T10:22:52.757719+0000","last_undegraded":"2024-04-20T10:22:52.757719+0000","last_fullsized":"2024-04-20T10:22:52.757719+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:09:44.641825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.251590+0000","last_change":"2024-04-20T10:22:50.198717+0000","last_active":"2024-04-20T10:22:52.251590+0000","last_peered":"2024-04-20T10:22:52.251590+0000","last_clean":"2024-04-20T10:22:52.251590+0000","last_became_active":"2024-04-20T10:22:50.197112+0000","last_became_peered":"2024-04-20T10:22:50.197112+0000","last_unstale":"2024-04-20T10:22:52.251590+0000","last_undegraded":"2024-04-20T10:22:52.251590+0000","last_fullsized":"2024-04-20T10:22:52.251590+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:54:45.115849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.760929+0000","last_change":"2024-04-20T10:22:51.226656+0000","last_active":"2024-04-20T10:22:52.760929+0000","last_peered":"2024-04-20T10:22:52.760929+0000","last_clean":"2024-04-20T10:22:52.760929+0000","last_became_active":"2024-04-20T10:22:51.226325+0000","last_became_peered":"2024-04-20T10:22:51.226325+0000","last_unstale":"2024-04-20T10:22:52.760929+0000","last_undegraded":"2024-04-20T10:22:52.760929+0000","last_fullsized":"2024-04-20T10:22:52.760929+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:50.161787+0000","last_clean_scrub_stamp":"2024-04-20T10:22:50.161787+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:15:26.440918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.757755+0000","last_change":"2024-04-20T10:22:50.203854+0000","last_active":"2024-04-20T10:22:52.757755+0000","last_peered":"2024-04-20T10:22:52.757755+0000","last_clean":"2024-04-20T10:22:52.757755+0000","last_became_active":"2024-04-20T10:22:50.203439+0000","last_became_peered":"2024-04-20T10:22:50.203439+0000","last_unstale":"2024-04-20T10:22:52.757755+0000","last_undegraded":"2024-04-20T10:22:52.757755+0000","last_fullsized":"2024-04-20T10:22:52.757755+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:22:49.145911+0000","last_clean_scrub_stamp":"2024-04-20T10:22:49.145911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:30:35.298814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":203,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-20T10:22:52.757056+0000","last_change":"2024-04-20T10:20:52.916129+0000","last_active":"2024-04-20T10:22:52.757056+0000","last_peered":"2024-04-20T10:22:52.757056+0000","last_clean":"2024-04-20T10:22:52.757056+0000","last_became_active":"2024-04-20T10:20:52.913922+0000","last_became_peered":"2024-04-20T10:20:52.913922+0000","last_unstale":"2024-04-20T10:22:52.757056+0000","last_undegraded":"2024-04-20T10:22:52.757056+0000","last_fullsized":"2024-04-20T10:22:52.757056+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:20:51.267588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:20:51.267588+0000","last_clean_scrub_stamp":"2024-04-20T10:20:51.267588+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:30:44.465836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542171,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1056768,"data_stored":785397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542172,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":193929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542171,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":193459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542171,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":454656,"data_stored":193951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542171,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":462848,"data_stored":194993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542171,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":193417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542173,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":450560,"data_stored":193437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542173,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343800,"statfs":{"total":96636764160,"available":96608051200,"internally_reserved":0,"allocated":1044480,"data_stored":783749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T10:23:01.048 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-20T10:23:01.048 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-20T10:23:01.301 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.302+0000 7f88aea00640 1 Processor -- start 2024-04-20T10:23:01.301 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.302+0000 7f88aea00640 1 -- start start 2024-04-20T10:23:01.303 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.303+0000 7f88aea00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f88a815a1f0 0x7f88a8072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:01.303 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.303+0000 7f88aea00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f88a8072190 0x7f88a8073420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:01.303 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.303+0000 7f88aea00640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88a815cc90 con 0x7f88a8159e10 2024-04-20T10:23:01.303 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.303+0000 7f88aea00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f88a8072900 con 0x7f88a815a1f0 2024-04-20T10:23:01.303 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.303+0000 7f88aea00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f88a815ca40 con 0x7f88a8072190 2024-04-20T10:23:01.304 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.304+0000 7f88acc00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f88a815a1f0 0x7f88a8072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:01.304 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.304+0000 7f88acc00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f88a815a1f0 0x7f88a8072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33270/0 (socket says 172.21.15.121:33270) 2024-04-20T10:23:01.304 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.304+0000 7f88acc00640 1 -- 172.21.15.121:0/688359083 learned_addr learned my addr 172.21.15.121:0/688359083 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:23:01.304 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.304+0000 7f88ae000640 1 --2- 172.21.15.121:0/688359083 >> v2:172.21.15.123:3300/0 conn(0x7f88a8072190 0x7f88a8073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:01.304 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.304+0000 7f88acc00640 1 -- 172.21.15.121:0/688359083 >> v2:172.21.15.123:3300/0 conn(0x7f88a8072190 msgr2=0x7f88a8073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:23:01.304 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.304+0000 7f88acc00640 1 --2- 172.21.15.121:0/688359083 >> v2:172.21.15.123:3300/0 conn(0x7f88a8072190 0x7f88a8073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:01.304 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.304+0000 7f88acc00640 1 -- 172.21.15.121:0/688359083 >> v1:172.21.15.123:6789/0 conn(0x7f88a8159e10 legacy=0x7f88a8071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:23:01.304 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.304+0000 7f88acc00640 1 -- 172.21.15.121:0/688359083 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a815cf70 con 0x7f88a815a1f0 2024-04-20T10:23:01.305 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.306+0000 7f88acc00640 1 --2- 172.21.15.121:0/688359083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88a815a1f0 0x7f88a8072ee0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f88a815daf0 tx=0x7f889402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33dec465fce5affd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:01.306 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.306+0000 7f88a7e00640 1 -- 172.21.15.121:0/688359083 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8894035020 con 0x7f88a815a1f0 2024-04-20T10:23:01.306 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.306+0000 7f88a7e00640 1 -- 172.21.15.121:0/688359083 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8894031870 con 0x7f88a815a1f0 2024-04-20T10:23:01.306 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.306+0000 7f88a7e00640 1 -- 172.21.15.121:0/688359083 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8894031c40 con 0x7f88a815a1f0 2024-04-20T10:23:01.307 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.307+0000 7f88aea00640 1 -- 172.21.15.121:0/688359083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88a815a1f0 msgr2=0x7f88a8072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:01.307 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.307+0000 7f88aea00640 1 --2- 172.21.15.121:0/688359083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88a815a1f0 0x7f88a8072ee0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f88a815daf0 tx=0x7f889402b910 comp rx=0 tx=0).stop 2024-04-20T10:23:01.307 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.307+0000 7f88aea00640 1 -- 172.21.15.121:0/688359083 shutdown_connections 2024-04-20T10:23:01.307 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.307+0000 7f88aea00640 1 --2- 172.21.15.121:0/688359083 >> v2:172.21.15.123:3300/0 conn(0x7f88a8072190 0x7f88a8073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:01.307 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.307+0000 7f88aea00640 1 --2- 172.21.15.121:0/688359083 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88a815a1f0 0x7f88a8072ee0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:01.307 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.307+0000 7f88aea00640 1 -- 172.21.15.121:0/688359083 >> 172.21.15.121:0/688359083 conn(0x7f88a80d0040 msgr2=0x7f88a80d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:01.307 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.307+0000 7f88aea00640 1 -- 172.21.15.121:0/688359083 shutdown_connections 2024-04-20T10:23:01.307 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.307+0000 7f88aea00640 1 -- 172.21.15.121:0/688359083 wait complete. 2024-04-20T10:23:01.308 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.309+0000 7f88aea00640 1 Processor -- start 2024-04-20T10:23:01.308 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.309+0000 7f88aea00640 1 -- start start 2024-04-20T10:23:01.311 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.310+0000 7f88aea00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88a8072190 0x7f88a814da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:01.311 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.310+0000 7f88aea00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88a8159e10 0x7f88a806b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:01.311 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.310+0000 7f88aea00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f88a815a1f0 0x7f88a80b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:01.311 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.310+0000 7f88aea00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f88a815d810 con 0x7f88a8159e10 2024-04-20T10:23:01.311 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.310+0000 7f88aea00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f88a815d530 con 0x7f88a8072190 2024-04-20T10:23:01.311 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.310+0000 7f88aea00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f88a806da90 con 0x7f88a815a1f0 2024-04-20T10:23:01.311 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.311+0000 7f88acc00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88a8159e10 0x7f88a806b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:01.311 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.311+0000 7f88acc00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88a8159e10 0x7f88a806b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33278/0 (socket says 172.21.15.121:33278) 2024-04-20T10:23:01.311 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.311+0000 7f88acc00640 1 -- 172.21.15.121:0/348881680 learned_addr learned my addr 172.21.15.121:0/348881680 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:23:01.312 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.311+0000 7f88ad600640 1 --2- 172.21.15.121:0/348881680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88a8072190 0x7f88a814da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:01.312 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.311+0000 7f88ae000640 1 --2- 172.21.15.121:0/348881680 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f88a815a1f0 0x7f88a80b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:01.312 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.311+0000 7f88acc00640 1 -- 172.21.15.121:0/348881680 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f88a815a1f0 msgr2=0x7f88a80b4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:01.312 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.311+0000 7f88acc00640 1 --2- 172.21.15.121:0/348881680 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f88a815a1f0 0x7f88a80b4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:01.312 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.311+0000 7f88acc00640 1 -- 172.21.15.121:0/348881680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88a8072190 msgr2=0x7f88a814da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:01.312 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.311+0000 7f88acc00640 1 --2- 172.21.15.121:0/348881680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88a8072190 0x7f88a814da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:01.312 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.311+0000 7f88acc00640 1 -- 172.21.15.121:0/348881680 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a814df40 con 0x7f88a8159e10 2024-04-20T10:23:01.312 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.312+0000 7f88acc00640 1 --2- 172.21.15.121:0/348881680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88a8159e10 0x7f88a806b900 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8894000c00 tx=0x7f88940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:01.313 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.312+0000 7f88a5600640 1 -- 172.21.15.121:0/348881680 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8894040070 con 0x7f88a8159e10 2024-04-20T10:23:01.313 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.312+0000 7f88a5600640 1 -- 172.21.15.121:0/348881680 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8894030d00 con 0x7f88a8159e10 2024-04-20T10:23:01.313 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.312+0000 7f88aea00640 1 -- 172.21.15.121:0/348881680 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88a806c150 con 0x7f88a8159e10 2024-04-20T10:23:01.313 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.312+0000 7f88a5600640 1 -- 172.21.15.121:0/348881680 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8894035040 con 0x7f88a8159e10 2024-04-20T10:23:01.313 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.312+0000 7f88aea00640 1 -- 172.21.15.121:0/348881680 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88a80b4220 con 0x7f88a8159e10 2024-04-20T10:23:01.313 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.313+0000 7f88aea00640 1 -- 172.21.15.121:0/348881680 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88a806c670 con 0x7f88a8159e10 2024-04-20T10:23:01.314 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.314+0000 7f88a5600640 1 -- 172.21.15.121:0/348881680 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f889404b060 con 0x7f88a8159e10 2024-04-20T10:23:01.315 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.315+0000 7f88a5600640 1 --2- 172.21.15.121:0/348881680 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f888806c8f0 0x7f888806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:01.315 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.315+0000 7f88a5600640 1 -- 172.21.15.121:0/348881680 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f88940ada60 con 0x7f88a8159e10 2024-04-20T10:23:01.315 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.316+0000 7f88ad600640 1 --2- 172.21.15.121:0/348881680 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f888806c8f0 0x7f888806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:01.316 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.316+0000 7f88ad600640 1 --2- 172.21.15.121:0/348881680 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f888806c8f0 0x7f888806edb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f889c004ea0 tx=0x7f889c009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:01.319 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.320+0000 7f88a5600640 1 -- 172.21.15.121:0/348881680 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f88a806c670 con 0x7f88a8159e10 2024-04-20T10:23:01.593 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.594+0000 7f88aea00640 1 -- 172.21.15.121:0/348881680 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f88a81595c0 con 0x7f88a8159e10 2024-04-20T10:23:01.599 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.599+0000 7f88a5600640 1 -- 172.21.15.121:0/348881680 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7f889407bcf0 con 0x7f88a8159e10 2024-04-20T10:23:01.599 INFO:teuthology.orchestra.run.smithi121.stdout:[client.0] 2024-04-20T10:23:01.599 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDPliNmP7t4HxAACvwRBNvbdSP0BmDmaoo32g== 2024-04-20T10:23:01.599 INFO:teuthology.orchestra.run.smithi121.stdout: caps mds = "allow" 2024-04-20T10:23:01.599 INFO:teuthology.orchestra.run.smithi121.stdout: caps mon = "allow r" 2024-04-20T10:23:01.599 INFO:teuthology.orchestra.run.smithi121.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-20T10:23:01.600 INFO:teuthology.orchestra.run.smithi121.stderr:updated caps for client.0 2024-04-20T10:23:01.601 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.602+0000 7f88aea00640 1 -- 172.21.15.121:0/348881680 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f888806c8f0 msgr2=0x7f888806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:01.601 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.602+0000 7f88aea00640 1 --2- 172.21.15.121:0/348881680 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f888806c8f0 0x7f888806edb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f889c004ea0 tx=0x7f889c009040 comp rx=0 tx=0).stop 2024-04-20T10:23:01.602 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.602+0000 7f88aea00640 1 -- 172.21.15.121:0/348881680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88a8159e10 msgr2=0x7f88a806b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:01.602 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.602+0000 7f88aea00640 1 --2- 172.21.15.121:0/348881680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88a8159e10 0x7f88a806b900 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8894000c00 tx=0x7f88940046c0 comp rx=0 tx=0).stop 2024-04-20T10:23:01.602 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.602+0000 7f88aea00640 1 -- 172.21.15.121:0/348881680 shutdown_connections 2024-04-20T10:23:01.602 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.602+0000 7f88aea00640 1 --2- 172.21.15.121:0/348881680 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f88a815a1f0 0x7f88a80b4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:01.602 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.602+0000 7f88aea00640 1 --2- 172.21.15.121:0/348881680 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f88a8159e10 0x7f88a806b900 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:01.602 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.602+0000 7f88aea00640 1 --2- 172.21.15.121:0/348881680 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f888806c8f0 0x7f888806edb0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:01.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.602+0000 7f88aea00640 1 --2- 172.21.15.121:0/348881680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88a8072190 0x7f88a814da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:01.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.602+0000 7f88aea00640 1 -- 172.21.15.121:0/348881680 >> 172.21.15.121:0/348881680 conn(0x7f88a80d0040 msgr2=0x7f88a8156b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:01.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.602+0000 7f88aea00640 1 -- 172.21.15.121:0/348881680 shutdown_connections 2024-04-20T10:23:01.603 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.602+0000 7f88aea00640 1 -- 172.21.15.121:0/348881680 wait complete. 2024-04-20T10:23:01.620 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:23:01.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.821+0000 7f214b200640 1 Processor -- start 2024-04-20T10:23:01.821 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.821+0000 7f214b200640 1 -- start start 2024-04-20T10:23:01.823 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.823+0000 7f214b200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f214c161d50 0x7f214c0d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:01.823 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.823+0000 7f214b200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f214c06cde0 0x7f214c0b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:01.823 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.823+0000 7f214b200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f214c06c190 con 0x7f214c0b3330 2024-04-20T10:23:01.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.823+0000 7f214b200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f214c162130 con 0x7f214c161d50 2024-04-20T10:23:01.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.823+0000 7f214b200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f214c06c470 con 0x7f214c06cde0 2024-04-20T10:23:01.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.823+0000 7f2149e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f214c161d50 0x7f214c0d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:01.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.823+0000 7f2143e00640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f214c0b3330 0x7f214c06c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:50456/0 (socket says 172.21.15.121:50456) 2024-04-20T10:23:01.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.823+0000 7f2143e00640 1 -- 172.21.15.121:0/751392387 learned_addr learned my addr 172.21.15.121:0/751392387 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:23:01.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.824+0000 7f214a800640 1 --2- 172.21.15.121:0/751392387 >> v2:172.21.15.123:3300/0 conn(0x7f214c06cde0 0x7f214c0b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:01.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.824+0000 7f2149e00640 1 -- 172.21.15.121:0/751392387 >> v2:172.21.15.123:3300/0 conn(0x7f214c06cde0 msgr2=0x7f214c0b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:23:01.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.824+0000 7f2149e00640 1 --2- 172.21.15.121:0/751392387 >> v2:172.21.15.123:3300/0 conn(0x7f214c06cde0 0x7f214c0b3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:01.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.824+0000 7f2149e00640 1 -- 172.21.15.121:0/751392387 >> v1:172.21.15.121:6789/0 conn(0x7f214c0b3330 legacy=0x7f214c06c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:01.824 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.824+0000 7f2149e00640 1 -- 172.21.15.121:0/751392387 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f214c0d2e00 con 0x7f214c161d50 2024-04-20T10:23:01.825 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.825+0000 7f2149e00640 1 --2- 172.21.15.121:0/751392387 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f214c161d50 0x7f214c0d38e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f214c0cee50 tx=0x7f213c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e86cb91752280631 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:01.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.825+0000 7f2149400640 1 -- 172.21.15.121:0/751392387 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213c035020 con 0x7f214c161d50 2024-04-20T10:23:01.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.825+0000 7f2149400640 1 -- 172.21.15.121:0/751392387 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f213c031830 con 0x7f214c161d50 2024-04-20T10:23:01.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.825+0000 7f2149400640 1 -- 172.21.15.121:0/751392387 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f213c031c00 con 0x7f214c161d50 2024-04-20T10:23:01.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.826+0000 7f214b200640 1 -- 172.21.15.121:0/751392387 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f214c161d50 msgr2=0x7f214c0d38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:01.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.826+0000 7f214b200640 1 --2- 172.21.15.121:0/751392387 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f214c161d50 0x7f214c0d38e0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f214c0cee50 tx=0x7f213c02b8d0 comp rx=0 tx=0).stop 2024-04-20T10:23:01.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.826+0000 7f214b200640 1 -- 172.21.15.121:0/751392387 shutdown_connections 2024-04-20T10:23:01.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.826+0000 7f214b200640 1 --2- 172.21.15.121:0/751392387 >> v2:172.21.15.123:3300/0 conn(0x7f214c06cde0 0x7f214c0b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:01.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.826+0000 7f214b200640 1 --2- 172.21.15.121:0/751392387 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f214c161d50 0x7f214c0d38e0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:01.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.826+0000 7f214b200640 1 -- 172.21.15.121:0/751392387 >> 172.21.15.121:0/751392387 conn(0x7f214c0d4450 msgr2=0x7f214c0d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:01.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.826+0000 7f214b200640 1 -- 172.21.15.121:0/751392387 shutdown_connections 2024-04-20T10:23:01.826 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.826+0000 7f214b200640 1 -- 172.21.15.121:0/751392387 wait complete. 2024-04-20T10:23:01.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.828+0000 7f214b200640 1 Processor -- start 2024-04-20T10:23:01.827 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.828+0000 7f214b200640 1 -- start start 2024-04-20T10:23:01.829 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.828+0000 7f214b200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f214c06cde0 0x7f214c1f4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:01.829 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.829+0000 7f214b200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f214c0b3330 0x7f214c149ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:01.830 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.829+0000 7f214b200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f214c161d50 0x7f214c0cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:01.830 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.829+0000 7f214b200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f214c0b7660 con 0x7f214c0b3330 2024-04-20T10:23:01.830 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.829+0000 7f214b200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f214c0d33c0 con 0x7f214c161d50 2024-04-20T10:23:01.830 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.829+0000 7f214b200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f214c0b77e0 con 0x7f214c06cde0 2024-04-20T10:23:01.830 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.829+0000 7f2143e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f214c0b3330 0x7f214c149ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:01.830 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.829+0000 7f2143e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f214c0b3330 0x7f214c149ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33294/0 (socket says 172.21.15.121:33294) 2024-04-20T10:23:01.830 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.829+0000 7f2143e00640 1 -- 172.21.15.121:0/4071636491 learned_addr learned my addr 172.21.15.121:0/4071636491 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:23:01.830 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.829+0000 7f2149e00640 1 --2- 172.21.15.121:0/4071636491 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f214c06cde0 0x7f214c1f4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:01.830 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.829+0000 7f214a800640 1 --2- 172.21.15.121:0/4071636491 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f214c161d50 0x7f214c0cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:01.830 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.829+0000 7f2143e00640 1 -- 172.21.15.121:0/4071636491 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f214c06cde0 msgr2=0x7f214c1f4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:01.830 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.829+0000 7f2143e00640 1 --2- 172.21.15.121:0/4071636491 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f214c06cde0 0x7f214c1f4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:01.830 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.829+0000 7f2143e00640 1 -- 172.21.15.121:0/4071636491 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f214c161d50 msgr2=0x7f214c0cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:01.830 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.829+0000 7f2143e00640 1 --2- 172.21.15.121:0/4071636491 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f214c161d50 0x7f214c0cb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:01.831 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.829+0000 7f2143e00640 1 -- 172.21.15.121:0/4071636491 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f214c14ae50 con 0x7f214c0b3330 2024-04-20T10:23:01.831 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.829+0000 7f2149e00640 1 --2- 172.21.15.121:0/4071636491 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f214c06cde0 0x7f214c1f4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:23:01.831 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.829+0000 7f214a800640 1 --2- 172.21.15.121:0/4071636491 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f214c161d50 0x7f214c0cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:23:01.831 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.830+0000 7f2143e00640 1 --2- 172.21.15.121:0/4071636491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f214c0b3330 0x7f214c149ee0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f213400c430 tx=0x7f213400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:01.831 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.830+0000 7f2142000640 1 -- 172.21.15.121:0/4071636491 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2134017070 con 0x7f214c0b3330 2024-04-20T10:23:01.831 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.830+0000 7f2142000640 1 -- 172.21.15.121:0/4071636491 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2134005ce0 con 0x7f214c0b3330 2024-04-20T10:23:01.831 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.830+0000 7f2142000640 1 -- 172.21.15.121:0/4071636491 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21340072f0 con 0x7f214c0b3330 2024-04-20T10:23:01.831 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.830+0000 7f214b200640 1 -- 172.21.15.121:0/4071636491 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f214c14a890 con 0x7f214c0b3330 2024-04-20T10:23:01.831 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.830+0000 7f214b200640 1 -- 172.21.15.121:0/4071636491 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f214c0b3830 con 0x7f214c0b3330 2024-04-20T10:23:01.832 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.831+0000 7f214b200640 1 -- 172.21.15.121:0/4071636491 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f214c075b60 con 0x7f214c0b3330 2024-04-20T10:23:01.832 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.832+0000 7f2142000640 1 -- 172.21.15.121:0/4071636491 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f213401a070 con 0x7f214c0b3330 2024-04-20T10:23:01.832 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.833+0000 7f2142000640 1 --2- 172.21.15.121:0/4071636491 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f212806c8f0 0x7f212806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:01.833 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.833+0000 7f2142000640 1 -- 172.21.15.121:0/4071636491 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f213408b140 con 0x7f214c0b3330 2024-04-20T10:23:01.833 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.833+0000 7f2149e00640 1 --2- 172.21.15.121:0/4071636491 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f212806c8f0 0x7f212806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:01.833 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.834+0000 7f2149e00640 1 --2- 172.21.15.121:0/4071636491 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f212806c8f0 0x7f212806edb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f213c036000 tx=0x7f213c008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:01.837 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:01.837+0000 7f2142000640 1 -- 172.21.15.121:0/4071636491 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2134059340 con 0x7f214c0b3330 2024-04-20T10:23:02.078 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.079+0000 7f214b200640 1 -- 172.21.15.121:0/4071636491 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f214c15bfd0 con 0x7f214c0b3330 2024-04-20T10:23:02.079 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.080+0000 7f2142000640 1 -- 172.21.15.121:0/4071636491 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5202 (secure 0 0 0) 0x7f2134058c40 con 0x7f214c0b3330 2024-04-20T10:23:02.079 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:23:02.080 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4715,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/788634820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":788634820},{"type":"v1","addr":"172.21.15.123:6835","nonce":788634820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4942,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/331480838","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":331480838},{"type":"v1","addr":"172.21.15.123:6833","nonce":331480838}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5562,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.121:6835/2525740087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6834","nonce":2525740087},{"type":"v1","addr":"172.21.15.121:6835","nonce":2525740087}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:22:50.639487+0000","modified":"2024-04-20T10:22:53.697379+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5568},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5568":{"gid":5568,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.121:6837/257724942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":257724942},{"type":"v1","addr":"172.21.15.121:6837","nonce":257724942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5568,"qdb_cluster":[5568]},"id":2}]} 2024-04-20T10:23:02.080 INFO:teuthology.orchestra.run.smithi121.stderr:dumped fsmap epoch 17 2024-04-20T10:23:02.082 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.082+0000 7f214b200640 1 -- 172.21.15.121:0/4071636491 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f212806c8f0 msgr2=0x7f212806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:02.082 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.082+0000 7f214b200640 1 --2- 172.21.15.121:0/4071636491 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f212806c8f0 0x7f212806edb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f213c036000 tx=0x7f213c008030 comp rx=0 tx=0).stop 2024-04-20T10:23:02.082 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.082+0000 7f214b200640 1 -- 172.21.15.121:0/4071636491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f214c0b3330 msgr2=0x7f214c149ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:02.082 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.082+0000 7f214b200640 1 --2- 172.21.15.121:0/4071636491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f214c0b3330 0x7f214c149ee0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f213400c430 tx=0x7f213400c8f0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.082 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.082+0000 7f214b200640 1 -- 172.21.15.121:0/4071636491 shutdown_connections 2024-04-20T10:23:02.082 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.082+0000 7f214b200640 1 --2- 172.21.15.121:0/4071636491 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f212806c8f0 0x7f212806edb0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.083 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.082+0000 7f214b200640 1 --2- 172.21.15.121:0/4071636491 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f214c161d50 0x7f214c0cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.083 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.082+0000 7f214b200640 1 --2- 172.21.15.121:0/4071636491 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f214c0b3330 0x7f214c149ee0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.083 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.082+0000 7f214b200640 1 --2- 172.21.15.121:0/4071636491 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f214c06cde0 0x7f214c1f4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.083 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.082+0000 7f214b200640 1 -- 172.21.15.121:0/4071636491 >> 172.21.15.121:0/4071636491 conn(0x7f214c0d4450 msgr2=0x7f214c0cf2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:02.083 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.083+0000 7f214b200640 1 -- 172.21.15.121:0/4071636491 shutdown_connections 2024-04-20T10:23:02.083 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.083+0000 7f214b200640 1 -- 172.21.15.121:0/4071636491 wait complete. 2024-04-20T10:23:02.503 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T10:22:50.639487+0000', 'modified': '2024-04-20T10:22:53.697379+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5568}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5568': {'gid': 5568, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.121:6837/257724942', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.121:6836', 'nonce': 257724942}, {'type': 'v1', 'addr': '172.21.15.121:6837', 'nonce': 257724942}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5568, 'qdb_cluster': [5568]} 2024-04-20T10:23:02.504 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T10:23:02.504 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-20T10:23:02.714 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.715+0000 7ff3a0a00640 1 Processor -- start 2024-04-20T10:23:02.714 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.715+0000 7ff3a0a00640 1 -- start start 2024-04-20T10:23:02.716 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.716+0000 7ff3a0a00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7ff39c15a1f0 0x7ff39c072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:02.716 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.716+0000 7ff3a0a00640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff39c072190 0x7ff39c073420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:02.716 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.716+0000 7ff3a0a00640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff39c15cc90 con 0x7ff39c159e10 2024-04-20T10:23:02.716 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.716+0000 7ff3a0a00640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7ff39c072900 con 0x7ff39c15a1f0 2024-04-20T10:23:02.716 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.716+0000 7ff3a0a00640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7ff39c15ca40 con 0x7ff39c072190 2024-04-20T10:23:02.716 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.717+0000 7ff39b400640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7ff39c159e10 0x7ff39c071a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:50472/0 (socket says 172.21.15.121:50472) 2024-04-20T10:23:02.716 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.717+0000 7ff39b400640 1 -- 172.21.15.121:0/2151607 learned_addr learned my addr 172.21.15.121:0/2151607 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:23:02.717 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.717+0000 7ff392a00640 1 --2- 172.21.15.121:0/2151607 >> v2:172.21.15.121:3300/0 conn(0x7ff39c15a1f0 0x7ff39c072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:02.719 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.717+0000 7ff39be00640 1 --2- 172.21.15.121:0/2151607 >> v2:172.21.15.123:3300/0 conn(0x7ff39c072190 0x7ff39c073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:02.720 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.717+0000 7ff39aa00640 1 -- 172.21.15.121:0/2151607 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 263384567 0 0) 0x7ff39c15cc90 con 0x7ff39c159e10 2024-04-20T10:23:02.720 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.717+0000 7ff39aa00640 1 -- 172.21.15.121:0/2151607 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff380003520 con 0x7ff39c159e10 2024-04-20T10:23:02.720 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.717+0000 7ff392a00640 1 -- 172.21.15.121:0/2151607 >> v2:172.21.15.123:3300/0 conn(0x7ff39c072190 msgr2=0x7ff39c073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:23:02.720 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.717+0000 7ff392a00640 1 --2- 172.21.15.121:0/2151607 >> v2:172.21.15.123:3300/0 conn(0x7ff39c072190 0x7ff39c073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.720 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.717+0000 7ff392a00640 1 -- 172.21.15.121:0/2151607 >> v1:172.21.15.121:6789/0 conn(0x7ff39c159e10 legacy=0x7ff39c071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:02.721 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.717+0000 7ff392a00640 1 -- 172.21.15.121:0/2151607 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff39c15cf70 con 0x7ff39c15a1f0 2024-04-20T10:23:02.721 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.717+0000 7ff39be00640 1 --2- 172.21.15.121:0/2151607 >> v2:172.21.15.123:3300/0 conn(0x7ff39c072190 0x7ff39c073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:23:02.721 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.718+0000 7ff392a00640 1 --2- 172.21.15.121:0/2151607 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff39c15a1f0 0x7ff39c072ee0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff39c15daf0 tx=0x7ff38402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2abea8d06b802cea server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:02.721 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.719+0000 7ff39aa00640 1 -- 172.21.15.121:0/2151607 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384035020 con 0x7ff39c15a1f0 2024-04-20T10:23:02.721 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.719+0000 7ff39aa00640 1 -- 172.21.15.121:0/2151607 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff384031830 con 0x7ff39c15a1f0 2024-04-20T10:23:02.722 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.719+0000 7ff39aa00640 1 -- 172.21.15.121:0/2151607 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384031c00 con 0x7ff39c15a1f0 2024-04-20T10:23:02.722 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.719+0000 7ff3a0a00640 1 -- 172.21.15.121:0/2151607 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff39c15a1f0 msgr2=0x7ff39c072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:02.722 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.719+0000 7ff3a0a00640 1 --2- 172.21.15.121:0/2151607 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff39c15a1f0 0x7ff39c072ee0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7ff39c15daf0 tx=0x7ff38402b8d0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.722 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.719+0000 7ff3a0a00640 1 -- 172.21.15.121:0/2151607 shutdown_connections 2024-04-20T10:23:02.722 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.719+0000 7ff3a0a00640 1 --2- 172.21.15.121:0/2151607 >> v2:172.21.15.123:3300/0 conn(0x7ff39c072190 0x7ff39c073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.722 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.719+0000 7ff3a0a00640 1 --2- 172.21.15.121:0/2151607 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff39c15a1f0 0x7ff39c072ee0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.723 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.719+0000 7ff3a0a00640 1 -- 172.21.15.121:0/2151607 >> 172.21.15.121:0/2151607 conn(0x7ff39c0d0040 msgr2=0x7ff39c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:02.723 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.720+0000 7ff3a0a00640 1 -- 172.21.15.121:0/2151607 shutdown_connections 2024-04-20T10:23:02.723 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.720+0000 7ff3a0a00640 1 -- 172.21.15.121:0/2151607 wait complete. 2024-04-20T10:23:02.725 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.722+0000 7ff3a0a00640 1 Processor -- start 2024-04-20T10:23:02.725 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.722+0000 7ff3a0a00640 1 -- start start 2024-04-20T10:23:02.726 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.723+0000 7ff3a0a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff39c072190 0x7ff39c1451c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:02.726 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.723+0000 7ff3a0a00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff39c159e10 0x7ff39c0b0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:02.726 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.723+0000 7ff3a0a00640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff39c15a1f0 0x7ff39c0b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:02.726 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.723+0000 7ff3a0a00640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff39c15d810 con 0x7ff39c072190 2024-04-20T10:23:02.726 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.723+0000 7ff3a0a00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff39c15d530 con 0x7ff39c159e10 2024-04-20T10:23:02.726 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.723+0000 7ff3a0a00640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ff39c06da90 con 0x7ff39c15a1f0 2024-04-20T10:23:02.726 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.723+0000 7ff39b400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff39c072190 0x7ff39c1451c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:02.726 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.723+0000 7ff39b400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff39c072190 0x7ff39c1451c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33312/0 (socket says 172.21.15.121:33312) 2024-04-20T10:23:02.726 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.723+0000 7ff39b400640 1 -- 172.21.15.121:0/2553919519 learned_addr learned my addr 172.21.15.121:0/2553919519 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:23:02.726 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.724+0000 7ff392a00640 1 --2- 172.21.15.121:0/2553919519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff39c159e10 0x7ff39c0b0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:02.726 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.724+0000 7ff39b400640 1 -- 172.21.15.121:0/2553919519 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff39c15a1f0 msgr2=0x7ff39c0b0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:02.727 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.724+0000 7ff39be00640 1 --2- 172.21.15.121:0/2553919519 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff39c15a1f0 0x7ff39c0b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:02.727 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.724+0000 7ff39b400640 1 --2- 172.21.15.121:0/2553919519 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff39c15a1f0 0x7ff39c0b0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.727 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.724+0000 7ff39b400640 1 -- 172.21.15.121:0/2553919519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff39c159e10 msgr2=0x7ff39c0b0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:02.727 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.724+0000 7ff39b400640 1 --2- 172.21.15.121:0/2553919519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff39c159e10 0x7ff39c0b0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.727 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.724+0000 7ff39b400640 1 -- 172.21.15.121:0/2553919519 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff39c1478d0 con 0x7ff39c072190 2024-04-20T10:23:02.727 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.724+0000 7ff392a00640 1 --2- 172.21.15.121:0/2553919519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff39c159e10 0x7ff39c0b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:23:02.727 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.724+0000 7ff39b400640 1 --2- 172.21.15.121:0/2553919519 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff39c072190 0x7ff39c1451c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ff38c00cb40 tx=0x7ff38c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:02.727 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.724+0000 7ff393e00640 1 -- 172.21.15.121:0/2553919519 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c017070 con 0x7ff39c072190 2024-04-20T10:23:02.727 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.724+0000 7ff393e00640 1 -- 172.21.15.121:0/2553919519 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff38c00aca0 con 0x7ff39c072190 2024-04-20T10:23:02.727 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.724+0000 7ff3a0a00640 1 -- 172.21.15.121:0/2553919519 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff39c147310 con 0x7ff39c072190 2024-04-20T10:23:02.727 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.724+0000 7ff393e00640 1 -- 172.21.15.121:0/2553919519 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c012620 con 0x7ff39c072190 2024-04-20T10:23:02.727 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.725+0000 7ff3a0a00640 1 -- 172.21.15.121:0/2553919519 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff39c0c9390 con 0x7ff39c072190 2024-04-20T10:23:02.728 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.727+0000 7ff393e00640 1 -- 172.21.15.121:0/2553919519 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff38c01a070 con 0x7ff39c072190 2024-04-20T10:23:02.728 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.728+0000 7ff393e00640 1 --2- 172.21.15.121:0/2553919519 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff37806c8f0 0x7ff37806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:02.729 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.728+0000 7ff393e00640 1 -- 172.21.15.121:0/2553919519 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7ff38c08bba0 con 0x7ff39c072190 2024-04-20T10:23:02.729 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.728+0000 7ff392a00640 1 --2- 172.21.15.121:0/2553919519 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff37806c8f0 0x7ff37806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:02.729 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.729+0000 7ff3a4c63740 1 Processor -- start 2024-04-20T10:23:02.729 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.729+0000 7ff3a4c63740 1 -- start start 2024-04-20T10:23:02.729 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.729+0000 7ff392a00640 1 --2- 172.21.15.121:0/2553919519 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff37806c8f0 0x7ff37806edb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff384036000 tx=0x7ff384008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:02.731 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.730+0000 7ff3a4c63740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559b4d63c670 0x559b4d5f3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:02.731 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.730+0000 7ff3a4c63740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b4d5f26f0 0x559b4d5f41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:02.731 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.730+0000 7ff3a4c63740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x559b4d5eeb90 0x559b4d61e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:02.731 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.730+0000 7ff3a4c63740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559b4d55a9d0 con 0x559b4d63c670 2024-04-20T10:23:02.731 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.730+0000 7ff3a4c63740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559b4d501270 con 0x559b4d5f26f0 2024-04-20T10:23:02.731 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.730+0000 7ff3a4c63740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x559b4d556990 con 0x559b4d5eeb90 2024-04-20T10:23:02.731 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.730+0000 7ff39b400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559b4d63c670 0x559b4d5f3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:02.731 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.730+0000 7ff39b400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559b4d63c670 0x559b4d5f3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33326/0 (socket says 172.21.15.121:33326) 2024-04-20T10:23:02.731 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.730+0000 7ff39b400640 1 -- 172.21.15.121:0/1641086076 learned_addr learned my addr 172.21.15.121:0/1641086076 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:23:02.731 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.730+0000 7ff39b400640 1 -- 172.21.15.121:0/1641086076 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x559b4d5eeb90 msgr2=0x559b4d61e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:23:02.732 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.730+0000 7ff39b400640 1 --2- 172.21.15.121:0/1641086076 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x559b4d5eeb90 0x559b4d61e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.732 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.730+0000 7ff39b400640 1 -- 172.21.15.121:0/1641086076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b4d5f26f0 msgr2=0x559b4d5f41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:23:02.732 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.730+0000 7ff39b400640 1 --2- 172.21.15.121:0/1641086076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b4d5f26f0 0x559b4d5f41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.732 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.730+0000 7ff39b400640 1 -- 172.21.15.121:0/1641086076 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b4d26d2d0 con 0x559b4d63c670 2024-04-20T10:23:02.732 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.731+0000 7ff39b400640 1 --2- 172.21.15.121:0/1641086076 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559b4d63c670 0x559b4d5f3ca0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff38c0012b0 tx=0x7ff38c059f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30aff55e9d6cfe0e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:02.732 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.731+0000 7ff38b400640 1 -- 172.21.15.121:0/1641086076 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c062310 con 0x559b4d63c670 2024-04-20T10:23:02.732 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.731+0000 7ff38b400640 1 -- 172.21.15.121:0/1641086076 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff38c057a30 con 0x559b4d63c670 2024-04-20T10:23:02.732 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.731+0000 7ff38b400640 1 -- 172.21.15.121:0/1641086076 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c061e80 con 0x559b4d63c670 2024-04-20T10:23:02.732 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.732+0000 7ff3a4c63740 1 -- 172.21.15.121:0/1641086076 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559b4d63c670 msgr2=0x559b4d5f3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:02.732 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.732+0000 7ff3a4c63740 1 --2- 172.21.15.121:0/1641086076 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559b4d63c670 0x559b4d5f3ca0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff38c0012b0 tx=0x7ff38c059f70 comp rx=0 tx=0).stop 2024-04-20T10:23:02.735 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.732+0000 7ff3a4c63740 1 -- 172.21.15.121:0/1641086076 shutdown_connections 2024-04-20T10:23:02.736 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.732+0000 7ff3a4c63740 1 --2- 172.21.15.121:0/1641086076 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x559b4d5eeb90 0x559b4d61e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.736 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.732+0000 7ff3a4c63740 1 --2- 172.21.15.121:0/1641086076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b4d5f26f0 0x559b4d5f41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.736 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.732+0000 7ff3a4c63740 1 --2- 172.21.15.121:0/1641086076 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559b4d63c670 0x559b4d5f3ca0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.736 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.732+0000 7ff3a4c63740 1 -- 172.21.15.121:0/1641086076 >> 172.21.15.121:0/1641086076 conn(0x559b4d5f0e00 msgr2=0x559b4d639980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:02.736 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.732+0000 7ff3a4c63740 1 -- 172.21.15.121:0/1641086076 shutdown_connections 2024-04-20T10:23:02.736 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.732+0000 7ff3a4c63740 1 -- 172.21.15.121:0/1641086076 wait complete. 2024-04-20T10:23:02.736 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.733+0000 7ff3a4c63740 1 -- start start 2024-04-20T10:23:02.737 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.733+0000 7ff3a4c63740 1 Processor -- start 2024-04-20T10:23:02.737 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.734+0000 7ff3a4c63740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b4d5f26f0 0x559b4d640b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:02.737 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.734+0000 7ff3a4c63740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559b4d63c670 0x559b4d619c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:02.737 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.734+0000 7ff3a4c63740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x559b4d615830 0x559b4d61a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:02.737 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.734+0000 7ff3a4c63740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559b4d5f2c00 con 0x559b4d63c670 2024-04-20T10:23:02.738 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.734+0000 7ff3a4c63740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559b4d5145e0 con 0x559b4d5f26f0 2024-04-20T10:23:02.738 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.734+0000 7ff3a4c63740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x559b4d5f2ee0 con 0x559b4d615830 2024-04-20T10:23:02.738 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.734+0000 7ff3a4c63740 10 client.0 _ll_register_callbacks cb 0x559b4d26cac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:23:02.738 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.734+0000 7ff392a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559b4d63c670 0x559b4d619c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:02.738 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.734+0000 7ff392a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559b4d63c670 0x559b4d619c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33330/0 (socket says 172.21.15.121:33330) 2024-04-20T10:23:02.738 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.734+0000 7ff392a00640 1 -- 172.21.15.121:0/1029123918 learned_addr learned my addr 172.21.15.121:0/1029123918 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:23:02.739 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.734+0000 7ff39be00640 1 --2- 172.21.15.121:0/1029123918 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x559b4d615830 0x559b4d61a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:02.739 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.735+0000 7ff392a00640 1 -- 172.21.15.121:0/1029123918 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x559b4d615830 msgr2=0x559b4d61a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:23:02.739 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.735+0000 7ff39b400640 1 --2- 172.21.15.121:0/1029123918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b4d5f26f0 0x559b4d640b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:02.739 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.735+0000 7ff392a00640 1 --2- 172.21.15.121:0/1029123918 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x559b4d615830 0x559b4d61a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.739 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.735+0000 7ff392a00640 1 -- 172.21.15.121:0/1029123918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b4d5f26f0 msgr2=0x559b4d640b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:23:02.740 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.735+0000 7ff392a00640 1 --2- 172.21.15.121:0/1029123918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b4d5f26f0 0x559b4d640b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.740 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.735+0000 7ff392a00640 1 -- 172.21.15.121:0/1029123918 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b4d3376b0 con 0x559b4d63c670 2024-04-20T10:23:02.740 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.735+0000 7ff392a00640 1 --2- 172.21.15.121:0/1029123918 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559b4d63c670 0x559b4d619c80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff38402bee0 tx=0x7ff3840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28fb4997ef4cbb46 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:02.740 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.735+0000 7ff37fe00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-04-20T10:23:02.740 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.735+0000 7ff37fe00640 1 -- 172.21.15.121:0/1029123918 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384035040 con 0x559b4d63c670 2024-04-20T10:23:02.740 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.735+0000 7ff37fe00640 1 -- 172.21.15.121:0/1029123918 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7ff384045400 con 0x559b4d63c670 2024-04-20T10:23:02.741 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.735+0000 7ff3a4c63740 1 -- 172.21.15.121:0/1029123918 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff38c057a30 con 0x559b4d63c670 2024-04-20T10:23:02.741 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.735+0000 7ff37fe00640 1 -- 172.21.15.121:0/1029123918 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7ff384045600 con 0x559b4d63c670 2024-04-20T10:23:02.741 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.735+0000 7ff3a4c63740 10 client.5757 fetch_fsmap learned FSMap version 17 2024-04-20T10:23:02.741 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.735+0000 7ff3a4c63740 1 -- 172.21.15.121:0/1029123918 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x559b4d640070 con 0x559b4d63c670 2024-04-20T10:23:02.741 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.736+0000 7ff37fe00640 1 -- 172.21.15.121:0/1029123918 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff38c057a30 con 0x559b4d63c670 2024-04-20T10:23:02.741 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.736+0000 7ff37fe00640 1 -- 172.21.15.121:0/1029123918 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 17) ==== 3495+0+0 (secure 0 0 0) 0x7ff3840457a0 con 0x559b4d63c670 2024-04-20T10:23:02.741 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.736+0000 7ff3a4c63740 10 client.5757 fetch_fsmap finished waiting for FSMap version 17 2024-04-20T10:23:02.741 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.736+0000 7ff3a4c63740 10 client.5757 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-20T10:23:02.741 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.736+0000 7ff3a4c63740 20 client.5757 populate_metadata read hostname 'smithi121' 2024-04-20T10:23:02.741 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.736+0000 7ff3a4c63740 1 --2- 172.21.15.121:0/1029123918 >> [v2:172.21.15.121:6836/257724942,v1:172.21.15.121:6837/257724942] conn(0x559b4d64ad70 0x559b4d658fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:02.741 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.736+0000 7ff3a4c63740 4 client.5757 mds_command: new command op to 5568 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:23:02.741 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.736+0000 7ff3a4c63740 1 -- 172.21.15.121:0/1029123918 --> [v2:172.21.15.121:6836/257724942,v1:172.21.15.121:6837/257724942] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b4d5683f0 con 0x559b4d64ad70 2024-04-20T10:23:02.741 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.736+0000 7ff39b400640 1 --2- 172.21.15.121:0/1029123918 >> [v2:172.21.15.121:6836/257724942,v1:172.21.15.121:6837/257724942] conn(0x559b4d64ad70 0x559b4d658fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:02.743 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.737+0000 7ff39b400640 1 --2- 172.21.15.121:0/1029123918 >> [v2:172.21.15.121:6836/257724942,v1:172.21.15.121:6837/257724942] conn(0x559b4d64ad70 0x559b4d658fd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a117d487c2bb371 server_cookie=efee740be05377b7 in_seq=0 out_seq=0 2024-04-20T10:23:02.743 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.737+0000 7ff37fe00640 10 client.5757 ms_handle_connect on v2:172.21.15.121:6836/257724942 2024-04-20T10:23:02.743 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.738+0000 7ff37fe00640 1 -- 172.21.15.121:0/1029123918 <== mds.0 v2:172.21.15.121:6836/257724942 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559b4d5683f0 con 0x559b4d64ad70 2024-04-20T10:23:02.743 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.738+0000 7ff37fe00640 10 client.5757 handle_command_reply: tid=0 2024-04-20T10:23:02.743 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.738+0000 7ff3a4c63740 1 client.5757 shutdown 2024-04-20T10:23:02.743 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.738+0000 7ff3a4c63740 1 -- 172.21.15.121:0/1029123918 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559b4d63c670 msgr2=0x559b4d619c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:02.743 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.738+0000 7ff3a4c63740 1 --2- 172.21.15.121:0/1029123918 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559b4d63c670 0x559b4d619c80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff38402bee0 tx=0x7ff3840046c0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.743 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.738+0000 7ff3a4c63740 1 -- 172.21.15.121:0/1029123918 shutdown_connections 2024-04-20T10:23:02.743 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.738+0000 7ff3a4c63740 1 --2- 172.21.15.121:0/1029123918 >> [v2:172.21.15.121:6836/257724942,v1:172.21.15.121:6837/257724942] conn(0x559b4d64ad70 0x559b4d658fd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.743 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.738+0000 7ff3a4c63740 1 --2- 172.21.15.121:0/1029123918 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x559b4d615830 0x559b4d61a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.743 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.738+0000 7ff3a4c63740 1 --2- 172.21.15.121:0/1029123918 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559b4d63c670 0x559b4d619c80 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.744 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.738+0000 7ff3a4c63740 1 --2- 172.21.15.121:0/1029123918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b4d5f26f0 0x559b4d640b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.744 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.738+0000 7ff3a4c63740 1 -- 172.21.15.121:0/1029123918 >> 172.21.15.121:0/1029123918 conn(0x559b4d5eeb90 msgr2=0x559b4d639780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:02.744 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.738+0000 7ff37fe00640 0 client.5757 ms_handle_reset on v2:172.21.15.121:6836/257724942 2024-04-20T10:23:02.744 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.738+0000 7ff3a4c63740 1 -- 172.21.15.121:0/1029123918 shutdown_connections 2024-04-20T10:23:02.744 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.738+0000 7ff3a4c63740 1 -- 172.21.15.121:0/1029123918 wait complete. 2024-04-20T10:23:02.744 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.738+0000 7ff3a4c63740 20 client.5757 trim_cache size 0 max 16384 2024-04-20T10:23:02.758 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.758+0000 7ff3a4c63740 1 Processor -- start 2024-04-20T10:23:02.758 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.758+0000 7ff3a4c63740 1 -- start start 2024-04-20T10:23:02.759 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.759+0000 7ff3a4c63740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x559b4d5f26f0 0x559b4d619c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:02.759 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.759+0000 7ff3a4c63740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559b4d615830 0x559b4d63f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:02.759 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.759+0000 7ff3a4c63740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b4d63c670 0x559b4d65e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:02.760 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.759+0000 7ff3a4c63740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7ff39c1ffeb0 con 0x559b4d615830 2024-04-20T10:23:02.760 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.759+0000 7ff3a4c63740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff39c0ca780 con 0x559b4d63c670 2024-04-20T10:23:02.760 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.759+0000 7ff3a4c63740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7ff39c0cadd0 con 0x559b4d5f26f0 2024-04-20T10:23:02.760 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.759+0000 7ff392a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559b4d615830 0x559b4d63f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:02.760 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.760+0000 7ff392a00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559b4d615830 0x559b4d63f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33336/0 (socket says 172.21.15.121:33336) 2024-04-20T10:23:02.760 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.760+0000 7ff392a00640 1 -- 172.21.15.121:0/2277471788 learned_addr learned my addr 172.21.15.121:0/2277471788 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:23:02.760 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.760+0000 7ff39b400640 1 --2- 172.21.15.121:0/2277471788 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x559b4d5f26f0 0x559b4d619c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:02.760 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.760+0000 7ff39be00640 1 --2- 172.21.15.121:0/2277471788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b4d63c670 0x559b4d65e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:02.760 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.760+0000 7ff392a00640 1 -- 172.21.15.121:0/2277471788 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x559b4d5f26f0 msgr2=0x559b4d619c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:23:02.760 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.760+0000 7ff392a00640 1 --2- 172.21.15.121:0/2277471788 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x559b4d5f26f0 0x559b4d619c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.760 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.760+0000 7ff392a00640 1 -- 172.21.15.121:0/2277471788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b4d63c670 msgr2=0x559b4d65e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:23:02.760 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.760+0000 7ff392a00640 1 --2- 172.21.15.121:0/2277471788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b4d63c670 0x559b4d65e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.760 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.760+0000 7ff392a00640 1 -- 172.21.15.121:0/2277471788 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b4d6406a0 con 0x559b4d615830 2024-04-20T10:23:02.761 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.760+0000 7ff39be00640 1 --2- 172.21.15.121:0/2277471788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b4d63c670 0x559b4d65e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:23:02.761 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.760+0000 7ff392a00640 1 --2- 172.21.15.121:0/2277471788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559b4d615830 0x559b4d63f450 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff3840319d0 tx=0x7ff384049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f8cf3737f533e5f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:02.762 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.760+0000 7ff38aa00640 1 -- 172.21.15.121:0/2277471788 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38405e070 con 0x559b4d615830 2024-04-20T10:23:02.762 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.760+0000 7ff38aa00640 1 -- 172.21.15.121:0/2277471788 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3840457a0 con 0x559b4d615830 2024-04-20T10:23:02.762 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.760+0000 7ff38aa00640 1 -- 172.21.15.121:0/2277471788 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384035040 con 0x559b4d615830 2024-04-20T10:23:02.762 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.761+0000 7ff3a4c63740 1 -- 172.21.15.121:0/2277471788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559b4d615830 msgr2=0x559b4d63f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:02.762 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.761+0000 7ff3a4c63740 1 --2- 172.21.15.121:0/2277471788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559b4d615830 0x559b4d63f450 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff3840319d0 tx=0x7ff384049000 comp rx=0 tx=0).stop 2024-04-20T10:23:02.762 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.761+0000 7ff3a4c63740 1 -- 172.21.15.121:0/2277471788 shutdown_connections 2024-04-20T10:23:02.762 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.761+0000 7ff3a4c63740 1 --2- 172.21.15.121:0/2277471788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b4d63c670 0x559b4d65e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.762 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.761+0000 7ff3a4c63740 1 --2- 172.21.15.121:0/2277471788 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559b4d615830 0x559b4d63f450 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.763 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.761+0000 7ff3a4c63740 1 --2- 172.21.15.121:0/2277471788 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x559b4d5f26f0 0x559b4d619c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.763 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.761+0000 7ff3a4c63740 1 -- 172.21.15.121:0/2277471788 >> 172.21.15.121:0/2277471788 conn(0x559b4d5eeb90 msgr2=0x559b4d6418a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:02.763 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.761+0000 7ff3a4c63740 1 -- 172.21.15.121:0/2277471788 shutdown_connections 2024-04-20T10:23:02.763 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.761+0000 7ff3a4c63740 1 -- 172.21.15.121:0/2277471788 wait complete. 2024-04-20T10:23:02.763 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.762+0000 7ff3a4c63740 1 -- start start 2024-04-20T10:23:02.763 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.762+0000 7ff3a4c63740 1 Processor -- start 2024-04-20T10:23:02.766 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.763+0000 7ff3a4c63740 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x559b4d5f26f0 0x559b4d5f4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:02.766 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.763+0000 7ff3a4c63740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b4d615830 0x559b4d6bca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:02.766 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.763+0000 7ff3a4c63740 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559b4d63c670 0x559b4d6bcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:02.766 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.763+0000 7ff3a4c63740 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x559b4d64abf0 con 0x559b4d63c670 2024-04-20T10:23:02.766 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.763+0000 7ff3a4c63740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559b4d64a720 con 0x559b4d615830 2024-04-20T10:23:02.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.763+0000 7ff3a4c63740 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x559b4d63e050 con 0x559b4d5f26f0 2024-04-20T10:23:02.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.763+0000 7ff3a4c63740 10 client.0 _ll_register_callbacks cb 0x7ff3800036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:23:02.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.763+0000 7ff39be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559b4d63c670 0x559b4d6bcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:02.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.763+0000 7ff39be00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559b4d63c670 0x559b4d6bcfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33348/0 (socket says 172.21.15.121:33348) 2024-04-20T10:23:02.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.763+0000 7ff39be00640 1 -- 172.21.15.121:0/1397212494 learned_addr learned my addr 172.21.15.121:0/1397212494 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:23:02.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.764+0000 7ff39b400640 1 --2- 172.21.15.121:0/1397212494 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x559b4d5f26f0 0x559b4d5f4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:02.767 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.764+0000 7ff39be00640 1 -- 172.21.15.121:0/1397212494 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x559b4d5f26f0 msgr2=0x559b4d5f4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:23:02.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.764+0000 7ff392a00640 1 --2- 172.21.15.121:0/1397212494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b4d615830 0x559b4d6bca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:02.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.764+0000 7ff39be00640 1 --2- 172.21.15.121:0/1397212494 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x559b4d5f26f0 0x559b4d5f4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.764+0000 7ff39be00640 1 -- 172.21.15.121:0/1397212494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b4d615830 msgr2=0x559b4d6bca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:23:02.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.764+0000 7ff39be00640 1 --2- 172.21.15.121:0/1397212494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b4d615830 0x559b4d6bca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.764+0000 7ff39be00640 1 -- 172.21.15.121:0/1397212494 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b4d5f50f0 con 0x559b4d63c670 2024-04-20T10:23:02.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.764+0000 7ff39be00640 1 --2- 172.21.15.121:0/1397212494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559b4d63c670 0x559b4d6bcfd0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ff39400df00 tx=0x7ff39400f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e317e800e0a50cb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:02.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.764+0000 7ff37fe00640 10 client.0 ms_handle_connect on v2:172.21.15.121:3300/0 2024-04-20T10:23:02.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.764+0000 7ff37fe00640 1 -- 172.21.15.121:0/1397212494 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff394012630 con 0x559b4d63c670 2024-04-20T10:23:02.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.764+0000 7ff3a4c63740 1 -- 172.21.15.121:0/1397212494 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3840457a0 con 0x559b4d63c670 2024-04-20T10:23:02.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.764+0000 7ff37fe00640 1 -- 172.21.15.121:0/1397212494 <== mon.0 v2:172.21.15.121:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7ff39401cc40 con 0x559b4d63c670 2024-04-20T10:23:02.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.765+0000 7ff37fe00640 1 -- 172.21.15.121:0/1397212494 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7ff39401ce40 con 0x559b4d63c670 2024-04-20T10:23:02.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.765+0000 7ff3a4c63740 10 client.5763 fetch_fsmap learned FSMap version 17 2024-04-20T10:23:02.768 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.765+0000 7ff3a4c63740 1 -- 172.21.15.121:0/1397212494 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x559b4d65e700 con 0x559b4d63c670 2024-04-20T10:23:02.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.765+0000 7ff37fe00640 1 -- 172.21.15.121:0/1397212494 <== mon.0 v2:172.21.15.121:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3840457a0 con 0x559b4d63c670 2024-04-20T10:23:02.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.765+0000 7ff37fe00640 1 -- 172.21.15.121:0/1397212494 <== mon.0 v2:172.21.15.121:3300/0 5 ==== fsmap(e 17) ==== 3495+0+0 (secure 0 0 0) 0x7ff3940127d0 con 0x559b4d63c670 2024-04-20T10:23:02.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.765+0000 7ff3a4c63740 10 client.5763 fetch_fsmap finished waiting for FSMap version 17 2024-04-20T10:23:02.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.765+0000 7ff3a4c63740 10 client.5763 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-20T10:23:02.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.765+0000 7ff3a4c63740 20 client.5763 populate_metadata read hostname 'smithi121' 2024-04-20T10:23:02.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.765+0000 7ff3a4c63740 1 --2- 172.21.15.121:0/1397212494 >> [v2:172.21.15.121:6836/257724942,v1:172.21.15.121:6837/257724942] conn(0x559b4d6bdc00 0x559b4d6bdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:02.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.765+0000 7ff3a4c63740 4 client.5763 mds_command: new command op to 5568 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T10:23:02.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.765+0000 7ff3a4c63740 1 -- 172.21.15.121:0/1397212494 --> [v2:172.21.15.121:6836/257724942,v1:172.21.15.121:6837/257724942] -- command(tid 0: {"prefix": "status"}) -- 0x559b4d562260 con 0x559b4d6bdc00 2024-04-20T10:23:02.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.765+0000 7ff39b400640 1 --2- 172.21.15.121:0/1397212494 >> [v2:172.21.15.121:6836/257724942,v1:172.21.15.121:6837/257724942] conn(0x559b4d6bdc00 0x559b4d6bdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:02.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.766+0000 7ff39b400640 1 --2- 172.21.15.121:0/1397212494 >> [v2:172.21.15.121:6836/257724942,v1:172.21.15.121:6837/257724942] conn(0x559b4d6bdc00 0x559b4d6bdfe0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd511eb97c2f41a0 server_cookie=94e9c6c8ac4e73bf in_seq=0 out_seq=0 2024-04-20T10:23:02.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.766+0000 7ff37fe00640 10 client.5763 ms_handle_connect on v2:172.21.15.121:6836/257724942 2024-04-20T10:23:02.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.766+0000 7ff37fe00640 1 -- 172.21.15.121:0/1397212494 <== mds.0 v2:172.21.15.121:6836/257724942 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x559b4d562260 con 0x559b4d6bdc00 2024-04-20T10:23:02.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.766+0000 7ff37fe00640 10 client.5763 handle_command_reply: tid=0 2024-04-20T10:23:02.769 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.766+0000 7ff3a4c63740 1 client.5763 shutdown 2024-04-20T10:23:02.770 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.766+0000 7ff3a4c63740 1 -- 172.21.15.121:0/1397212494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559b4d63c670 msgr2=0x559b4d6bcfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:02.770 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.766+0000 7ff3a4c63740 1 --2- 172.21.15.121:0/1397212494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559b4d63c670 0x559b4d6bcfd0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ff39400df00 tx=0x7ff39400f670 comp rx=0 tx=0).stop 2024-04-20T10:23:02.770 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.766+0000 7ff3a4c63740 1 -- 172.21.15.121:0/1397212494 shutdown_connections 2024-04-20T10:23:02.770 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.766+0000 7ff3a4c63740 1 --2- 172.21.15.121:0/1397212494 >> [v2:172.21.15.121:6836/257724942,v1:172.21.15.121:6837/257724942] conn(0x559b4d6bdc00 0x559b4d6bdfe0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.770 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.766+0000 7ff3a4c63740 1 --2- 172.21.15.121:0/1397212494 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x559b4d63c670 0x559b4d6bcfd0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.770 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.766+0000 7ff3a4c63740 1 --2- 172.21.15.121:0/1397212494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b4d615830 0x559b4d6bca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.770 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.766+0000 7ff3a4c63740 1 --2- 172.21.15.121:0/1397212494 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x559b4d5f26f0 0x559b4d5f4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.770 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.766+0000 7ff3a4c63740 1 -- 172.21.15.121:0/1397212494 >> 172.21.15.121:0/1397212494 conn(0x559b4d5eeb90 msgr2=0x559b4d5edb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:02.770 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.766+0000 7ff37fe00640 0 client.5763 ms_handle_reset on v2:172.21.15.121:6836/257724942 2024-04-20T10:23:02.770 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.767+0000 7ff3a4c63740 1 -- 172.21.15.121:0/1397212494 shutdown_connections 2024-04-20T10:23:02.770 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.767+0000 7ff3a4c63740 1 -- 172.21.15.121:0/1397212494 wait complete. 2024-04-20T10:23:02.770 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.767+0000 7ff3a4c63740 20 client.5763 trim_cache size 0 max 16384 2024-04-20T10:23:02.771 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T10:23:02.771 INFO:teuthology.orchestra.run.smithi121.stdout: "cluster_fsid": "ceaac016-4ab2-472d-a36e-7bfe9497c762", 2024-04-20T10:23:02.771 INFO:teuthology.orchestra.run.smithi121.stdout: "whoami": 0, 2024-04-20T10:23:02.771 INFO:teuthology.orchestra.run.smithi121.stdout: "id": 5568, 2024-04-20T10:23:02.771 INFO:teuthology.orchestra.run.smithi121.stdout: "want_state": "up:active", 2024-04-20T10:23:02.771 INFO:teuthology.orchestra.run.smithi121.stdout: "state": "up:active", 2024-04-20T10:23:02.771 INFO:teuthology.orchestra.run.smithi121.stdout: "fs_name": "cephfs", 2024-04-20T10:23:02.771 INFO:teuthology.orchestra.run.smithi121.stdout: "rank_uptime": 11.591756613999999, 2024-04-20T10:23:02.771 INFO:teuthology.orchestra.run.smithi121.stdout: "mdsmap_epoch": 17, 2024-04-20T10:23:02.771 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch": 39, 2024-04-20T10:23:02.771 INFO:teuthology.orchestra.run.smithi121.stdout: "osdmap_epoch_barrier": 39, 2024-04-20T10:23:02.772 INFO:teuthology.orchestra.run.smithi121.stdout: "uptime": 19.910086841999998 2024-04-20T10:23:02.772 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T10:23:02.772 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.767+0000 7ff3a0a00640 1 -- 172.21.15.121:0/2553919519 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff37806c8f0 msgr2=0x7ff37806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:02.772 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.767+0000 7ff3a0a00640 1 --2- 172.21.15.121:0/2553919519 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff37806c8f0 0x7ff37806edb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff384036000 tx=0x7ff384008030 comp rx=0 tx=0).stop 2024-04-20T10:23:02.772 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.767+0000 7ff3a0a00640 1 -- 172.21.15.121:0/2553919519 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff39c072190 msgr2=0x7ff39c1451c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:02.772 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.767+0000 7ff3a0a00640 1 --2- 172.21.15.121:0/2553919519 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff39c072190 0x7ff39c1451c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ff38c00cb40 tx=0x7ff38c00a060 comp rx=0 tx=0).stop 2024-04-20T10:23:02.772 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.767+0000 7ff3a0a00640 1 -- 172.21.15.121:0/2553919519 shutdown_connections 2024-04-20T10:23:02.773 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.767+0000 7ff3a0a00640 1 --2- 172.21.15.121:0/2553919519 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7ff39c15a1f0 0x7ff39c0b0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.773 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.767+0000 7ff3a0a00640 1 --2- 172.21.15.121:0/2553919519 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7ff37806c8f0 0x7ff37806edb0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.773 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.767+0000 7ff3a0a00640 1 --2- 172.21.15.121:0/2553919519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff39c159e10 0x7ff39c0b0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.773 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.767+0000 7ff3a0a00640 1 --2- 172.21.15.121:0/2553919519 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7ff39c072190 0x7ff39c1451c0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.773 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.767+0000 7ff3a0a00640 1 -- 172.21.15.121:0/2553919519 >> 172.21.15.121:0/2553919519 conn(0x7ff39c0d0040 msgr2=0x7ff39c0b1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:02.773 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.768+0000 7ff3a0a00640 1 -- 172.21.15.121:0/2553919519 shutdown_connections 2024-04-20T10:23:02.773 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.768+0000 7ff3a0a00640 1 -- 172.21.15.121:0/2553919519 wait complete. 2024-04-20T10:23:02.783 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T10:23:02.977 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.977+0000 7f55fd800640 1 Processor -- start 2024-04-20T10:23:02.977 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.977+0000 7f55fd800640 1 -- start start 2024-04-20T10:23:02.979 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.979+0000 7f55fd800640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f55f80b4320 0x7f55f80adf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:02.979 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.979+0000 7f55fd800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f55f806b900 0x7f55f806bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:02.979 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.979+0000 7f55fd800640 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f55f8071a80 con 0x7f55f80b2f70 2024-04-20T10:23:02.979 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.979+0000 7f55fd800640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f55f806c240 con 0x7f55f80b4320 2024-04-20T10:23:02.979 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.979+0000 7f55fd800640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f55f8071cf0 con 0x7f55f806b900 2024-04-20T10:23:02.979 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.979+0000 7f55f7400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f55f80b4320 0x7f55f80adf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:02.979 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.979+0000 7f55f7400640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f55f80b4320 0x7f55f80adf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33364/0 (socket says 172.21.15.121:33364) 2024-04-20T10:23:02.979 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.979+0000 7f55f7400640 1 -- 172.21.15.121:0/949548626 learned_addr learned my addr 172.21.15.121:0/949548626 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:23:02.979 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.979+0000 7f55fce00640 1 --2- 172.21.15.121:0/949548626 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f55f806b900 0x7f55f806bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:02.979 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.979+0000 7f55f7400640 1 -- 172.21.15.121:0/949548626 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f55f806b900 msgr2=0x7f55f806bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:23:02.979 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.979+0000 7f55f7400640 1 --2- 172.21.15.121:0/949548626 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f55f806b900 0x7f55f806bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.980 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.979+0000 7f55f7400640 1 -- 172.21.15.121:0/949548626 >> v1:172.21.15.123:6789/0 conn(0x7f55f80b2f70 legacy=0x7f55f815f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:23:02.980 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.980+0000 7f55f7400640 1 -- 172.21.15.121:0/949548626 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f8071fd0 con 0x7f55f80b4320 2024-04-20T10:23:02.980 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.981+0000 7f55f7400640 1 --2- 172.21.15.121:0/949548626 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55f80b4320 0x7f55f80adf60 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f55e4009870 tx=0x7f55e402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f563ba08f9ccad server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:02.980 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.981+0000 7f55f6a00640 1 -- 172.21.15.121:0/949548626 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e4035020 con 0x7f55f80b4320 2024-04-20T10:23:02.980 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.981+0000 7f55f6a00640 1 -- 172.21.15.121:0/949548626 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55e4031640 con 0x7f55f80b4320 2024-04-20T10:23:02.980 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.981+0000 7f55f6a00640 1 -- 172.21.15.121:0/949548626 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e402d040 con 0x7f55f80b4320 2024-04-20T10:23:02.981 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.982+0000 7f55fd800640 1 -- 172.21.15.121:0/949548626 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55f80b4320 msgr2=0x7f55f80adf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:02.981 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.982+0000 7f55fd800640 1 --2- 172.21.15.121:0/949548626 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55f80b4320 0x7f55f80adf60 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f55e4009870 tx=0x7f55e402b8f0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.981 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.982+0000 7f55fd800640 1 -- 172.21.15.121:0/949548626 shutdown_connections 2024-04-20T10:23:02.982 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.982+0000 7f55fd800640 1 --2- 172.21.15.121:0/949548626 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f55f806b900 0x7f55f806bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.982 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.982+0000 7f55fd800640 1 --2- 172.21.15.121:0/949548626 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55f80b4320 0x7f55f80adf60 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.982 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.982+0000 7f55fd800640 1 -- 172.21.15.121:0/949548626 >> 172.21.15.121:0/949548626 conn(0x7f55f80d0040 msgr2=0x7f55f80d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:02.982 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.982+0000 7f55fd800640 1 -- 172.21.15.121:0/949548626 shutdown_connections 2024-04-20T10:23:02.982 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.982+0000 7f55fd800640 1 -- 172.21.15.121:0/949548626 wait complete. 2024-04-20T10:23:02.983 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.984+0000 7f55fd800640 1 Processor -- start 2024-04-20T10:23:02.983 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.984+0000 7f55fd800640 1 -- start start 2024-04-20T10:23:02.984 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.985+0000 7f55fd800640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55f806b900 0x7f55f81ee700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:02.984 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.985+0000 7f55fd800640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f55f80b2f70 0x7f55f815b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:02.984 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.985+0000 7f55fd800640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55f80b4320 0x7f55f809c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:02.984 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.985+0000 7f55fd800640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f55f815fcc0 con 0x7f55f806b900 2024-04-20T10:23:02.985 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.985+0000 7f55fd800640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f55f80736c0 con 0x7f55f80b4320 2024-04-20T10:23:02.985 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.985+0000 7f55fd800640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f55f815fe40 con 0x7f55f80b2f70 2024-04-20T10:23:02.985 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.985+0000 7f55f7e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55f806b900 0x7f55f81ee700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:02.985 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.985+0000 7f55f7e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55f806b900 0x7f55f81ee700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:33372/0 (socket says 172.21.15.121:33372) 2024-04-20T10:23:02.985 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.985+0000 7f55f7e00640 1 -- 172.21.15.121:0/3625238464 learned_addr learned my addr 172.21.15.121:0/3625238464 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:23:02.986 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.985+0000 7f55f7400640 1 --2- 172.21.15.121:0/3625238464 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f55f80b2f70 0x7f55f815b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:02.986 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.985+0000 7f55f7e00640 1 -- 172.21.15.121:0/3625238464 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f55f80b2f70 msgr2=0x7f55f815b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:02.986 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.985+0000 7f55f7e00640 1 --2- 172.21.15.121:0/3625238464 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f55f80b2f70 0x7f55f815b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.986 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.985+0000 7f55f7e00640 1 -- 172.21.15.121:0/3625238464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55f80b4320 msgr2=0x7f55f809c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:02.986 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.985+0000 7f55f7e00640 1 --2- 172.21.15.121:0/3625238464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55f80b4320 0x7f55f809c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:02.986 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.985+0000 7f55f7e00640 1 -- 172.21.15.121:0/3625238464 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55f8159750 con 0x7f55f806b900 2024-04-20T10:23:02.987 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.986+0000 7f55f7e00640 1 --2- 172.21.15.121:0/3625238464 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55f806b900 0x7f55f81ee700 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f55e800c430 tx=0x7f55e800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:02.987 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.986+0000 7f55efe00640 1 -- 172.21.15.121:0/3625238464 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e8011790 con 0x7f55f806b900 2024-04-20T10:23:02.987 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.986+0000 7f55efe00640 1 -- 172.21.15.121:0/3625238464 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55e8017070 con 0x7f55f806b900 2024-04-20T10:23:02.987 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.986+0000 7f55efe00640 1 -- 172.21.15.121:0/3625238464 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55e8011bb0 con 0x7f55f806b900 2024-04-20T10:23:02.987 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.986+0000 7f55fd800640 1 -- 172.21.15.121:0/3625238464 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55f8159a30 con 0x7f55f806b900 2024-04-20T10:23:02.987 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.986+0000 7f55fd800640 1 -- 172.21.15.121:0/3625238464 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55f8159ff0 con 0x7f55f806b900 2024-04-20T10:23:02.987 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.987+0000 7f55fd800640 1 -- 172.21.15.121:0/3625238464 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55f8075b60 con 0x7f55f806b900 2024-04-20T10:23:02.988 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.988+0000 7f55efe00640 1 -- 172.21.15.121:0/3625238464 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f55e801c460 con 0x7f55f806b900 2024-04-20T10:23:02.989 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.989+0000 7f55efe00640 1 --2- 172.21.15.121:0/3625238464 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f55d806c8f0 0x7f55d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:23:02.989 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.989+0000 7f55efe00640 1 -- 172.21.15.121:0/3625238464 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f55e808bb90 con 0x7f55f806b900 2024-04-20T10:23:02.989 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.990+0000 7f55f7400640 1 --2- 172.21.15.121:0/3625238464 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f55d806c8f0 0x7f55d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:23:02.990 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.990+0000 7f55f7400640 1 --2- 172.21.15.121:0/3625238464 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f55d806c8f0 0x7f55d806edb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f55e4036000 tx=0x7f55e4008030 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:23:02.994 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:02.994+0000 7f55efe00640 1 -- 172.21.15.121:0/3625238464 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f55e8059e20 con 0x7f55f806b900 2024-04-20T10:23:03.238 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:03.238+0000 7f55fd800640 1 -- 172.21.15.121:0/3625238464 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f55f80d04c0 con 0x7f55f806b900 2024-04-20T10:23:03.238 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:03.239+0000 7f55efe00640 1 -- 172.21.15.121:0/3625238464 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f55e8059720 con 0x7f55f806b900 2024-04-20T10:23:03.239 INFO:teuthology.orchestra.run.smithi121.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T10:23:03.241 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:03.241+0000 7f55fd800640 1 -- 172.21.15.121:0/3625238464 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f55d806c8f0 msgr2=0x7f55d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:03.241 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:03.241+0000 7f55fd800640 1 --2- 172.21.15.121:0/3625238464 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f55d806c8f0 0x7f55d806edb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f55e4036000 tx=0x7f55e4008030 comp rx=0 tx=0).stop 2024-04-20T10:23:03.241 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:03.241+0000 7f55fd800640 1 -- 172.21.15.121:0/3625238464 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55f806b900 msgr2=0x7f55f81ee700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:23:03.241 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:03.241+0000 7f55fd800640 1 --2- 172.21.15.121:0/3625238464 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55f806b900 0x7f55f81ee700 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f55e800c430 tx=0x7f55e800c8f0 comp rx=0 tx=0).stop 2024-04-20T10:23:03.241 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:03.241+0000 7f55fd800640 1 -- 172.21.15.121:0/3625238464 shutdown_connections 2024-04-20T10:23:03.242 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:03.241+0000 7f55fd800640 1 --2- 172.21.15.121:0/3625238464 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f55d806c8f0 0x7f55d806edb0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:03.242 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:03.241+0000 7f55fd800640 1 --2- 172.21.15.121:0/3625238464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55f80b4320 0x7f55f809c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:03.242 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:03.241+0000 7f55fd800640 1 --2- 172.21.15.121:0/3625238464 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f55f80b2f70 0x7f55f815b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:03.242 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:03.241+0000 7f55fd800640 1 --2- 172.21.15.121:0/3625238464 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f55f806b900 0x7f55f81ee700 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:23:03.242 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:03.242+0000 7f55fd800640 1 -- 172.21.15.121:0/3625238464 >> 172.21.15.121:0/3625238464 conn(0x7f55f80d0040 msgr2=0x7f55f80b0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:23:03.242 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:03.242+0000 7f55fd800640 1 -- 172.21.15.121:0/3625238464 shutdown_connections 2024-04-20T10:23:03.242 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:03.242+0000 7f55fd800640 1 -- 172.21.15.121:0/3625238464 wait complete. 2024-04-20T10:23:03.260 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T10:23:03.260 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T10:23:03.260 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi121.front.sepia.ceph.com 2024-04-20T10:23:03.260 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T10:23:03.260 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T10:23:03.260 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T10:23:03.260 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T10:23:03.260 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T10:23:03.260 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T10:23:03.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:23:03.261 DEBUG:teuthology.orchestra.run.smithi121:> ip addr 2024-04-20T10:23:03.308 INFO:teuthology.orchestra.run.smithi121.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T10:23:03.308 INFO:teuthology.orchestra.run.smithi121.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T10:23:03.308 INFO:teuthology.orchestra.run.smithi121.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T10:23:03.309 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:23:03.309 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 ::1/128 scope host 2024-04-20T10:23:03.309 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:23:03.309 INFO:teuthology.orchestra.run.smithi121.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:23:03.309 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:82 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:23:03.309 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f0 2024-04-20T10:23:03.309 INFO:teuthology.orchestra.run.smithi121.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:23:03.309 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:d8:a4:83 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:23:03.309 INFO:teuthology.orchestra.run.smithi121.stdout: altname enp7s0f1 2024-04-20T10:23:03.309 INFO:teuthology.orchestra.run.smithi121.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:23:03.309 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a2 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:23:03.309 INFO:teuthology.orchestra.run.smithi121.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T10:23:03.309 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 0c:c4:7a:8f:cb:a3 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:23:03.309 INFO:teuthology.orchestra.run.smithi121.stdout: inet 172.21.15.121/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T10:23:03.310 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft 42525sec preferred_lft 42525sec 2024-04-20T10:23:03.310 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::ec4:7aff:fe8f:cba3/64 scope link 2024-04-20T10:23:03.310 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:23:03.310 INFO:teuthology.orchestra.run.smithi121.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T10:23:03.310 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 2a:73:a9:6c:a9:c5 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:23:03.310 INFO:teuthology.orchestra.run.smithi121.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T10:23:03.310 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:23:03.310 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::30ec:1cff:fe04:5f23/64 scope link 2024-04-20T10:23:03.310 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:23:03.310 INFO:teuthology.orchestra.run.smithi121.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T10:23:03.310 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether 2a:73:a9:6c:a9:c5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:23:03.310 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::2873:a9ff:fe6c:a9c5/64 scope link 2024-04-20T10:23:03.310 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:23:03.311 INFO:teuthology.orchestra.run.smithi121.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T10:23:03.311 INFO:teuthology.orchestra.run.smithi121.stdout: link/ether ce:9b:c8:da:0b:77 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T10:23:03.311 INFO:teuthology.orchestra.run.smithi121.stdout: inet6 fe80::cc9b:c8ff:feda:b77/64 scope link 2024-04-20T10:23:03.311 INFO:teuthology.orchestra.run.smithi121.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:23:03.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:23:03.311 DEBUG:teuthology.orchestra.run.smithi121:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T10:23:03.375 INFO:teuthology.orchestra.run.smithi121.stdout:1 2024-04-20T10:23:03.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:23:03.379 DEBUG:teuthology.orchestra.run.smithi121:> route 2024-04-20T10:23:03.426 INFO:teuthology.orchestra.run.smithi121.stdout:Kernel IP routing table 2024-04-20T10:23:03.426 INFO:teuthology.orchestra.run.smithi121.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T10:23:03.426 INFO:teuthology.orchestra.run.smithi121.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T10:23:03.426 INFO:teuthology.orchestra.run.smithi121.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T10:23:03.426 INFO:teuthology.orchestra.run.smithi121.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T10:23:03.427 DEBUG:teuthology.orchestra.run.smithi121:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:23:03.427 DEBUG:teuthology.orchestra.run.smithi121:> set -e 2024-04-20T10:23:03.427 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T10:23:03.427 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T10:23:03.427 DEBUG:teuthology.orchestra.run.smithi121:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T10:23:03.427 DEBUG:teuthology.orchestra.run.smithi121:> ') 2024-04-20T10:23:03.525 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:23:03.620 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:23:03.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:23:03.630 DEBUG:teuthology.orchestra.run.smithi121:> ip netns list 2024-04-20T10:23:03.674 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T10:23:03.674 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-20T10:23:03.675 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T10:23:03.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:23:03.675 DEBUG:teuthology.orchestra.run.smithi121:> sudo ip link set brx.0 up 2024-04-20T10:23:03.801 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-20T10:23:03.801 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T10:23:03.801 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T10:23:03.855 INFO:teuthology.orchestra.run.smithi121.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T10:23:03.856 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T10:23:03.856 DEBUG:teuthology.orchestra.run.smithi121:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T10:23:03.904 DEBUG:teuthology.orchestra.run.smithi121:> sudo modprobe fuse 2024-04-20T10:23:03.993 DEBUG:teuthology.orchestra.run.smithi121:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T10:23:04.049 INFO:teuthology.orchestra.run.smithi121.stdout:/proc 2024-04-20T10:23:04.050 INFO:teuthology.orchestra.run.smithi121.stdout:/sys 2024-04-20T10:23:04.050 INFO:teuthology.orchestra.run.smithi121.stdout:/dev 2024-04-20T10:23:04.050 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/security 2024-04-20T10:23:04.050 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/shm 2024-04-20T10:23:04.050 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/pts 2024-04-20T10:23:04.050 INFO:teuthology.orchestra.run.smithi121.stdout:/run 2024-04-20T10:23:04.050 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/cgroup 2024-04-20T10:23:04.050 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/pstore 2024-04-20T10:23:04.050 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/bpf 2024-04-20T10:23:04.050 INFO:teuthology.orchestra.run.smithi121.stdout:/ 2024-04-20T10:23:04.050 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/selinux 2024-04-20T10:23:04.050 INFO:teuthology.orchestra.run.smithi121.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T10:23:04.051 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/hugepages 2024-04-20T10:23:04.051 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/mqueue 2024-04-20T10:23:04.051 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/debug 2024-04-20T10:23:04.051 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/tracing 2024-04-20T10:23:04.051 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/fuse/connections 2024-04-20T10:23:04.051 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/config 2024-04-20T10:23:04.051 INFO:teuthology.orchestra.run.smithi121.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T10:23:04.051 INFO:teuthology.orchestra.run.smithi121.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T10:23:04.051 INFO:teuthology.orchestra.run.smithi121.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T10:23:04.051 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T10:23:04.051 INFO:teuthology.orchestra.run.smithi121.stdout:/run/user/1000 2024-04-20T10:23:04.051 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph 2024-04-20T10:23:04.052 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T10:23:04.052 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T10:23:04.052 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T10:23:04.052 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T10:23:04.052 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns 2024-04-20T10:23:04.052 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:23:04.052 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:23:04.052 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T10:23:04.052 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T10:23:04.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:23:04.053 DEBUG:teuthology.orchestra.run.smithi121:> ls /sys/fs/fuse/connections 2024-04-20T10:23:04.103 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T10:23:04.103 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-20T10:23:04.108 DEBUG:teuthology.orchestra.run.smithi121:> sudo modprobe fuse 2024-04-20T10:23:04.174 DEBUG:teuthology.orchestra.run.smithi121:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T10:23:04.214 INFO:teuthology.orchestra.run.smithi121.stdout:/proc 2024-04-20T10:23:04.215 INFO:teuthology.orchestra.run.smithi121.stdout:/sys 2024-04-20T10:23:04.215 INFO:teuthology.orchestra.run.smithi121.stdout:/dev 2024-04-20T10:23:04.215 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/security 2024-04-20T10:23:04.215 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/shm 2024-04-20T10:23:04.215 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/pts 2024-04-20T10:23:04.215 INFO:teuthology.orchestra.run.smithi121.stdout:/run 2024-04-20T10:23:04.215 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/cgroup 2024-04-20T10:23:04.215 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/pstore 2024-04-20T10:23:04.215 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/bpf 2024-04-20T10:23:04.215 INFO:teuthology.orchestra.run.smithi121.stdout:/ 2024-04-20T10:23:04.215 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/selinux 2024-04-20T10:23:04.215 INFO:teuthology.orchestra.run.smithi121.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T10:23:04.216 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/hugepages 2024-04-20T10:23:04.216 INFO:teuthology.orchestra.run.smithi121.stdout:/dev/mqueue 2024-04-20T10:23:04.216 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/debug 2024-04-20T10:23:04.216 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/tracing 2024-04-20T10:23:04.216 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/fs/fuse/connections 2024-04-20T10:23:04.216 INFO:teuthology.orchestra.run.smithi121.stdout:/sys/kernel/config 2024-04-20T10:23:04.216 INFO:teuthology.orchestra.run.smithi121.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T10:23:04.216 INFO:teuthology.orchestra.run.smithi121.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T10:23:04.216 INFO:teuthology.orchestra.run.smithi121.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T10:23:04.216 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T10:23:04.216 INFO:teuthology.orchestra.run.smithi121.stdout:/run/user/1000 2024-04-20T10:23:04.216 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph 2024-04-20T10:23:04.216 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T10:23:04.216 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T10:23:04.216 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T10:23:04.217 INFO:teuthology.orchestra.run.smithi121.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T10:23:04.217 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns 2024-04-20T10:23:04.217 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:23:04.217 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:23:04.217 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T10:23:04.217 INFO:teuthology.orchestra.run.smithi121.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T10:23:04.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:23:04.218 DEBUG:teuthology.orchestra.run.smithi121:> ls /sys/fs/fuse/connections 2024-04-20T10:23:04.280 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi121.stderr:2024-04-20T10:23:04.280+0000 7f03577ee480 -1 init, newargv = 0x555dc2dfef10 newargc=15 2024-04-20T10:23:04.280 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi121.stderr:ceph-fuse[39624]: starting ceph client 2024-04-20T10:23:04.289 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi121.stderr:ceph-fuse[39624]: starting fuse 2024-04-20T10:23:04.301 INFO:teuthology.orchestra.run.smithi121.stdout:41 2024-04-20T10:23:04.302 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-20T10:23:04.302 DEBUG:teuthology.orchestra.run.smithi121:> sudo stdin-killer -- python3 -c ' 2024-04-20T10:23:04.302 DEBUG:teuthology.orchestra.run.smithi121:> import glob 2024-04-20T10:23:04.302 DEBUG:teuthology.orchestra.run.smithi121:> import re 2024-04-20T10:23:04.302 DEBUG:teuthology.orchestra.run.smithi121:> import os 2024-04-20T10:23:04.302 DEBUG:teuthology.orchestra.run.smithi121:> import subprocess 2024-04-20T10:23:04.302 DEBUG:teuthology.orchestra.run.smithi121:> 2024-04-20T10:23:04.302 DEBUG:teuthology.orchestra.run.smithi121:> def _find_admin_socket(client_name): 2024-04-20T10:23:04.303 DEBUG:teuthology.orchestra.run.smithi121:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-20T10:23:04.303 DEBUG:teuthology.orchestra.run.smithi121:> files = glob.glob(asok_path) 2024-04-20T10:23:04.303 DEBUG:teuthology.orchestra.run.smithi121:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-20T10:23:04.303 DEBUG:teuthology.orchestra.run.smithi121:> 2024-04-20T10:23:04.303 DEBUG:teuthology.orchestra.run.smithi121:> # Given a non-glob path, it better be there 2024-04-20T10:23:04.303 DEBUG:teuthology.orchestra.run.smithi121:> if "*" not in asok_path: 2024-04-20T10:23:04.303 DEBUG:teuthology.orchestra.run.smithi121:> assert(len(files) == 1) 2024-04-20T10:23:04.303 DEBUG:teuthology.orchestra.run.smithi121:> return files[0] 2024-04-20T10:23:04.303 DEBUG:teuthology.orchestra.run.smithi121:> 2024-04-20T10:23:04.303 DEBUG:teuthology.orchestra.run.smithi121:> for f in files: 2024-04-20T10:23:04.303 DEBUG:teuthology.orchestra.run.smithi121:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T10:23:04.303 DEBUG:teuthology.orchestra.run.smithi121:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T10:23:04.303 DEBUG:teuthology.orchestra.run.smithi121:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T10:23:04.303 DEBUG:teuthology.orchestra.run.smithi121:> contents = proc_f.read() 2024-04-20T10:23:04.303 DEBUG:teuthology.orchestra.run.smithi121:> if mountpoint in contents: 2024-04-20T10:23:04.303 DEBUG:teuthology.orchestra.run.smithi121:> return f 2024-04-20T10:23:04.303 DEBUG:teuthology.orchestra.run.smithi121:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T10:23:04.304 DEBUG:teuthology.orchestra.run.smithi121:> 2024-04-20T10:23:04.304 DEBUG:teuthology.orchestra.run.smithi121:> print(_find_admin_socket("client.0")) 2024-04-20T10:23:04.304 DEBUG:teuthology.orchestra.run.smithi121:> ' 2024-04-20T10:23:04.487 INFO:teuthology.orchestra.run.smithi121.stdout:/var/run/ceph/ceph-client.0.39624.asok 2024-04-20T10:23:04.491 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:23:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:23:04.504 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.39624.asok 2024-04-20T10:23:04.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:23:04.504 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.39624.asok status 2024-04-20T10:23:04.729 INFO:teuthology.orchestra.run.smithi121.stdout:{ 2024-04-20T10:23:04.729 INFO:teuthology.orchestra.run.smithi121.stdout: "metadata": { 2024-04-20T10:23:04.729 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T10:23:04.729 INFO:teuthology.orchestra.run.smithi121.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T10:23:04.730 INFO:teuthology.orchestra.run.smithi121.stdout: "entity_id": "0", 2024-04-20T10:23:04.730 INFO:teuthology.orchestra.run.smithi121.stdout: "hostname": "smithi121", 2024-04-20T10:23:04.730 INFO:teuthology.orchestra.run.smithi121.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-20T10:23:04.730 INFO:teuthology.orchestra.run.smithi121.stdout: "pid": "39624", 2024-04-20T10:23:04.730 INFO:teuthology.orchestra.run.smithi121.stdout: "root": "/" 2024-04-20T10:23:04.730 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T10:23:04.730 INFO:teuthology.orchestra.run.smithi121.stdout: "dentry_count": 0, 2024-04-20T10:23:04.730 INFO:teuthology.orchestra.run.smithi121.stdout: "dentry_pinned_count": 0, 2024-04-20T10:23:04.730 INFO:teuthology.orchestra.run.smithi121.stdout: "id": 5775, 2024-04-20T10:23:04.730 INFO:teuthology.orchestra.run.smithi121.stdout: "inst": { 2024-04-20T10:23:04.730 INFO:teuthology.orchestra.run.smithi121.stdout: "name": { 2024-04-20T10:23:04.731 INFO:teuthology.orchestra.run.smithi121.stdout: "type": "client", 2024-04-20T10:23:04.731 INFO:teuthology.orchestra.run.smithi121.stdout: "num": 5775 2024-04-20T10:23:04.731 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T10:23:04.731 INFO:teuthology.orchestra.run.smithi121.stdout: "addr": { 2024-04-20T10:23:04.731 INFO:teuthology.orchestra.run.smithi121.stdout: "type": "v1", 2024-04-20T10:23:04.731 INFO:teuthology.orchestra.run.smithi121.stdout: "addr": "192.168.0.1:0", 2024-04-20T10:23:04.731 INFO:teuthology.orchestra.run.smithi121.stdout: "nonce": 3185499585 2024-04-20T10:23:04.731 INFO:teuthology.orchestra.run.smithi121.stdout: } 2024-04-20T10:23:04.731 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T10:23:04.731 INFO:teuthology.orchestra.run.smithi121.stdout: "addr": { 2024-04-20T10:23:04.731 INFO:teuthology.orchestra.run.smithi121.stdout: "type": "v1", 2024-04-20T10:23:04.731 INFO:teuthology.orchestra.run.smithi121.stdout: "addr": "192.168.0.1:0", 2024-04-20T10:23:04.731 INFO:teuthology.orchestra.run.smithi121.stdout: "nonce": 3185499585 2024-04-20T10:23:04.732 INFO:teuthology.orchestra.run.smithi121.stdout: }, 2024-04-20T10:23:04.732 INFO:teuthology.orchestra.run.smithi121.stdout: "inst_str": "client.5775 192.168.0.1:0/3185499585", 2024-04-20T10:23:04.732 INFO:teuthology.orchestra.run.smithi121.stdout: "addr_str": "192.168.0.1:0/3185499585", 2024-04-20T10:23:04.732 INFO:teuthology.orchestra.run.smithi121.stdout: "inode_count": 1, 2024-04-20T10:23:04.732 INFO:teuthology.orchestra.run.smithi121.stdout: "mds_epoch": 17, 2024-04-20T10:23:04.732 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_epoch": 39, 2024-04-20T10:23:04.732 INFO:teuthology.orchestra.run.smithi121.stdout: "osd_epoch_barrier": 0, 2024-04-20T10:23:04.732 INFO:teuthology.orchestra.run.smithi121.stdout: "blocklisted": false, 2024-04-20T10:23:04.732 INFO:teuthology.orchestra.run.smithi121.stdout: "fs_name": "cephfs" 2024-04-20T10:23:04.732 INFO:teuthology.orchestra.run.smithi121.stdout:} 2024-04-20T10:23:04.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:23:04.744 DEBUG:teuthology.orchestra.run.smithi121:> stat --file-system '--printf=%T 2024-04-20T10:23:04.744 DEBUG:teuthology.orchestra.run.smithi121:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-20T10:23:04.796 INFO:teuthology.orchestra.run.smithi121.stdout:fuseblk 2024-04-20T10:23:04.796 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-20T10:23:04.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:23:04.797 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T10:23:04.873 DEBUG:teuthology.orchestra.run.smithi121:> mktemp -d --suffix=xfstests-dev 2024-04-20T10:23:04.923 INFO:teuthology.orchestra.run.smithi121.stdout:/tmp/tmp.yONpfXTHudxfstests-dev 2024-04-20T10:23:04.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:23:04.924 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo chmod a+rx /tmp/tmp.yONpfXTHudxfstests-dev) 2024-04-20T10:23:05.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:23:05.001 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone https://git.ceph.com/xfstests-dev.git --depth 1 /tmp/tmp.yONpfXTHudxfstests-dev) 2024-04-20T10:23:05.063 INFO:teuthology.orchestra.run.smithi121.stderr:Cloning into '/tmp/tmp.yONpfXTHudxfstests-dev'... 2024-04-20T10:23:06.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:23:06.141 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test) 2024-04-20T10:23:06.212 DEBUG:teuthology.orchestra.run.smithi121:> mktemp -d --suffix=test 2024-04-20T10:23:06.262 INFO:teuthology.orchestra.run.smithi121.stdout:/tmp/tmp.ru8ajEsoUutest 2024-04-20T10:23:06.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:23:06.262 DEBUG:teuthology.orchestra.run.smithi121:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scratch) 2024-04-20T10:23:06.316 DEBUG:teuthology.orchestra.run.smithi121:> mktemp -d --suffix=scratch 2024-04-20T10:23:06.367 INFO:teuthology.orchestra.run.smithi121.stdout:/tmp/tmp.ocGXkJD5RZscratch 2024-04-20T10:23:06.367 INFO:tasks.cephfs.xfstests_dev:distro and version detected is "centos" and "9". 2024-04-20T10:23:06.367 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum install -y acl attr automake bc dbench dump e2fsprogs fio gawk gcc gdbm-devel git indent kernel-devel libacl-devel libaio-devel libcap-devel libtool libuuid-devel lvm2 make psmisc python3 quota sed sqlite udftools xfsprogs 2024-04-20T10:23:07.145 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:07:17 ago on Sat 20 Apr 2024 10:15:50 AM UTC. 2024-04-20T10:23:07.240 INFO:teuthology.orchestra.run.smithi121.stdout:Package acl-2.3.1-4.el9.x86_64 is already installed. 2024-04-20T10:23:07.241 INFO:teuthology.orchestra.run.smithi121.stdout:Package attr-2.5.1-3.el9.x86_64 is already installed. 2024-04-20T10:23:07.242 INFO:teuthology.orchestra.run.smithi121.stdout:Package automake-1.16.2-8.el9.noarch is already installed. 2024-04-20T10:23:07.243 INFO:teuthology.orchestra.run.smithi121.stdout:Package bc-1.07.1-14.el9.x86_64 is already installed. 2024-04-20T10:23:07.244 INFO:teuthology.orchestra.run.smithi121.stdout:Package dbench-4.0-25.el9.x86_64 is already installed. 2024-04-20T10:23:07.246 INFO:teuthology.orchestra.run.smithi121.stdout:Package e2fsprogs-1.46.5-5.el9.x86_64 is already installed. 2024-04-20T10:23:07.247 INFO:teuthology.orchestra.run.smithi121.stdout:Package gawk-5.1.0-6.el9.x86_64 is already installed. 2024-04-20T10:23:07.248 INFO:teuthology.orchestra.run.smithi121.stdout:Package gcc-11.4.1-3.el9.x86_64 is already installed. 2024-04-20T10:23:07.249 INFO:teuthology.orchestra.run.smithi121.stdout:Package git-2.43.0-1.el9.x86_64 is already installed. 2024-04-20T10:23:07.252 INFO:teuthology.orchestra.run.smithi121.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-20T10:23:07.252 INFO:teuthology.orchestra.run.smithi121.stdout:Package libaio-devel-0.3.111-13.el9.x86_64 is already installed. 2024-04-20T10:23:07.254 INFO:teuthology.orchestra.run.smithi121.stdout:Package libtool-2.4.6-45.el9.x86_64 is already installed. 2024-04-20T10:23:07.255 INFO:teuthology.orchestra.run.smithi121.stdout:Package libuuid-devel-2.37.4-18.el9.x86_64 is already installed. 2024-04-20T10:23:07.256 INFO:teuthology.orchestra.run.smithi121.stdout:Package lvm2-9:2.03.23-2.el9.x86_64 is already installed. 2024-04-20T10:23:07.256 INFO:teuthology.orchestra.run.smithi121.stdout:Package make-1:4.3-8.el9.x86_64 is already installed. 2024-04-20T10:23:07.257 INFO:teuthology.orchestra.run.smithi121.stdout:Package psmisc-23.4-3.el9.x86_64 is already installed. 2024-04-20T10:23:07.258 INFO:teuthology.orchestra.run.smithi121.stdout:Package python3-3.9.18-3.el9.x86_64 is already installed. 2024-04-20T10:23:07.259 INFO:teuthology.orchestra.run.smithi121.stdout:Package quota-1:4.06-6.el9.x86_64 is already installed. 2024-04-20T10:23:07.260 INFO:teuthology.orchestra.run.smithi121.stdout:Package sed-4.8-9.el9.x86_64 is already installed. 2024-04-20T10:23:07.262 INFO:teuthology.orchestra.run.smithi121.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-20T10:23:07.308 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2024-04-20T10:23:07.309 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-04-20T10:23:07.309 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2024-04-20T10:23:07.309 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-04-20T10:23:07.309 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2024-04-20T10:23:07.309 INFO:teuthology.orchestra.run.smithi121.stdout: dump x86_64 1:0.4-0.51.b47.el9 epel 88 k 2024-04-20T10:23:07.309 INFO:teuthology.orchestra.run.smithi121.stdout: fio x86_64 3.35-1.el9 appstream 6.2 M 2024-04-20T10:23:07.310 INFO:teuthology.orchestra.run.smithi121.stdout: gdbm-devel x86_64 1:1.19-4.el9 crb 59 k 2024-04-20T10:23:07.310 INFO:teuthology.orchestra.run.smithi121.stdout: indent x86_64 2.2.13-5.el9 epel 179 k 2024-04-20T10:23:07.310 INFO:teuthology.orchestra.run.smithi121.stdout: kernel-devel x86_64 5.14.0-437.el9 appstream 16 M 2024-04-20T10:23:07.310 INFO:teuthology.orchestra.run.smithi121.stdout: libcap-devel x86_64 2.48-9.el9 appstream 41 k 2024-04-20T10:23:07.310 INFO:teuthology.orchestra.run.smithi121.stdout: sqlite x86_64 3.34.1-7.el9 appstream 748 k 2024-04-20T10:23:07.310 INFO:teuthology.orchestra.run.smithi121.stdout: udftools x86_64 2.2-5.el9 appstream 149 k 2024-04-20T10:23:07.310 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2024-04-20T10:23:07.310 INFO:teuthology.orchestra.run.smithi121.stdout: gdbm x86_64 1:1.19-4.el9 crb 132 k 2024-04-20T10:23:07.310 INFO:teuthology.orchestra.run.smithi121.stdout: libnbd x86_64 1.18.1-3.el9 appstream 166 k 2024-04-20T10:23:07.310 INFO:teuthology.orchestra.run.smithi121.stdout: rmt x86_64 2:1.6-6.el9 baseos 46 k 2024-04-20T10:23:07.310 INFO:teuthology.orchestra.run.smithi121.stdout:Installing weak dependencies: 2024-04-20T10:23:07.310 INFO:teuthology.orchestra.run.smithi121.stdout: fio-engine-http x86_64 3.35-1.el9 appstream 18 k 2024-04-20T10:23:07.311 INFO:teuthology.orchestra.run.smithi121.stdout: fio-engine-libaio x86_64 3.35-1.el9 appstream 16 k 2024-04-20T10:23:07.311 INFO:teuthology.orchestra.run.smithi121.stdout: fio-engine-nbd x86_64 3.35-1.el9 appstream 15 k 2024-04-20T10:23:07.311 INFO:teuthology.orchestra.run.smithi121.stdout: fio-engine-rados x86_64 3.35-1.el9 appstream 16 k 2024-04-20T10:23:07.311 INFO:teuthology.orchestra.run.smithi121.stdout: fio-engine-rbd x86_64 3.35-1.el9 appstream 16 k 2024-04-20T10:23:07.311 INFO:teuthology.orchestra.run.smithi121.stdout: fio-engine-rdma x86_64 3.35-1.el9 appstream 21 k 2024-04-20T10:23:07.311 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:23:07.311 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2024-04-20T10:23:07.311 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2024-04-20T10:23:07.311 INFO:teuthology.orchestra.run.smithi121.stdout:Install 17 Packages 2024-04-20T10:23:07.311 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:23:07.312 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 24 M 2024-04-20T10:23:07.312 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 78 M 2024-04-20T10:23:07.312 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2024-04-20T10:23:08.230 INFO:teuthology.orchestra.run.smithi121.stdout:(1/17): fio-engine-http-3.35-1.el9.x86_64.rpm 130 kB/s | 18 kB 00:00 2024-04-20T10:23:08.255 INFO:teuthology.orchestra.run.smithi121.stdout:(2/17): rmt-1.6-6.el9.x86_64.rpm 281 kB/s | 46 kB 00:00 2024-04-20T10:23:08.288 INFO:teuthology.orchestra.run.smithi121.stdout:(3/17): fio-engine-libaio-3.35-1.el9.x86_64.rpm 271 kB/s | 16 kB 00:00 2024-04-20T10:23:08.347 INFO:teuthology.orchestra.run.smithi121.stdout:(4/17): fio-engine-rados-3.35-1.el9.x86_64.rpm 273 kB/s | 16 kB 00:00 2024-04-20T10:23:08.381 INFO:teuthology.orchestra.run.smithi121.stdout:(5/17): fio-engine-nbd-3.35-1.el9.x86_64.rpm 117 kB/s | 15 kB 00:00 2024-04-20T10:23:08.415 INFO:teuthology.orchestra.run.smithi121.stdout:(6/17): fio-engine-rbd-3.35-1.el9.x86_64.rpm 244 kB/s | 16 kB 00:00 2024-04-20T10:23:08.440 INFO:teuthology.orchestra.run.smithi121.stdout:(7/17): fio-engine-rdma-3.35-1.el9.x86_64.rpm 358 kB/s | 21 kB 00:00 2024-04-20T10:23:08.624 INFO:teuthology.orchestra.run.smithi121.stdout:(8/17): fio-3.35-1.el9.x86_64.rpm 12 MB/s | 6.2 MB 00:00 2024-04-20T10:23:08.649 INFO:teuthology.orchestra.run.smithi121.stdout:(9/17): libcap-devel-2.48-9.el9.x86_64.rpm 195 kB/s | 41 kB 00:00 2024-04-20T10:23:08.683 INFO:teuthology.orchestra.run.smithi121.stdout:(10/17): libnbd-1.18.1-3.el9.x86_64.rpm 2.8 MB/s | 166 kB 00:00 2024-04-20T10:23:08.833 INFO:teuthology.orchestra.run.smithi121.stdout:(11/17): sqlite-3.34.1-7.el9.x86_64.rpm 4.0 MB/s | 748 kB 00:00 2024-04-20T10:23:08.925 INFO:teuthology.orchestra.run.smithi121.stdout:(12/17): udftools-2.2-5.el9.x86_64.rpm 619 kB/s | 149 kB 00:00 2024-04-20T10:23:09.283 INFO:teuthology.orchestra.run.smithi121.stdout:(13/17): kernel-devel-5.14.0-437.el9.x86_64.rpm 18 MB/s | 16 MB 00:00 2024-04-20T10:23:09.308 INFO:teuthology.orchestra.run.smithi121.stdout:(14/17): gdbm-devel-1.19-4.el9.x86_64.rpm 153 kB/s | 59 kB 00:00 2024-04-20T10:23:09.334 INFO:teuthology.orchestra.run.smithi121.stdout:(15/17): gdbm-1.19-4.el9.x86_64.rpm 265 kB/s | 132 kB 00:00 2024-04-20T10:23:09.559 INFO:teuthology.orchestra.run.smithi121.stdout:(16/17): dump-0.4-0.51.b47.el9.x86_64.rpm 320 kB/s | 88 kB 00:00 2024-04-20T10:23:09.609 INFO:teuthology.orchestra.run.smithi121.stdout:(17/17): indent-2.2.13-5.el9.x86_64.rpm 599 kB/s | 179 kB 00:00 2024-04-20T10:23:09.613 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:23:09.614 INFO:teuthology.orchestra.run.smithi121.stdout:Total 10 MB/s | 24 MB 00:02 2024-04-20T10:23:10.323 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2024-04-20T10:23:10.434 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2024-04-20T10:23:10.435 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2024-04-20T10:23:11.191 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2024-04-20T10:23:11.191 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2024-04-20T10:23:12.367 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2024-04-20T10:23:12.455 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : gdbm-1:1.19-4.el9.x86_64 1/17 2024-04-20T10:23:12.489 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libnbd-1.18.1-3.el9.x86_64 2/17 2024-04-20T10:23:12.523 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : fio-engine-http-3.35-1.el9.x86_64 3/17 2024-04-20T10:23:12.557 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : fio-engine-libaio-3.35-1.el9.x86_64 4/17 2024-04-20T10:23:12.601 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : fio-engine-nbd-3.35-1.el9.x86_64 5/17 2024-04-20T10:23:12.635 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : fio-engine-rados-3.35-1.el9.x86_64 6/17 2024-04-20T10:23:12.668 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : fio-engine-rbd-3.35-1.el9.x86_64 7/17 2024-04-20T10:23:12.929 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : fio-engine-rdma-3.35-1.el9.x86_64 8/17 2024-04-20T10:23:12.981 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : fio-3.35-1.el9.x86_64 9/17 2024-04-20T10:23:13.046 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rmt-2:1.6-6.el9.x86_64 10/17 2024-04-20T10:23:13.084 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : dump-1:0.4-0.51.b47.el9.x86_64 11/17 2024-04-20T10:23:13.168 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : gdbm-devel-1:1.19-4.el9.x86_64 12/17 2024-04-20T10:23:13.233 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : indent-2.2.13-5.el9.x86_64 13/17 2024-04-20T10:23:13.290 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : udftools-2.2-5.el9.x86_64 14/17 2024-04-20T10:23:13.365 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : sqlite-3.34.1-7.el9.x86_64 15/17 2024-04-20T10:23:24.871 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcap-devel-2.48-9.el9.x86_64 16/17 2024-04-20T10:23:25.199 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : kernel-devel-5.14.0-437.el9.x86_64 17/17 2024-04-20T10:23:25.847 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:23:25.853 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:23:56.008 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:23:56.017 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:24:14.911 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: kernel-devel-5.14.0-437.el9.x86_64 17/17 2024-04-20T10:24:14.911 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rmt-2:1.6-6.el9.x86_64 1/17 2024-04-20T10:24:14.911 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : fio-3.35-1.el9.x86_64 2/17 2024-04-20T10:24:14.912 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : fio-engine-http-3.35-1.el9.x86_64 3/17 2024-04-20T10:24:14.912 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : fio-engine-libaio-3.35-1.el9.x86_64 4/17 2024-04-20T10:24:14.912 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : fio-engine-nbd-3.35-1.el9.x86_64 5/17 2024-04-20T10:24:14.912 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : fio-engine-rados-3.35-1.el9.x86_64 6/17 2024-04-20T10:24:14.912 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : fio-engine-rbd-3.35-1.el9.x86_64 7/17 2024-04-20T10:24:14.912 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : fio-engine-rdma-3.35-1.el9.x86_64 8/17 2024-04-20T10:24:14.912 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : kernel-devel-5.14.0-437.el9.x86_64 9/17 2024-04-20T10:24:14.912 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcap-devel-2.48-9.el9.x86_64 10/17 2024-04-20T10:24:14.912 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libnbd-1.18.1-3.el9.x86_64 11/17 2024-04-20T10:24:14.912 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : sqlite-3.34.1-7.el9.x86_64 12/17 2024-04-20T10:24:14.912 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : udftools-2.2-5.el9.x86_64 13/17 2024-04-20T10:24:14.912 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : gdbm-1:1.19-4.el9.x86_64 14/17 2024-04-20T10:24:14.912 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : gdbm-devel-1:1.19-4.el9.x86_64 15/17 2024-04-20T10:24:14.912 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : dump-1:0.4-0.51.b47.el9.x86_64 16/17 2024-04-20T10:24:15.429 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : indent-2.2.13-5.el9.x86_64 17/17 2024-04-20T10:24:15.429 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:24:15.429 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2024-04-20T10:24:15.429 INFO:teuthology.orchestra.run.smithi121.stdout: dump-1:0.4-0.51.b47.el9.x86_64 fio-3.35-1.el9.x86_64 2024-04-20T10:24:15.429 INFO:teuthology.orchestra.run.smithi121.stdout: fio-engine-http-3.35-1.el9.x86_64 fio-engine-libaio-3.35-1.el9.x86_64 2024-04-20T10:24:15.429 INFO:teuthology.orchestra.run.smithi121.stdout: fio-engine-nbd-3.35-1.el9.x86_64 fio-engine-rados-3.35-1.el9.x86_64 2024-04-20T10:24:15.429 INFO:teuthology.orchestra.run.smithi121.stdout: fio-engine-rbd-3.35-1.el9.x86_64 fio-engine-rdma-3.35-1.el9.x86_64 2024-04-20T10:24:15.430 INFO:teuthology.orchestra.run.smithi121.stdout: gdbm-1:1.19-4.el9.x86_64 gdbm-devel-1:1.19-4.el9.x86_64 2024-04-20T10:24:15.430 INFO:teuthology.orchestra.run.smithi121.stdout: indent-2.2.13-5.el9.x86_64 kernel-devel-5.14.0-437.el9.x86_64 2024-04-20T10:24:15.430 INFO:teuthology.orchestra.run.smithi121.stdout: libcap-devel-2.48-9.el9.x86_64 libnbd-1.18.1-3.el9.x86_64 2024-04-20T10:24:15.430 INFO:teuthology.orchestra.run.smithi121.stdout: rmt-2:1.6-6.el9.x86_64 sqlite-3.34.1-7.el9.x86_64 2024-04-20T10:24:15.430 INFO:teuthology.orchestra.run.smithi121.stdout: udftools-2.2-5.el9.x86_64 2024-04-20T10:24:15.430 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:24:15.430 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2024-04-20T10:24:15.819 DEBUG:teuthology.orchestra.run.smithi121:> sudo useradd -m fsgqa 2024-04-20T10:24:15.887 INFO:teuthology.orchestra.run.smithi121.stderr:useradd: user 'fsgqa' already exists 2024-04-20T10:24:15.890 DEBUG:teuthology.orchestra.run:got remote process result: 9 2024-04-20T10:24:15.891 DEBUG:teuthology.orchestra.run.smithi121:> sudo groupadd fsgqa 2024-04-20T10:24:15.957 INFO:teuthology.orchestra.run.smithi121.stderr:groupadd: group 'fsgqa' already exists 2024-04-20T10:24:15.960 DEBUG:teuthology.orchestra.run:got remote process result: 9 2024-04-20T10:24:15.960 DEBUG:teuthology.orchestra.run.smithi121:> sudo useradd fsgqa2 2024-04-20T10:24:16.636 DEBUG:teuthology.orchestra.run.smithi121:> sudo useradd 123456-fsgqa 2024-04-20T10:24:17.291 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json mon dump 2024-04-20T10:24:17.463 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.463+0000 7f3d97200640 1 Processor -- start 2024-04-20T10:24:17.463 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.463+0000 7f3d97200640 1 -- start start 2024-04-20T10:24:17.465 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.464+0000 7f3d97200640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3d9815b400 0x7f3d98072d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:17.465 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.464+0000 7f3d97200640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3d9815fa40 0x7f3d9806ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:17.465 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.464+0000 7f3d97200640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d9815c7f0 con 0x7f3d9815eec0 2024-04-20T10:24:17.465 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.464+0000 7f3d97200640 1 -- --> v2:172.21.15.121:3300/0 -- mon_getmap magic: 0 -- 0x7f3d9806d280 con 0x7f3d9815b400 2024-04-20T10:24:17.465 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.464+0000 7f3d97200640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f3d9806ccc0 con 0x7f3d9815fa40 2024-04-20T10:24:17.465 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.465+0000 7f3d95e00640 1 --2- >> v2:172.21.15.121:3300/0 conn(0x7f3d9815b400 0x7f3d98072d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:17.465 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.465+0000 7f3d95400640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f3d9815eec0 0x7f3d9815f330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:47308/0 (socket says 172.21.15.121:47308) 2024-04-20T10:24:17.465 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.465+0000 7f3d95400640 1 -- 172.21.15.121:0/698483765 learned_addr learned my addr 172.21.15.121:0/698483765 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:24:17.465 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.465+0000 7f3d96800640 1 --2- 172.21.15.121:0/698483765 >> v2:172.21.15.123:3300/0 conn(0x7f3d9815fa40 0x7f3d9806ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:17.465 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.465+0000 7f3d95e00640 1 -- 172.21.15.121:0/698483765 >> v2:172.21.15.123:3300/0 conn(0x7f3d9815fa40 msgr2=0x7f3d9806ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:24:17.465 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.465+0000 7f3d95e00640 1 --2- 172.21.15.121:0/698483765 >> v2:172.21.15.123:3300/0 conn(0x7f3d9815fa40 0x7f3d9806ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:17.465 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.465+0000 7f3d95e00640 1 -- 172.21.15.121:0/698483765 >> v1:172.21.15.121:6789/0 conn(0x7f3d9815eec0 legacy=0x7f3d9815f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:17.466 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.465+0000 7f3d95e00640 1 -- 172.21.15.121:0/698483765 --> v2:172.21.15.121:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d9806cfa0 con 0x7f3d9815b400 2024-04-20T10:24:17.466 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.466+0000 7f3d95e00640 1 --2- 172.21.15.121:0/698483765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d9815b400 0x7f3d98072d00 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f3d98072250 tx=0x7f3d8802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a99e54610cfc73b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:24:17.466 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.467+0000 7f3d94a00640 1 -- 172.21.15.121:0/698483765 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d88035020 con 0x7f3d9815b400 2024-04-20T10:24:17.466 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.467+0000 7f3d94a00640 1 -- 172.21.15.121:0/698483765 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d88031830 con 0x7f3d9815b400 2024-04-20T10:24:17.466 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.467+0000 7f3d94a00640 1 -- 172.21.15.121:0/698483765 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d88031c00 con 0x7f3d9815b400 2024-04-20T10:24:17.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.467+0000 7f3d97200640 1 -- 172.21.15.121:0/698483765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d9815b400 msgr2=0x7f3d98072d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:17.467 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.467+0000 7f3d97200640 1 --2- 172.21.15.121:0/698483765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d9815b400 0x7f3d98072d00 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f3d98072250 tx=0x7f3d8802b8d0 comp rx=0 tx=0).stop 2024-04-20T10:24:17.468 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.467+0000 7f3d97200640 1 -- 172.21.15.121:0/698483765 shutdown_connections 2024-04-20T10:24:17.468 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.467+0000 7f3d97200640 1 --2- 172.21.15.121:0/698483765 >> v2:172.21.15.123:3300/0 conn(0x7f3d9815fa40 0x7f3d9806ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:17.468 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.467+0000 7f3d97200640 1 --2- 172.21.15.121:0/698483765 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d9815b400 0x7f3d98072d00 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:17.468 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.467+0000 7f3d97200640 1 -- 172.21.15.121:0/698483765 >> 172.21.15.121:0/698483765 conn(0x7f3d980d0040 msgr2=0x7f3d980d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:24:17.468 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.468+0000 7f3d97200640 1 -- 172.21.15.121:0/698483765 shutdown_connections 2024-04-20T10:24:17.468 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.468+0000 7f3d97200640 1 -- 172.21.15.121:0/698483765 wait complete. 2024-04-20T10:24:17.469 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.469+0000 7f3d97200640 1 Processor -- start 2024-04-20T10:24:17.469 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.469+0000 7f3d97200640 1 -- start start 2024-04-20T10:24:17.471 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.470+0000 7f3d97200640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d9815b400 0x7f3d98154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:17.471 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.470+0000 7f3d97200640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3d9815eec0 0x7f3d980ce920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:17.471 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.470+0000 7f3d97200640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d9815fa40 0x7f3d981536b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:17.471 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.470+0000 7f3d97200640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f3d98073240 con 0x7f3d9815b400 2024-04-20T10:24:17.471 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.470+0000 7f3d97200640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9806e720 con 0x7f3d9815fa40 2024-04-20T10:24:17.471 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.470+0000 7f3d97200640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f3d9806f180 con 0x7f3d9815eec0 2024-04-20T10:24:17.471 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.471+0000 7f3d95e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d9815b400 0x7f3d98154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:17.471 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.471+0000 7f3d95e00640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d9815b400 0x7f3d98154ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:57000/0 (socket says 172.21.15.121:57000) 2024-04-20T10:24:17.471 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.471+0000 7f3d95e00640 1 -- 172.21.15.121:0/3795017951 learned_addr learned my addr 172.21.15.121:0/3795017951 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:24:17.471 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.471+0000 7f3d95400640 1 --2- 172.21.15.121:0/3795017951 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3d9815eec0 0x7f3d980ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:17.471 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.471+0000 7f3d96800640 1 --2- 172.21.15.121:0/3795017951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d9815fa40 0x7f3d981536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:17.472 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.471+0000 7f3d95e00640 1 -- 172.21.15.121:0/3795017951 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3d9815eec0 msgr2=0x7f3d980ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:17.472 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.471+0000 7f3d95e00640 1 --2- 172.21.15.121:0/3795017951 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3d9815eec0 0x7f3d980ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:17.472 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.471+0000 7f3d95e00640 1 -- 172.21.15.121:0/3795017951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d9815fa40 msgr2=0x7f3d981536b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:17.472 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.471+0000 7f3d95e00640 1 --2- 172.21.15.121:0/3795017951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d9815fa40 0x7f3d981536b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:17.472 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.471+0000 7f3d95e00640 1 -- 172.21.15.121:0/3795017951 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d980cee60 con 0x7f3d9815b400 2024-04-20T10:24:17.472 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.471+0000 7f3d95e00640 1 --2- 172.21.15.121:0/3795017951 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d9815b400 0x7f3d98154ea0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3d88009890 tx=0x7f3d880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:24:17.473 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.472+0000 7f3d8e000640 1 -- 172.21.15.121:0/3795017951 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d88041070 con 0x7f3d9815b400 2024-04-20T10:24:17.473 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.472+0000 7f3d8e000640 1 -- 172.21.15.121:0/3795017951 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d8803aa30 con 0x7f3d9815b400 2024-04-20T10:24:17.473 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.472+0000 7f3d8e000640 1 -- 172.21.15.121:0/3795017951 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d88035040 con 0x7f3d9815b400 2024-04-20T10:24:17.473 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.472+0000 7f3d97200640 1 -- 172.21.15.121:0/3795017951 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d980cf130 con 0x7f3d9815b400 2024-04-20T10:24:17.473 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.472+0000 7f3d97200640 1 -- 172.21.15.121:0/3795017951 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d980cccb0 con 0x7f3d9815b400 2024-04-20T10:24:17.473 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.473+0000 7f3d97200640 1 -- 172.21.15.121:0/3795017951 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d98075b60 con 0x7f3d9815b400 2024-04-20T10:24:17.473 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.474+0000 7f3d8e000640 1 -- 172.21.15.121:0/3795017951 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3d88038020 con 0x7f3d9815b400 2024-04-20T10:24:17.475 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.475+0000 7f3d8e000640 1 --2- 172.21.15.121:0/3795017951 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f3d7806c8f0 0x7f3d7806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:17.475 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.475+0000 7f3d8e000640 1 -- 172.21.15.121:0/3795017951 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f3d880ad6c0 con 0x7f3d9815b400 2024-04-20T10:24:17.475 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.476+0000 7f3d95400640 1 --2- 172.21.15.121:0/3795017951 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f3d7806c8f0 0x7f3d7806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:17.476 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.476+0000 7f3d95400640 1 --2- 172.21.15.121:0/3795017951 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f3d7806c8f0 0x7f3d7806edb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3d98072250 tx=0x7f3d84009040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:24:17.480 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.480+0000 7f3d8e000640 1 -- 172.21.15.121:0/3795017951 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3d8807b9d0 con 0x7f3d9815b400 2024-04-20T10:24:17.779 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.780+0000 7f3d97200640 1 -- 172.21.15.121:0/3795017951 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f3d980b5220 con 0x7f3d9815b400 2024-04-20T10:24:17.780 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.780+0000 7f3d8e000640 1 -- 172.21.15.121:0/3795017951 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+1289 (secure 0 0 0) 0x7f3d8807b2d0 con 0x7f3d9815b400 2024-04-20T10:24:17.780 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:24:17.780 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":1,"fsid":"ceaac016-4ab2-472d-a36e-7bfe9497c762","modified":"2024-04-20T10:19:57.302223Z","created":"2024-04-20T10:19:57.302223Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:3300","nonce":0},{"type":"v1","addr":"172.21.15.121:6789","nonce":0}]},"addr":"172.21.15.121:6789/0","public_addr":"172.21.15.121:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:3300","nonce":0},{"type":"v1","addr":"172.21.15.123:6789","nonce":0}]},"addr":"172.21.15.123:6789/0","public_addr":"172.21.15.123:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:3301","nonce":0},{"type":"v1","addr":"172.21.15.123:6790","nonce":0}]},"addr":"172.21.15.123:6790/0","public_addr":"172.21.15.123:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-04-20T10:24:17.780 INFO:teuthology.orchestra.run.smithi121.stderr:dumped monmap epoch 1 2024-04-20T10:24:17.782 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.783+0000 7f3d97200640 1 -- 172.21.15.121:0/3795017951 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f3d7806c8f0 msgr2=0x7f3d7806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:17.783 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.783+0000 7f3d97200640 1 --2- 172.21.15.121:0/3795017951 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f3d7806c8f0 0x7f3d7806edb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3d98072250 tx=0x7f3d84009040 comp rx=0 tx=0).stop 2024-04-20T10:24:17.783 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.783+0000 7f3d97200640 1 -- 172.21.15.121:0/3795017951 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d9815b400 msgr2=0x7f3d98154ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:17.783 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.783+0000 7f3d97200640 1 --2- 172.21.15.121:0/3795017951 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d9815b400 0x7f3d98154ea0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3d88009890 tx=0x7f3d880046c0 comp rx=0 tx=0).stop 2024-04-20T10:24:17.783 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.783+0000 7f3d97200640 1 -- 172.21.15.121:0/3795017951 shutdown_connections 2024-04-20T10:24:17.783 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.783+0000 7f3d97200640 1 --2- 172.21.15.121:0/3795017951 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f3d7806c8f0 0x7f3d7806edb0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:17.783 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.783+0000 7f3d97200640 1 --2- 172.21.15.121:0/3795017951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d9815fa40 0x7f3d981536b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:17.783 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.783+0000 7f3d97200640 1 --2- 172.21.15.121:0/3795017951 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f3d9815eec0 0x7f3d980ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:17.783 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.783+0000 7f3d97200640 1 --2- 172.21.15.121:0/3795017951 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f3d9815b400 0x7f3d98154ea0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:17.783 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.783+0000 7f3d97200640 1 -- 172.21.15.121:0/3795017951 >> 172.21.15.121:0/3795017951 conn(0x7f3d980d0040 msgr2=0x7f3d9815ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:24:17.783 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.783+0000 7f3d97200640 1 -- 172.21.15.121:0/3795017951 shutdown_connections 2024-04-20T10:24:17.783 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.783+0000 7f3d97200640 1 -- 172.21.15.121:0/3795017951 wait complete. 2024-04-20T10:24:17.800 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get client.admin 2024-04-20T10:24:17.969 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.969+0000 7f7aef000640 1 Processor -- start 2024-04-20T10:24:17.969 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.969+0000 7f7aef000640 1 -- start start 2024-04-20T10:24:17.970 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.970+0000 7f7aef000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ae8162c00 0x7f7ae80d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:17.970 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.970+0000 7f7aef000640 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7ae806f090 0x7f7ae80d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:17.970 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.970+0000 7f7aef000640 1 -- --> v1:172.21.15.121:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ae806df70 con 0x7f7ae806fc30 2024-04-20T10:24:17.970 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.970+0000 7f7aef000640 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 -- 0x7f7ae8162fe0 con 0x7f7ae806f090 2024-04-20T10:24:17.970 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.970+0000 7f7aef000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f7ae806ec00 con 0x7f7ae8162c00 2024-04-20T10:24:17.970 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.971+0000 7f7aed000640 1 --1- >> v1:172.21.15.121:6789/0 conn(0x7f7ae806fc30 0x7f7ae806b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.121:6789/0 says I am v1:172.21.15.121:47318/0 (socket says 172.21.15.121:47318) 2024-04-20T10:24:17.970 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.971+0000 7f7aed000640 1 -- 172.21.15.121:0/794402127 learned_addr learned my addr 172.21.15.121:0/794402127 (peer_addr_for_me v1:172.21.15.121:0/0) 2024-04-20T10:24:17.972 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.971+0000 7f7ae7400640 1 --2- 172.21.15.121:0/794402127 >> v2:172.21.15.123:3300/0 conn(0x7f7ae806f090 0x7f7ae80d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:17.972 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.971+0000 7f7ae7e00640 1 --2- 172.21.15.121:0/794402127 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ae8162c00 0x7f7ae80d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:17.972 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.971+0000 7f7ae6a00640 1 -- 172.21.15.121:0/794402127 <== mon.0 v1:172.21.15.121:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 309811411 0 0) 0x7f7ae806df70 con 0x7f7ae806fc30 2024-04-20T10:24:17.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.971+0000 7f7ae6a00640 1 -- 172.21.15.121:0/794402127 --> v1:172.21.15.121:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7acc003520 con 0x7f7ae806fc30 2024-04-20T10:24:17.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.972+0000 7f7ae6a00640 1 -- 172.21.15.121:0/794402127 <== mon.0 v1:172.21.15.121:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1536368037 0 0) 0x7f7acc003520 con 0x7f7ae806fc30 2024-04-20T10:24:17.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.972+0000 7f7ae6a00640 1 -- 172.21.15.121:0/794402127 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ae8162c00 msgr2=0x7f7ae80d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:24:17.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.972+0000 7f7ae6a00640 1 --2- 172.21.15.121:0/794402127 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ae8162c00 0x7f7ae80d0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:17.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.972+0000 7f7ae6a00640 1 -- 172.21.15.121:0/794402127 >> v2:172.21.15.123:3300/0 conn(0x7f7ae806f090 msgr2=0x7f7ae80d08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:24:17.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.972+0000 7f7ae6a00640 1 --2- 172.21.15.121:0/794402127 >> v2:172.21.15.123:3300/0 conn(0x7f7ae806f090 0x7f7ae80d08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:17.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.972+0000 7f7ae6a00640 1 -- 172.21.15.121:0/794402127 --> v1:172.21.15.121:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ae806e310 con 0x7f7ae806fc30 2024-04-20T10:24:17.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.972+0000 7f7ae6a00640 1 -- 172.21.15.121:0/794402127 <== mon.0 v1:172.21.15.121:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2059816984 0 0) 0x7f7ae00019f0 con 0x7f7ae806fc30 2024-04-20T10:24:17.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.972+0000 7f7ae6a00640 1 -- 172.21.15.121:0/794402127 >> v1:172.21.15.121:6789/0 conn(0x7f7ae806fc30 legacy=0x7f7ae806b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:17.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.972+0000 7f7ae6a00640 1 --2- 172.21.15.121:0/794402127 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ae8162c00 0x7f7acc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:17.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.972+0000 7f7ae6a00640 1 --2- 172.21.15.121:0/794402127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7acc003f80 0x7f7acc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:17.973 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.972+0000 7f7ae6a00640 1 --2- 172.21.15.121:0/794402127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7acc004d40 0x7f7acc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:17.974 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.972+0000 7f7ae6a00640 1 -- 172.21.15.121:0/794402127 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae806ec00 con 0x7f7acc003f80 2024-04-20T10:24:17.974 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.972+0000 7f7ae6a00640 1 -- 172.21.15.121:0/794402127 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7acc00b9f0 con 0x7f7acc004d40 2024-04-20T10:24:17.974 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.972+0000 7f7ae6a00640 1 -- 172.21.15.121:0/794402127 --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f7acc00bbf0 con 0x7f7ae8162c00 2024-04-20T10:24:17.974 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.972+0000 7f7ae7400640 1 --2- 172.21.15.121:0/794402127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7acc003f80 0x7f7acc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:17.974 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.972+0000 7f7ae7e00640 1 --2- 172.21.15.121:0/794402127 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ae8162c00 0x7f7acc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:17.974 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.972+0000 7f7ae7400640 1 -- 172.21.15.121:0/794402127 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ae8162c00 msgr2=0x7f7acc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:24:17.974 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.972+0000 7f7aed000640 1 --2- 172.21.15.121:0/794402127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7acc004d40 0x7f7acc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:17.974 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.972+0000 7f7ae7400640 1 --2- 172.21.15.121:0/794402127 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ae8162c00 0x7f7acc0039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:17.974 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.972+0000 7f7ae7400640 1 -- 172.21.15.121:0/794402127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7acc004d40 msgr2=0x7f7acc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:24:17.974 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.972+0000 7f7ae7400640 1 --2- 172.21.15.121:0/794402127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7acc004d40 0x7f7acc0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:17.974 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.973+0000 7f7ae7400640 1 -- 172.21.15.121:0/794402127 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7acc003520 con 0x7f7acc003f80 2024-04-20T10:24:17.976 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.974+0000 7f7ae7400640 1 --2- 172.21.15.121:0/794402127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7acc003f80 0x7f7acc004800 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4009a90 tx=0x7f7acc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd9b52388a33c76 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:24:17.976 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.974+0000 7f7ae6a00640 1 -- 172.21.15.121:0/794402127 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4039070 con 0x7f7acc003f80 2024-04-20T10:24:17.977 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.974+0000 7f7ae6a00640 1 -- 172.21.15.121:0/794402127 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ad402c6b0 con 0x7f7acc003f80 2024-04-20T10:24:17.977 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.974+0000 7f7ae6a00640 1 -- 172.21.15.121:0/794402127 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4009690 con 0x7f7acc003f80 2024-04-20T10:24:17.977 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.974+0000 7f7aef000640 1 -- 172.21.15.121:0/794402127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7acc003f80 msgr2=0x7f7acc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:17.977 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.974+0000 7f7aef000640 1 --2- 172.21.15.121:0/794402127 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7acc003f80 0x7f7acc004800 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4009a90 tx=0x7f7acc005830 comp rx=0 tx=0).stop 2024-04-20T10:24:17.977 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.975+0000 7f7aed000640 1 -- 172.21.15.121:0/794402127 reap_dead start 2024-04-20T10:24:17.977 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.975+0000 7f7aef000640 1 -- 172.21.15.121:0/794402127 shutdown_connections 2024-04-20T10:24:17.977 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.975+0000 7f7aef000640 1 -- 172.21.15.121:0/794402127 >> 172.21.15.121:0/794402127 conn(0x7f7ae80b3350 msgr2=0x7f7ae80b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:24:17.977 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.975+0000 7f7aef000640 1 -- 172.21.15.121:0/794402127 shutdown_connections 2024-04-20T10:24:17.977 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.975+0000 7f7aef000640 1 -- 172.21.15.121:0/794402127 wait complete. 2024-04-20T10:24:17.977 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.977+0000 7f7aef000640 1 Processor -- start 2024-04-20T10:24:17.977 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.977+0000 7f7aef000640 1 -- start start 2024-04-20T10:24:17.979 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.978+0000 7f7aef000640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7acc003f80 0x7f7ae81fb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:17.979 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.978+0000 7f7aef000640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ae81639f0 0x7f7ae8163e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:17.979 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.978+0000 7f7aef000640 1 --2- >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ae81f8c10 0x7f7ae81f9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:17.979 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.978+0000 7f7aef000640 1 -- --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae806dbb0 con 0x7f7ae81639f0 2024-04-20T10:24:17.979 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.978+0000 7f7aef000640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae806e890 con 0x7f7acc003f80 2024-04-20T10:24:17.980 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.978+0000 7f7aef000640 1 -- --> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] -- mon_getmap magic: 0 -- 0x7f7ae806dd30 con 0x7f7ae81f8c10 2024-04-20T10:24:17.980 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.978+0000 7f7ae7400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ae81639f0 0x7f7ae8163e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:17.980 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.978+0000 7f7ae7400640 1 --2- >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ae81639f0 0x7f7ae8163e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.121:3300/0 says I am v2:172.21.15.121:57022/0 (socket says 172.21.15.121:57022) 2024-04-20T10:24:17.980 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.978+0000 7f7ae7400640 1 -- 172.21.15.121:0/3940669980 learned_addr learned my addr 172.21.15.121:0/3940669980 (peer_addr_for_me v2:172.21.15.121:0/0) 2024-04-20T10:24:17.980 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.978+0000 7f7ae7e00640 1 --2- 172.21.15.121:0/3940669980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7acc003f80 0x7f7ae81fb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:17.980 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.978+0000 7f7ae7400640 1 -- 172.21.15.121:0/3940669980 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ae81f8c10 msgr2=0x7f7ae81f9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:17.980 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.978+0000 7f7ae7400640 1 --2- 172.21.15.121:0/3940669980 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ae81f8c10 0x7f7ae81f9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:17.980 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.978+0000 7f7ae7400640 1 -- 172.21.15.121:0/3940669980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7acc003f80 msgr2=0x7f7ae81fb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:17.980 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.978+0000 7f7ae7400640 1 --2- 172.21.15.121:0/3940669980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7acc003f80 0x7f7ae81fb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:17.980 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.978+0000 7f7ae7400640 1 -- 172.21.15.121:0/3940669980 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ae81644b0 con 0x7f7ae81639f0 2024-04-20T10:24:17.980 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.979+0000 7f7ae7400640 1 --2- 172.21.15.121:0/3940669980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ae81639f0 0x7f7ae8163e70 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4009a90 tx=0x7f7ad4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:24:17.980 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.979+0000 7f7adfe00640 1 -- 172.21.15.121:0/3940669980 <== mon.0 v2:172.21.15.121:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4039050 con 0x7f7ae81639f0 2024-04-20T10:24:17.981 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.979+0000 7f7adfe00640 1 -- 172.21.15.121:0/3940669980 <== mon.0 v2:172.21.15.121:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ad4045070 con 0x7f7ae81639f0 2024-04-20T10:24:17.981 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.979+0000 7f7adfe00640 1 -- 172.21.15.121:0/3940669980 <== mon.0 v2:172.21.15.121:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4003a50 con 0x7f7ae81639f0 2024-04-20T10:24:17.981 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.979+0000 7f7aef000640 1 -- 172.21.15.121:0/3940669980 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ae8164790 con 0x7f7ae81639f0 2024-04-20T10:24:17.981 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.979+0000 7f7aef000640 1 -- 172.21.15.121:0/3940669980 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ae8164d50 con 0x7f7ae81639f0 2024-04-20T10:24:17.981 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.980+0000 7f7aef000640 1 -- 172.21.15.121:0/3940669980 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ae8068980 con 0x7f7ae81639f0 2024-04-20T10:24:17.982 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.981+0000 7f7adfe00640 1 -- 172.21.15.121:0/3940669980 <== mon.0 v2:172.21.15.121:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ad4003bf0 con 0x7f7ae81639f0 2024-04-20T10:24:17.982 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.982+0000 7f7adfe00640 1 --2- 172.21.15.121:0/3940669980 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7ac806c920 0x7f7ac806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:24:17.982 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.982+0000 7f7adfe00640 1 -- 172.21.15.121:0/3940669980 <== mon.0 v2:172.21.15.121:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f7ad40adff0 con 0x7f7ae81639f0 2024-04-20T10:24:17.982 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.982+0000 7f7ae7e00640 1 --2- 172.21.15.121:0/3940669980 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7ac806c920 0x7f7ac806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:24:17.983 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.983+0000 7f7ae7e00640 1 --2- 172.21.15.121:0/3940669980 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7ac806c920 0x7f7ac806ede0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8009830 tx=0x7f7ad8008040 comp rx=0 tx=0).ready entity=mgr.4103 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:24:17.987 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:17.988+0000 7f7adfe00640 1 -- 172.21.15.121:0/3940669980 <== mon.0 v2:172.21.15.121:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7ad407c450 con 0x7f7ae81639f0 2024-04-20T10:24:18.268 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:18.268+0000 7f7aef000640 1 -- 172.21.15.121:0/3940669980 --> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] -- mon_command({"prefix": "auth get", "entity": "client.admin"} v 0) -- 0x7f7ae80b08e0 con 0x7f7ae81639f0 2024-04-20T10:24:18.268 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:18.269+0000 7f7adfe00640 1 -- 172.21.15.121:0/3940669980 <== mon.0 v2:172.21.15.121:3300/0 7 ==== mon_command_ack([{"prefix": "auth get", "entity": "client.admin"}]=0 v13) ==== 82+0+151 (secure 0 0 0) 0x7f7ad407bd50 con 0x7f7ae81639f0 2024-04-20T10:24:18.269 INFO:teuthology.orchestra.run.smithi121.stdout:[client.admin] 2024-04-20T10:24:18.269 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQDNliNmYSb0JxAAuAj2WcluQMgpSBhyiCaS4Q== 2024-04-20T10:24:18.269 INFO:teuthology.orchestra.run.smithi121.stdout: caps mds = "allow *" 2024-04-20T10:24:18.269 INFO:teuthology.orchestra.run.smithi121.stdout: caps mgr = "allow *" 2024-04-20T10:24:18.269 INFO:teuthology.orchestra.run.smithi121.stdout: caps mon = "allow *" 2024-04-20T10:24:18.269 INFO:teuthology.orchestra.run.smithi121.stdout: caps osd = "allow *" 2024-04-20T10:24:18.271 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:18.271+0000 7f7aef000640 1 -- 172.21.15.121:0/3940669980 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7ac806c920 msgr2=0x7f7ac806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:18.271 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:18.271+0000 7f7aef000640 1 --2- 172.21.15.121:0/3940669980 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7ac806c920 0x7f7ac806ede0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8009830 tx=0x7f7ad8008040 comp rx=0 tx=0).stop 2024-04-20T10:24:18.271 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:18.271+0000 7f7aef000640 1 -- 172.21.15.121:0/3940669980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ae81639f0 msgr2=0x7f7ae8163e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:24:18.272 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:18.271+0000 7f7aef000640 1 --2- 172.21.15.121:0/3940669980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ae81639f0 0x7f7ae8163e70 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4009a90 tx=0x7f7ad4004730 comp rx=0 tx=0).stop 2024-04-20T10:24:18.272 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:18.271+0000 7f7aef000640 1 -- 172.21.15.121:0/3940669980 shutdown_connections 2024-04-20T10:24:18.272 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:18.271+0000 7f7aef000640 1 --2- 172.21.15.121:0/3940669980 >> [v2:172.21.15.123:3301/0,v1:172.21.15.123:6790/0] conn(0x7f7ae81f8c10 0x7f7ae81f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:18.272 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:18.271+0000 7f7aef000640 1 --2- 172.21.15.121:0/3940669980 >> [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] conn(0x7f7ae81639f0 0x7f7ae8163e70 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:18.272 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:18.271+0000 7f7aef000640 1 --2- 172.21.15.121:0/3940669980 >> [v2:172.21.15.121:6832/1408233272,v1:172.21.15.121:6833/1408233272] conn(0x7f7ac806c920 0x7f7ac806ede0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:18.272 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:18.271+0000 7f7aef000640 1 --2- 172.21.15.121:0/3940669980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7acc003f80 0x7f7ae81fb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:24:18.272 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:18.272+0000 7f7aef000640 1 -- 172.21.15.121:0/3940669980 >> 172.21.15.121:0/3940669980 conn(0x7f7ae80b3350 msgr2=0x7f7ae80ae2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:24:18.272 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:18.272+0000 7f7aef000640 1 -- 172.21.15.121:0/3940669980 shutdown_connections 2024-04-20T10:24:18.272 INFO:teuthology.orchestra.run.smithi121.stderr:2024-04-20T10:24:18.272+0000 7f7aef000640 1 -- 172.21.15.121:0/3940669980 wait complete. 2024-04-20T10:24:18.293 DEBUG:teuthology.orchestra.run.smithi121:> mktemp 2024-04-20T10:24:18.331 INFO:teuthology.orchestra.run.smithi121.stdout:/tmp/tmp.N1D7U5fCo2 2024-04-20T10:24:18.331 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T10:24:18.331 DEBUG:teuthology.orchestra.run.smithi121:> dd of=/tmp/tmp.N1D7U5fCo2 2024-04-20T10:24:18.409 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T10:24:18.409 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/tmp/tmp.yONpfXTHudxfstests-dev/local.config 2024-04-20T10:24:18.518 INFO:tasks.cephfs.xfstests_dev:local.config's contents - export FSTYP=ceph-fuse export CEPH_FUSE_BIN_PATH=ceph-fuse export TEST_DEV=ceph-fuse # without this tests won't get started export TEST_DIR=/tmp/tmp.ru8ajEsoUutest export SCRATCH_DEV= export SCRATCH_MNT=/tmp/tmp.ocGXkJD5RZscratch export CEPHFS_MOUNT_OPTIONS="-m 172.21.15.121:6789 -k /tmp/tmp.N1D7U5fCo2 --client_mountpoint /test" 2024-04-20T10:24:18.518 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2024-04-20T10:24:18.518 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/tmp/tmp.yONpfXTHudxfstests-dev/ceph.exclude 2024-04-20T10:24:18.627 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo make) 2024-04-20T10:24:18.742 INFO:teuthology.orchestra.run.smithi121.stdout:libtoolize -cfi 2024-04-20T10:24:19.630 INFO:teuthology.orchestra.run.smithi121.stdout:libtoolize: putting auxiliary files in '.'. 2024-04-20T10:24:19.630 INFO:teuthology.orchestra.run.smithi121.stdout:libtoolize: copying file './config.guess' 2024-04-20T10:24:19.644 INFO:teuthology.orchestra.run.smithi121.stdout:libtoolize: copying file './config.sub' 2024-04-20T10:24:19.658 INFO:teuthology.orchestra.run.smithi121.stdout:libtoolize: copying file './install-sh' 2024-04-20T10:24:19.672 INFO:teuthology.orchestra.run.smithi121.stdout:libtoolize: copying file './ltmain.sh' 2024-04-20T10:24:19.759 INFO:teuthology.orchestra.run.smithi121.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-04-20T10:24:19.759 INFO:teuthology.orchestra.run.smithi121.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-04-20T10:24:19.775 INFO:teuthology.orchestra.run.smithi121.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-04-20T10:24:19.792 INFO:teuthology.orchestra.run.smithi121.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-04-20T10:24:19.809 INFO:teuthology.orchestra.run.smithi121.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-04-20T10:24:19.825 INFO:teuthology.orchestra.run.smithi121.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-04-20T10:24:19.847 INFO:teuthology.orchestra.run.smithi121.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-04-20T10:24:19.851 INFO:teuthology.orchestra.run.smithi121.stdout:cp include/install-sh . 2024-04-20T10:24:19.854 INFO:teuthology.orchestra.run.smithi121.stdout:aclocal -I m4 2024-04-20T10:24:21.260 INFO:teuthology.orchestra.run.smithi121.stdout:autoheader 2024-04-20T10:24:21.667 INFO:teuthology.orchestra.run.smithi121.stdout:autoconf 2024-04-20T10:24:21.923 INFO:teuthology.orchestra.run.smithi121.stdout:./configure \ 2024-04-20T10:24:21.923 INFO:teuthology.orchestra.run.smithi121.stdout: --libexecdir=/usr/lib \ 2024-04-20T10:24:21.923 INFO:teuthology.orchestra.run.smithi121.stdout: --exec_prefix=/var/lib 2024-04-20T10:24:22.597 INFO:teuthology.orchestra.run.smithi121.stdout:checking build system type... x86_64-pc-linux-gnu 2024-04-20T10:24:22.598 INFO:teuthology.orchestra.run.smithi121.stdout:checking host system type... x86_64-pc-linux-gnu 2024-04-20T10:24:22.603 INFO:teuthology.orchestra.run.smithi121.stdout:checking how to print strings... printf 2024-04-20T10:24:22.603 INFO:teuthology.orchestra.run.smithi121.stdout:checking for gcc... gcc 2024-04-20T10:24:22.802 INFO:teuthology.orchestra.run.smithi121.stdout:checking whether the C compiler works... yes 2024-04-20T10:24:22.802 INFO:teuthology.orchestra.run.smithi121.stdout:checking for C compiler default output file name... a.out 2024-04-20T10:24:22.871 INFO:teuthology.orchestra.run.smithi121.stdout:checking for suffix of executables... 2024-04-20T10:24:23.058 INFO:teuthology.orchestra.run.smithi121.stdout:checking whether we are cross compiling... no 2024-04-20T10:24:23.113 INFO:teuthology.orchestra.run.smithi121.stdout:checking for suffix of object files... o 2024-04-20T10:24:23.158 INFO:teuthology.orchestra.run.smithi121.stdout:checking whether we are using the GNU C compiler... yes 2024-04-20T10:24:23.206 INFO:teuthology.orchestra.run.smithi121.stdout:checking whether gcc accepts -g... yes 2024-04-20T10:24:23.312 INFO:teuthology.orchestra.run.smithi121.stdout:checking for gcc option to accept ISO C89... none needed 2024-04-20T10:24:23.323 INFO:teuthology.orchestra.run.smithi121.stdout:checking for a sed that does not truncate output... /bin/sed 2024-04-20T10:24:23.333 INFO:teuthology.orchestra.run.smithi121.stdout:checking for grep that handles long lines and -e... /bin/grep 2024-04-20T10:24:23.338 INFO:teuthology.orchestra.run.smithi121.stdout:checking for egrep... /bin/grep -E 2024-04-20T10:24:23.343 INFO:teuthology.orchestra.run.smithi121.stdout:checking for fgrep... /bin/grep -F 2024-04-20T10:24:23.356 INFO:teuthology.orchestra.run.smithi121.stdout:checking for ld used by gcc... /bin/ld 2024-04-20T10:24:23.363 INFO:teuthology.orchestra.run.smithi121.stdout:checking if the linker (/bin/ld) is GNU ld... yes 2024-04-20T10:24:23.381 INFO:teuthology.orchestra.run.smithi121.stdout:checking for BSD- or MS-compatible name lister (nm)... /bin/nm -B 2024-04-20T10:24:24.398 INFO:teuthology.orchestra.run.smithi121.stdout:checking the name lister (/bin/nm -B) interface... BSD nm 2024-04-20T10:24:24.398 INFO:teuthology.orchestra.run.smithi121.stdout:checking whether ln -s works... yes 2024-04-20T10:24:24.441 INFO:teuthology.orchestra.run.smithi121.stdout:checking the maximum length of command line arguments... 1572864 2024-04-20T10:24:24.451 INFO:teuthology.orchestra.run.smithi121.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-04-20T10:24:24.451 INFO:teuthology.orchestra.run.smithi121.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-04-20T10:24:24.451 INFO:teuthology.orchestra.run.smithi121.stdout:checking for /bin/ld option to reload object files... -r 2024-04-20T10:24:24.452 INFO:teuthology.orchestra.run.smithi121.stdout:checking for objdump... objdump 2024-04-20T10:24:24.453 INFO:teuthology.orchestra.run.smithi121.stdout:checking how to recognize dependent libraries... pass_all 2024-04-20T10:24:24.455 INFO:teuthology.orchestra.run.smithi121.stdout:checking for dlltool... no 2024-04-20T10:24:24.455 INFO:teuthology.orchestra.run.smithi121.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-04-20T10:24:24.455 INFO:teuthology.orchestra.run.smithi121.stdout:checking for ar... ar 2024-04-20T10:24:24.557 INFO:teuthology.orchestra.run.smithi121.stdout:checking for archiver @FILE support... @ 2024-04-20T10:24:24.557 INFO:teuthology.orchestra.run.smithi121.stdout:checking for strip... strip 2024-04-20T10:24:24.558 INFO:teuthology.orchestra.run.smithi121.stdout:checking for ranlib... ranlib 2024-04-20T10:24:24.559 INFO:teuthology.orchestra.run.smithi121.stdout:checking for gawk... gawk 2024-04-20T10:24:24.731 INFO:teuthology.orchestra.run.smithi121.stdout:checking command to parse /bin/nm -B output from gcc object... ok 2024-04-20T10:24:24.735 INFO:teuthology.orchestra.run.smithi121.stdout:checking for sysroot... no 2024-04-20T10:24:24.746 INFO:teuthology.orchestra.run.smithi121.stdout:checking for a working dd... /bin/dd 2024-04-20T10:24:24.755 INFO:teuthology.orchestra.run.smithi121.stdout:checking how to truncate binary pipes... /bin/dd bs=4096 count=1 2024-04-20T10:24:24.795 INFO:teuthology.orchestra.run.smithi121.stdout:checking for mt... no 2024-04-20T10:24:24.803 INFO:teuthology.orchestra.run.smithi121.stdout:checking if : is a manifest tool... no 2024-04-20T10:24:24.898 INFO:teuthology.orchestra.run.smithi121.stdout:checking how to run the C preprocessor... gcc -E 2024-04-20T10:24:25.264 INFO:teuthology.orchestra.run.smithi121.stdout:checking for ANSI C header files... yes 2024-04-20T10:24:25.326 INFO:teuthology.orchestra.run.smithi121.stdout:checking for sys/types.h... yes 2024-04-20T10:24:25.398 INFO:teuthology.orchestra.run.smithi121.stdout:checking for sys/stat.h... yes 2024-04-20T10:24:25.465 INFO:teuthology.orchestra.run.smithi121.stdout:checking for stdlib.h... yes 2024-04-20T10:24:25.536 INFO:teuthology.orchestra.run.smithi121.stdout:checking for string.h... yes 2024-04-20T10:24:25.606 INFO:teuthology.orchestra.run.smithi121.stdout:checking for memory.h... yes 2024-04-20T10:24:25.674 INFO:teuthology.orchestra.run.smithi121.stdout:checking for strings.h... yes 2024-04-20T10:24:25.738 INFO:teuthology.orchestra.run.smithi121.stdout:checking for inttypes.h... yes 2024-04-20T10:24:25.808 INFO:teuthology.orchestra.run.smithi121.stdout:checking for stdint.h... yes 2024-04-20T10:24:25.888 INFO:teuthology.orchestra.run.smithi121.stdout:checking for unistd.h... yes 2024-04-20T10:24:25.956 INFO:teuthology.orchestra.run.smithi121.stdout:checking for dlfcn.h... yes 2024-04-20T10:24:25.968 INFO:teuthology.orchestra.run.smithi121.stdout:checking for objdir... .libs 2024-04-20T10:24:26.130 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:24:26.135 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:24:26.147 INFO:teuthology.orchestra.run.smithi121.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-04-20T10:24:26.147 INFO:teuthology.orchestra.run.smithi121.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-04-20T10:24:26.192 INFO:teuthology.orchestra.run.smithi121.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-04-20T10:24:26.241 INFO:teuthology.orchestra.run.smithi121.stdout:checking if gcc static flag -static works... no 2024-04-20T10:24:26.312 INFO:teuthology.orchestra.run.smithi121.stdout:checking if gcc supports -c -o file.o... yes 2024-04-20T10:24:26.312 INFO:teuthology.orchestra.run.smithi121.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-04-20T10:24:26.335 INFO:teuthology.orchestra.run.smithi121.stdout:checking whether the gcc linker (/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-04-20T10:24:26.405 INFO:teuthology.orchestra.run.smithi121.stdout:checking whether -lc should be explicitly linked in... no 2024-04-20T10:24:26.543 INFO:teuthology.orchestra.run.smithi121.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-04-20T10:24:26.543 INFO:teuthology.orchestra.run.smithi121.stdout:checking how to hardcode library paths into programs... immediate 2024-04-20T10:24:26.548 INFO:teuthology.orchestra.run.smithi121.stdout:checking whether stripping libraries is possible... yes 2024-04-20T10:24:26.548 INFO:teuthology.orchestra.run.smithi121.stdout:checking if libtool supports shared libraries... yes 2024-04-20T10:24:26.548 INFO:teuthology.orchestra.run.smithi121.stdout:checking whether to build shared libraries... yes 2024-04-20T10:24:26.548 INFO:teuthology.orchestra.run.smithi121.stdout:checking whether to build static libraries... yes 2024-04-20T10:24:26.569 INFO:teuthology.orchestra.run.smithi121.stdout:checking for gcc... (cached) gcc 2024-04-20T10:24:26.629 INFO:teuthology.orchestra.run.smithi121.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-04-20T10:24:26.629 INFO:teuthology.orchestra.run.smithi121.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-20T10:24:26.631 INFO:teuthology.orchestra.run.smithi121.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-04-20T10:24:26.631 INFO:teuthology.orchestra.run.smithi121.stdout:checking for gmake... /usr/bin/gmake 2024-04-20T10:24:26.632 INFO:teuthology.orchestra.run.smithi121.stderr:./configure: line 12082: test: $(SHELL): binary operator expected 2024-04-20T10:24:26.632 INFO:teuthology.orchestra.run.smithi121.stdout:checking for tar... /bin/tar 2024-04-20T10:24:26.633 INFO:teuthology.orchestra.run.smithi121.stdout:checking for gzip... /bin/gzip 2024-04-20T10:24:26.675 INFO:teuthology.orchestra.run.smithi121.stdout:checking whether gcc -MM is supported... yes 2024-04-20T10:24:26.676 INFO:teuthology.orchestra.run.smithi121.stdout:checking for sort... /bin/sort 2024-04-20T10:24:26.676 INFO:teuthology.orchestra.run.smithi121.stdout:checking whether ln -s works... yes 2024-04-20T10:24:26.676 INFO:teuthology.orchestra.run.smithi121.stdout:checking for rpm... /bin/rpm 2024-04-20T10:24:26.688 INFO:teuthology.orchestra.run.smithi121.stdout:checking for rpmbuild... no 2024-04-20T10:24:26.752 INFO:teuthology.orchestra.run.smithi121.stdout:checking assert.h usability... yes 2024-04-20T10:24:26.776 INFO:teuthology.orchestra.run.smithi121.stdout:checking assert.h presence... yes 2024-04-20T10:24:26.776 INFO:teuthology.orchestra.run.smithi121.stdout:checking for assert.h... yes 2024-04-20T10:24:26.847 INFO:teuthology.orchestra.run.smithi121.stdout:checking bstring.h usability... no 2024-04-20T10:24:26.884 INFO:teuthology.orchestra.run.smithi121.stdout:checking bstring.h presence... no 2024-04-20T10:24:26.884 INFO:teuthology.orchestra.run.smithi121.stdout:checking for bstring.h... no 2024-04-20T10:24:26.951 INFO:teuthology.orchestra.run.smithi121.stdout:checking libgen.h usability... yes 2024-04-20T10:24:26.975 INFO:teuthology.orchestra.run.smithi121.stdout:checking libgen.h presence... yes 2024-04-20T10:24:26.975 INFO:teuthology.orchestra.run.smithi121.stdout:checking for libgen.h... yes 2024-04-20T10:24:27.047 INFO:teuthology.orchestra.run.smithi121.stdout:checking dirent.h usability... yes 2024-04-20T10:24:27.075 INFO:teuthology.orchestra.run.smithi121.stdout:checking dirent.h presence... yes 2024-04-20T10:24:27.075 INFO:teuthology.orchestra.run.smithi121.stdout:checking for dirent.h... yes 2024-04-20T10:24:27.163 INFO:teuthology.orchestra.run.smithi121.stdout:checking errno.h usability... yes 2024-04-20T10:24:27.186 INFO:teuthology.orchestra.run.smithi121.stdout:checking errno.h presence... yes 2024-04-20T10:24:27.186 INFO:teuthology.orchestra.run.smithi121.stdout:checking for errno.h... yes 2024-04-20T10:24:27.256 INFO:teuthology.orchestra.run.smithi121.stdout:checking malloc.h usability... yes 2024-04-20T10:24:27.280 INFO:teuthology.orchestra.run.smithi121.stdout:checking malloc.h presence... yes 2024-04-20T10:24:27.280 INFO:teuthology.orchestra.run.smithi121.stdout:checking for malloc.h... yes 2024-04-20T10:24:27.345 INFO:teuthology.orchestra.run.smithi121.stdout:checking uuid.h usability... no 2024-04-20T10:24:27.374 INFO:teuthology.orchestra.run.smithi121.stdout:checking uuid.h presence... no 2024-04-20T10:24:27.374 INFO:teuthology.orchestra.run.smithi121.stdout:checking for uuid.h... no 2024-04-20T10:24:27.432 INFO:teuthology.orchestra.run.smithi121.stdout:checking uuid/uuid.h usability... yes 2024-04-20T10:24:27.461 INFO:teuthology.orchestra.run.smithi121.stdout:checking uuid/uuid.h presence... yes 2024-04-20T10:24:27.461 INFO:teuthology.orchestra.run.smithi121.stdout:checking for uuid/uuid.h... yes 2024-04-20T10:24:27.523 INFO:teuthology.orchestra.run.smithi121.stdout:checking sys/uuid.h usability... no 2024-04-20T10:24:27.551 INFO:teuthology.orchestra.run.smithi121.stdout:checking sys/uuid.h presence... no 2024-04-20T10:24:27.551 INFO:teuthology.orchestra.run.smithi121.stdout:checking for sys/uuid.h... no 2024-04-20T10:24:27.621 INFO:teuthology.orchestra.run.smithi121.stdout:checking sys/file.h usability... yes 2024-04-20T10:24:27.648 INFO:teuthology.orchestra.run.smithi121.stdout:checking sys/file.h presence... yes 2024-04-20T10:24:27.648 INFO:teuthology.orchestra.run.smithi121.stdout:checking for sys/file.h... yes 2024-04-20T10:24:27.741 INFO:teuthology.orchestra.run.smithi121.stdout:checking sys/fcntl.h usability... yes 2024-04-20T10:24:27.765 INFO:teuthology.orchestra.run.smithi121.stdout:checking sys/fcntl.h presence... yes 2024-04-20T10:24:27.765 INFO:teuthology.orchestra.run.smithi121.stdout:checking for sys/fcntl.h... yes 2024-04-20T10:24:27.829 INFO:teuthology.orchestra.run.smithi121.stdout:checking sys/syssgi.h usability... no 2024-04-20T10:24:27.864 INFO:teuthology.orchestra.run.smithi121.stdout:checking sys/syssgi.h presence... no 2024-04-20T10:24:27.864 INFO:teuthology.orchestra.run.smithi121.stdout:checking for sys/syssgi.h... no 2024-04-20T10:24:27.949 INFO:teuthology.orchestra.run.smithi121.stdout:checking sys/param.h usability... yes 2024-04-20T10:24:27.984 INFO:teuthology.orchestra.run.smithi121.stdout:checking sys/param.h presence... yes 2024-04-20T10:24:27.984 INFO:teuthology.orchestra.run.smithi121.stdout:checking for sys/param.h... yes 2024-04-20T10:24:27.998 INFO:teuthology.orchestra.run.smithi121.stdout:checking for sys/stat.h... (cached) yes 2024-04-20T10:24:28.061 INFO:teuthology.orchestra.run.smithi121.stdout:checking sys/statvfs.h usability... yes 2024-04-20T10:24:28.086 INFO:teuthology.orchestra.run.smithi121.stdout:checking sys/statvfs.h presence... yes 2024-04-20T10:24:28.086 INFO:teuthology.orchestra.run.smithi121.stdout:checking for sys/statvfs.h... yes 2024-04-20T10:24:28.152 INFO:teuthology.orchestra.run.smithi121.stdout:checking sys/time.h usability... yes 2024-04-20T10:24:28.180 INFO:teuthology.orchestra.run.smithi121.stdout:checking sys/time.h presence... yes 2024-04-20T10:24:28.180 INFO:teuthology.orchestra.run.smithi121.stdout:checking for sys/time.h... yes 2024-04-20T10:24:28.258 INFO:teuthology.orchestra.run.smithi121.stdout:checking sys/ioctl.h usability... yes 2024-04-20T10:24:28.279 INFO:teuthology.orchestra.run.smithi121.stdout:checking sys/ioctl.h presence... yes 2024-04-20T10:24:28.279 INFO:teuthology.orchestra.run.smithi121.stdout:checking for sys/ioctl.h... yes 2024-04-20T10:24:28.348 INFO:teuthology.orchestra.run.smithi121.stdout:checking sys/wait.h usability... yes 2024-04-20T10:24:28.378 INFO:teuthology.orchestra.run.smithi121.stdout:checking sys/wait.h presence... yes 2024-04-20T10:24:28.378 INFO:teuthology.orchestra.run.smithi121.stdout:checking for sys/wait.h... yes 2024-04-20T10:24:28.394 INFO:teuthology.orchestra.run.smithi121.stdout:checking for sys/types.h... (cached) yes 2024-04-20T10:24:28.408 INFO:teuthology.orchestra.run.smithi121.stdout:checking for strings.h... (cached) yes 2024-04-20T10:24:28.480 INFO:teuthology.orchestra.run.smithi121.stdout:checking err.h usability... yes 2024-04-20T10:24:28.507 INFO:teuthology.orchestra.run.smithi121.stdout:checking err.h presence... yes 2024-04-20T10:24:28.507 INFO:teuthology.orchestra.run.smithi121.stdout:checking for err.h... yes 2024-04-20T10:24:28.586 INFO:teuthology.orchestra.run.smithi121.stdout:checking linux/falloc.h usability... yes 2024-04-20T10:24:28.613 INFO:teuthology.orchestra.run.smithi121.stdout:checking linux/falloc.h presence... yes 2024-04-20T10:24:28.613 INFO:teuthology.orchestra.run.smithi121.stdout:checking for linux/falloc.h... yes 2024-04-20T10:24:28.689 INFO:teuthology.orchestra.run.smithi121.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-04-20T10:24:28.730 INFO:teuthology.orchestra.run.smithi121.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-04-20T10:24:28.730 INFO:teuthology.orchestra.run.smithi121.stdout:checking for sys/fs/xfs_fsops.h... no 2024-04-20T10:24:28.800 INFO:teuthology.orchestra.run.smithi121.stdout:checking sys/fs/xfs_itable.h usability... no 2024-04-20T10:24:28.839 INFO:teuthology.orchestra.run.smithi121.stdout:checking sys/fs/xfs_itable.h presence... no 2024-04-20T10:24:28.839 INFO:teuthology.orchestra.run.smithi121.stdout:checking for sys/fs/xfs_itable.h... no 2024-04-20T10:24:28.907 INFO:teuthology.orchestra.run.smithi121.stdout:checking xfs/platform_defs.h usability... no 2024-04-20T10:24:28.947 INFO:teuthology.orchestra.run.smithi121.stdout:checking xfs/platform_defs.h presence... no 2024-04-20T10:24:28.947 INFO:teuthology.orchestra.run.smithi121.stdout:checking for xfs/platform_defs.h... no 2024-04-20T10:24:29.023 INFO:teuthology.orchestra.run.smithi121.stdout:checking btrfs/ioctl.h usability... no 2024-04-20T10:24:29.063 INFO:teuthology.orchestra.run.smithi121.stdout:checking btrfs/ioctl.h presence... no 2024-04-20T10:24:29.064 INFO:teuthology.orchestra.run.smithi121.stdout:checking for btrfs/ioctl.h... no 2024-04-20T10:24:29.137 INFO:teuthology.orchestra.run.smithi121.stdout:checking cifs/ioctl.h usability... no 2024-04-20T10:24:29.178 INFO:teuthology.orchestra.run.smithi121.stdout:checking cifs/ioctl.h presence... no 2024-04-20T10:24:29.178 INFO:teuthology.orchestra.run.smithi121.stdout:checking for cifs/ioctl.h... no 2024-04-20T10:24:29.257 INFO:teuthology.orchestra.run.smithi121.stdout:checking sys/mman.h usability... yes 2024-04-20T10:24:29.285 INFO:teuthology.orchestra.run.smithi121.stdout:checking sys/mman.h presence... yes 2024-04-20T10:24:29.286 INFO:teuthology.orchestra.run.smithi121.stdout:checking for sys/mman.h... yes 2024-04-20T10:24:29.368 INFO:teuthology.orchestra.run.smithi121.stdout:checking linux/ext4.h usability... no 2024-04-20T10:24:29.408 INFO:teuthology.orchestra.run.smithi121.stdout:checking linux/ext4.h presence... no 2024-04-20T10:24:29.408 INFO:teuthology.orchestra.run.smithi121.stdout:checking for linux/ext4.h... no 2024-04-20T10:24:29.456 INFO:teuthology.orchestra.run.smithi121.stdout:checking for xfs/xfs_log_format.h... no 2024-04-20T10:24:29.562 INFO:teuthology.orchestra.run.smithi121.stdout:checking for uuid_compare... no 2024-04-20T10:24:29.639 INFO:teuthology.orchestra.run.smithi121.stdout:checking for uuid_compare in -luuid... yes 2024-04-20T10:24:29.751 INFO:teuthology.orchestra.run.smithi121.stdout:checking for xfs/xfs.h... yes 2024-04-20T10:24:29.794 INFO:teuthology.orchestra.run.smithi121.stdout:checking for xfs/libxfs.h... no 2024-04-20T10:24:29.849 INFO:teuthology.orchestra.run.smithi121.stdout:checking whether xlog_assign_lsn is declared... no 2024-04-20T10:24:29.922 INFO:teuthology.orchestra.run.smithi121.stdout:checking for xfs/xqm.h... yes 2024-04-20T10:24:30.031 INFO:teuthology.orchestra.run.smithi121.stdout:checking xfsctl from xfs/xfs.h... ok 2024-04-20T10:24:30.094 INFO:teuthology.orchestra.run.smithi121.stdout:checking xfs/handle.h usability... yes 2024-04-20T10:24:30.119 INFO:teuthology.orchestra.run.smithi121.stdout:checking xfs/handle.h presence... yes 2024-04-20T10:24:30.119 INFO:teuthology.orchestra.run.smithi121.stdout:checking for xfs/handle.h... yes 2024-04-20T10:24:30.218 INFO:teuthology.orchestra.run.smithi121.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-04-20T10:24:30.286 INFO:teuthology.orchestra.run.smithi121.stdout:checking sys/acl.h usability... yes 2024-04-20T10:24:30.318 INFO:teuthology.orchestra.run.smithi121.stdout:checking sys/acl.h presence... yes 2024-04-20T10:24:30.318 INFO:teuthology.orchestra.run.smithi121.stdout:checking for sys/acl.h... yes 2024-04-20T10:24:30.392 INFO:teuthology.orchestra.run.smithi121.stdout:checking acl/libacl.h usability... yes 2024-04-20T10:24:30.421 INFO:teuthology.orchestra.run.smithi121.stdout:checking acl/libacl.h presence... yes 2024-04-20T10:24:30.422 INFO:teuthology.orchestra.run.smithi121.stdout:checking for acl/libacl.h... yes 2024-04-20T10:24:30.505 INFO:teuthology.orchestra.run.smithi121.stdout:checking for acl_init in -lacl... yes 2024-04-20T10:24:30.571 INFO:teuthology.orchestra.run.smithi121.stdout:checking gdbm-ndbm.h usability... no 2024-04-20T10:24:30.612 INFO:teuthology.orchestra.run.smithi121.stdout:checking gdbm-ndbm.h presence... no 2024-04-20T10:24:30.613 INFO:teuthology.orchestra.run.smithi121.stdout:checking for gdbm-ndbm.h... no 2024-04-20T10:24:30.676 INFO:teuthology.orchestra.run.smithi121.stdout:checking gdbm/ndbm.h usability... yes 2024-04-20T10:24:30.705 INFO:teuthology.orchestra.run.smithi121.stdout:checking gdbm/ndbm.h presence... yes 2024-04-20T10:24:30.705 INFO:teuthology.orchestra.run.smithi121.stdout:checking for gdbm/ndbm.h... yes 2024-04-20T10:24:30.795 INFO:teuthology.orchestra.run.smithi121.stdout:checking for dbm_open in -lgdbm... no 2024-04-20T10:24:30.885 INFO:teuthology.orchestra.run.smithi121.stdout:checking for dbm_fetch in -lgdbm... no 2024-04-20T10:24:30.978 INFO:teuthology.orchestra.run.smithi121.stdout:checking for dbm_store in -lgdbm... no 2024-04-20T10:24:31.064 INFO:teuthology.orchestra.run.smithi121.stdout:checking for dbm_close in -lgdbm... no 2024-04-20T10:24:31.159 INFO:teuthology.orchestra.run.smithi121.stdout:checking for dbm_open in -lgdbm_compat... yes 2024-04-20T10:24:31.244 INFO:teuthology.orchestra.run.smithi121.stdout:checking for dbm_fetch in -lgdbm_compat... yes 2024-04-20T10:24:31.327 INFO:teuthology.orchestra.run.smithi121.stdout:checking for dbm_store in -lgdbm_compat... yes 2024-04-20T10:24:31.412 INFO:teuthology.orchestra.run.smithi121.stdout:checking for dbm_close in -lgdbm_compat... yes 2024-04-20T10:24:31.473 INFO:teuthology.orchestra.run.smithi121.stdout:checking libaio.h usability... yes 2024-04-20T10:24:31.505 INFO:teuthology.orchestra.run.smithi121.stdout:checking libaio.h presence... yes 2024-04-20T10:24:31.506 INFO:teuthology.orchestra.run.smithi121.stdout:checking for libaio.h... yes 2024-04-20T10:24:31.575 INFO:teuthology.orchestra.run.smithi121.stdout:checking liburing.h usability... no 2024-04-20T10:24:31.615 INFO:teuthology.orchestra.run.smithi121.stdout:checking liburing.h presence... no 2024-04-20T10:24:31.616 INFO:teuthology.orchestra.run.smithi121.stdout:checking for liburing.h... no 2024-04-20T10:24:31.680 INFO:teuthology.orchestra.run.smithi121.stdout:checking sys/capability.h usability... yes 2024-04-20T10:24:31.712 INFO:teuthology.orchestra.run.smithi121.stdout:checking sys/capability.h presence... yes 2024-04-20T10:24:31.712 INFO:teuthology.orchestra.run.smithi121.stdout:checking for sys/capability.h... yes 2024-04-20T10:24:31.781 INFO:teuthology.orchestra.run.smithi121.stdout:checking linux/fiemap.h usability... yes 2024-04-20T10:24:31.808 INFO:teuthology.orchestra.run.smithi121.stdout:checking linux/fiemap.h presence... yes 2024-04-20T10:24:31.808 INFO:teuthology.orchestra.run.smithi121.stdout:checking for linux/fiemap.h... yes 2024-04-20T10:24:31.896 INFO:teuthology.orchestra.run.smithi121.stdout:checking for fallocate... yes 2024-04-20T10:24:31.978 INFO:teuthology.orchestra.run.smithi121.stdout:checking for open_by_handle_at... yes 2024-04-20T10:24:32.041 INFO:teuthology.orchestra.run.smithi121.stdout:checking linux/fs.h usability... yes 2024-04-20T10:24:32.067 INFO:teuthology.orchestra.run.smithi121.stdout:checking linux/fs.h presence... yes 2024-04-20T10:24:32.067 INFO:teuthology.orchestra.run.smithi121.stdout:checking for linux/fs.h... yes 2024-04-20T10:24:32.128 INFO:teuthology.orchestra.run.smithi121.stdout:checking btrfsutil.h usability... no 2024-04-20T10:24:32.169 INFO:teuthology.orchestra.run.smithi121.stdout:checking btrfsutil.h presence... no 2024-04-20T10:24:32.169 INFO:teuthology.orchestra.run.smithi121.stdout:checking for btrfsutil.h... no 2024-04-20T10:24:32.259 INFO:teuthology.orchestra.run.smithi121.stdout:checking for copy_file_range... yes 2024-04-20T10:24:32.349 INFO:teuthology.orchestra.run.smithi121.stdout:checking for SEEK_DATA... yes 2024-04-20T10:24:32.455 INFO:teuthology.orchestra.run.smithi121.stdout:checking for BMV_OF_SHARED... yes 2024-04-20T10:24:32.545 INFO:teuthology.orchestra.run.smithi121.stdout:checking for nftw... yes 2024-04-20T10:24:32.637 INFO:teuthology.orchestra.run.smithi121.stdout:checking for RLIMIT_NOFILE... yes 2024-04-20T10:24:32.715 INFO:teuthology.orchestra.run.smithi121.stdout:checking for XFS_IOC_EXCHANGE_RANGE... no 2024-04-20T10:24:32.805 INFO:teuthology.orchestra.run.smithi121.stdout:checking for FICLONE... yes 2024-04-20T10:24:32.886 INFO:teuthology.orchestra.run.smithi121.stdout:checking for renameat2... yes 2024-04-20T10:24:32.961 INFO:teuthology.orchestra.run.smithi121.stdout:checking for reallocarray... yes 2024-04-20T10:24:33.051 INFO:teuthology.orchestra.run.smithi121.stdout:checking for struct mount_attr... yes 2024-04-20T10:24:33.155 INFO:teuthology.orchestra.run.smithi121.stdout:checking for struct btrfs_qgroup_limit... yes 2024-04-20T10:24:33.260 INFO:teuthology.orchestra.run.smithi121.stdout:checking for struct btrfs_qgroup_inherit... yes 2024-04-20T10:24:33.364 INFO:teuthology.orchestra.run.smithi121.stdout:checking for struct btrfs_ioctl_vol_args... yes 2024-04-20T10:24:33.464 INFO:teuthology.orchestra.run.smithi121.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2024-04-20T10:24:33.569 INFO:teuthology.orchestra.run.smithi121.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2024-04-20T10:24:33.667 INFO:teuthology.orchestra.run.smithi121.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2024-04-20T10:24:33.769 INFO:teuthology.orchestra.run.smithi121.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2024-04-20T10:24:33.847 INFO:teuthology.orchestra.run.smithi121.stdout:checking linux/btrfs.h usability... yes 2024-04-20T10:24:33.875 INFO:teuthology.orchestra.run.smithi121.stdout:checking linux/btrfs.h presence... yes 2024-04-20T10:24:33.875 INFO:teuthology.orchestra.run.smithi121.stdout:checking for linux/btrfs.h... yes 2024-04-20T10:24:33.946 INFO:teuthology.orchestra.run.smithi121.stdout:checking linux/btrfs_tree.h usability... yes 2024-04-20T10:24:33.974 INFO:teuthology.orchestra.run.smithi121.stdout:checking linux/btrfs_tree.h presence... yes 2024-04-20T10:24:33.974 INFO:teuthology.orchestra.run.smithi121.stdout:checking for linux/btrfs_tree.h... yes 2024-04-20T10:24:34.038 INFO:teuthology.orchestra.run.smithi121.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... yes 2024-04-20T10:24:34.102 INFO:teuthology.orchestra.run.smithi121.stdout:checking whether BTRFS_IOC_SNAP_DESTROY_V2 is declared... yes 2024-04-20T10:24:34.142 INFO:teuthology.orchestra.run.smithi121.stdout:configure: creating ./config.status 2024-04-20T10:24:35.317 INFO:teuthology.orchestra.run.smithi121.stdout:config.status: creating include/builddefs 2024-04-20T10:24:35.358 INFO:teuthology.orchestra.run.smithi121.stdout:config.status: creating include/config.h 2024-04-20T10:24:35.388 INFO:teuthology.orchestra.run.smithi121.stdout:config.status: executing libtool commands 2024-04-20T10:24:35.492 INFO:teuthology.orchestra.run.smithi121.stdout:Building include 2024-04-20T10:24:35.503 INFO:teuthology.orchestra.run.smithi121.stdout:Building lib 2024-04-20T10:24:35.512 INFO:teuthology.orchestra.run.smithi121.stdout: [LTDEP] 2024-04-20T10:24:35.835 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] dataascii.lo 2024-04-20T10:24:36.108 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] databin.lo 2024-04-20T10:24:36.384 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] datapid.lo 2024-04-20T10:24:36.612 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] file_lock.lo 2024-04-20T10:24:36.906 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] forker.lo 2024-04-20T10:24:37.185 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] pattern.lo 2024-04-20T10:24:37.441 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] open_flags.lo 2024-04-20T10:24:37.732 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] random_range.lo 2024-04-20T10:24:38.096 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] string_to_tokens.lo 2024-04-20T10:24:38.321 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] str_to_bytes.lo 2024-04-20T10:24:38.591 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] tlibio.lo 2024-04-20T10:24:38.978 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] write_log.lo 2024-04-20T10:24:39.372 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] random.lo 2024-04-20T10:24:39.617 INFO:teuthology.orchestra.run.smithi121.stdout: [LD] 2024-04-20T10:24:39.830 INFO:teuthology.orchestra.run.smithi121.stdout:Building ltp 2024-04-20T10:24:39.840 INFO:teuthology.orchestra.run.smithi121.stdout: [DEP] 2024-04-20T10:24:40.088 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] doio 2024-04-20T10:24:40.962 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] fsstress 2024-04-20T10:24:42.539 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] fsx 2024-04-20T10:24:43.493 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] growfiles 2024-04-20T10:24:44.217 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] iogen 2024-04-20T10:24:44.780 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] aio-stress 2024-04-20T10:24:45.388 INFO:teuthology.orchestra.run.smithi121.stdout:Building src 2024-04-20T10:24:45.401 INFO:teuthology.orchestra.run.smithi121.stdout: [DEP] 2024-04-20T10:24:49.268 INFO:teuthology.orchestra.run.smithi121.stdout: [DEP] 2024-04-20T10:24:49.345 INFO:teuthology.orchestra.run.smithi121.stdout: [DEP] 2024-04-20T10:24:49.365 INFO:teuthology.orchestra.run.smithi121.stdout: [DEP] 2024-04-20T10:24:49.845 INFO:teuthology.orchestra.run.smithi121.stdout: [DEP] 2024-04-20T10:24:49.900 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] dirstress 2024-04-20T10:24:50.302 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] fill 2024-04-20T10:24:50.610 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] fill2 2024-04-20T10:24:51.010 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] getpagesize 2024-04-20T10:24:51.278 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] holes 2024-04-20T10:24:51.652 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] lstat64 2024-04-20T10:24:51.989 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] nametest 2024-04-20T10:24:52.418 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] permname 2024-04-20T10:24:52.796 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] randholes 2024-04-20T10:24:53.231 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] runas 2024-04-20T10:24:53.563 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] truncfile 2024-04-20T10:24:53.900 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] usemem 2024-04-20T10:24:54.242 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] mmapcat 2024-04-20T10:24:54.557 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] append_reader 2024-04-20T10:24:54.881 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] append_writer 2024-04-20T10:24:55.194 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] dirperf 2024-04-20T10:24:55.578 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] metaperf 2024-04-20T10:24:56.028 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] devzero 2024-04-20T10:24:56.259 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:24:56.267 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:24:56.371 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] feature 2024-04-20T10:24:56.756 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] alloc 2024-04-20T10:24:57.147 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] fault 2024-04-20T10:24:57.485 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] fstest 2024-04-20T10:24:57.929 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] t_access_root 2024-04-20T10:24:58.249 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] godown 2024-04-20T10:24:58.586 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] resvtest 2024-04-20T10:24:58.946 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] writemod 2024-04-20T10:24:59.283 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] writev_on_pagefault 2024-04-20T10:24:59.593 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] makeextents 2024-04-20T10:24:59.914 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] itrash 2024-04-20T10:25:00.252 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] rename 2024-04-20T10:25:00.570 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] multi_open_unlink 2024-04-20T10:25:00.893 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] unwritten_sync 2024-04-20T10:25:01.258 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] genhashnames 2024-04-20T10:25:01.621 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] t_holes 2024-04-20T10:25:01.925 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] t_mmap_writev 2024-04-20T10:25:02.248 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] t_truncate_cmtime 2024-04-20T10:25:02.575 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] dirhash_collide 2024-04-20T10:25:02.943 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] t_rename_overwrite 2024-04-20T10:25:03.253 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] holetest 2024-04-20T10:25:03.670 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] t_truncate_self 2024-04-20T10:25:03.957 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] af_unix 2024-04-20T10:25:04.294 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] t_mmap_stale_pmd 2024-04-20T10:25:04.640 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] t_mmap_cow_race 2024-04-20T10:25:04.956 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] t_mmap_fallocate 2024-04-20T10:25:05.269 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] fsync-err 2024-04-20T10:25:05.633 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] t_mmap_write_ro 2024-04-20T10:25:05.960 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] t_ext4_dax_journal_corruption 2024-04-20T10:25:06.297 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] t_ext4_dax_inline_corruption 2024-04-20T10:25:06.622 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] t_ofd_locks 2024-04-20T10:25:06.977 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] t_mmap_collision 2024-04-20T10:25:07.372 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] mmap-write-concurrent 2024-04-20T10:25:07.725 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] t_get_file_time 2024-04-20T10:25:08.050 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] t_create_short_dirs 2024-04-20T10:25:08.397 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] t_create_long_dirs 2024-04-20T10:25:08.732 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] t_enospc 2024-04-20T10:25:09.113 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] t_mmap_writev_overlap 2024-04-20T10:25:09.466 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] checkpoint_journal 2024-04-20T10:25:09.797 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] mmap-rw-fault 2024-04-20T10:25:10.158 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] allocstale 2024-04-20T10:25:10.497 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] t_mmap_cow_memory_failure 2024-04-20T10:25:10.815 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] fake-dump-rootino 2024-04-20T10:25:11.140 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] dio-buf-fault 2024-04-20T10:25:11.463 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] rewinddir-test 2024-04-20T10:25:11.803 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] readdir-while-renames 2024-04-20T10:25:12.116 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] xfsctl 2024-04-20T10:25:12.478 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] bstat 2024-04-20T10:25:12.845 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] t_mtab 2024-04-20T10:25:13.214 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] getdevicesize 2024-04-20T10:25:13.498 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] preallo_rw_pattern_reader 2024-04-20T10:25:13.839 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] preallo_rw_pattern_writer 2024-04-20T10:25:14.179 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] ftrunc 2024-04-20T10:25:14.465 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] trunc 2024-04-20T10:25:14.770 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] fs_perms 2024-04-20T10:25:15.106 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] testx 2024-04-20T10:25:15.412 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] looptest 2024-04-20T10:25:15.760 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] locktest 2024-04-20T10:25:16.267 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] unwritten_mmap 2024-04-20T10:25:16.624 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] bulkstat_unlink_test 2024-04-20T10:25:17.012 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] deduperace 2024-04-20T10:25:17.384 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] bulkstat_unlink_test_modified 2024-04-20T10:25:17.737 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] t_dir_offset 2024-04-20T10:25:18.021 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] t_futimens 2024-04-20T10:25:18.331 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] t_immutable 2024-04-20T10:25:19.156 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] stale_handle 2024-04-20T10:25:19.504 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] pwrite_mmap_blocked 2024-04-20T10:25:19.814 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] t_dir_offset2 2024-04-20T10:25:20.135 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] seek_sanity_test 2024-04-20T10:25:20.743 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] seek_copy_test 2024-04-20T10:25:21.088 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] t_readdir_1 2024-04-20T10:25:21.424 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] t_readdir_2 2024-04-20T10:25:21.748 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] fsync-tester 2024-04-20T10:25:22.193 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] nsexec 2024-04-20T10:25:22.540 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] cloner 2024-04-20T10:25:22.893 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] renameat2 2024-04-20T10:25:23.218 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] t_getcwd 2024-04-20T10:25:23.549 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] e4compact 2024-04-20T10:25:23.936 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] test-nextquota 2024-04-20T10:25:24.281 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] punch-alternating 2024-04-20T10:25:24.620 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] attr-list-by-handle-cursor-test 2024-04-20T10:25:24.966 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] listxattr 2024-04-20T10:25:25.256 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] dio-interleaved 2024-04-20T10:25:25.599 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] t_dir_type 2024-04-20T10:25:25.912 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] dio-invalidate-cache 2024-04-20T10:25:26.306 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] stat_test 2024-04-20T10:25:26.380 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:25:26.386 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:25:26.757 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] t_encrypted_d_revalidate 2024-04-20T10:25:27.094 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] attr_replace_test 2024-04-20T10:25:27.438 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] swapon 2024-04-20T10:25:27.797 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] mkswap 2024-04-20T10:25:28.107 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] t_attr_corruption 2024-04-20T10:25:28.446 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] t_open_tmpfiles 2024-04-20T10:25:28.792 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] fscrypt-crypt-util 2024-04-20T10:25:29.640 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] bulkstat_null_ocount 2024-04-20T10:25:29.970 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] splice-test 2024-04-20T10:25:30.332 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] chprojid_fail 2024-04-20T10:25:30.665 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] detached_mounts_propagation 2024-04-20T10:25:31.001 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] ext4_resize 2024-04-20T10:25:31.327 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] t_readdir_3 2024-04-20T10:25:31.707 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] splice2pipe 2024-04-20T10:25:32.044 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] uuid_ioctl 2024-04-20T10:25:32.351 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] t_snapshot_deleted_subvolume 2024-04-20T10:25:32.694 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] fiemap-fault 2024-04-20T10:25:33.008 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] fiemap-tester 2024-04-20T10:25:33.442 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] t_stripealign 2024-04-20T10:25:33.755 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] open_by_handle 2024-04-20T10:25:34.172 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] dbtest 2024-04-20T10:25:34.613 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] t_mmap_dio 2024-04-20T10:25:34.931 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] xfsfind 2024-04-20T10:25:35.278 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] t_reflink_read_race 2024-04-20T10:25:35.649 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] fssum 2024-04-20T10:25:36.289 INFO:teuthology.orchestra.run.smithi121.stdout:Building log-writes 2024-04-20T10:25:36.303 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] replay-log 2024-04-20T10:25:36.831 INFO:teuthology.orchestra.run.smithi121.stdout:Building perf 2024-04-20T10:25:36.841 INFO:teuthology.orchestra.run.smithi121.stdout:Building aio-dio-regress 2024-04-20T10:25:36.855 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] aiocp 2024-04-20T10:25:37.251 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] aio-dio-append-write-fallocate-race 2024-04-20T10:25:37.571 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] aio-dio-append-write-read-race 2024-04-20T10:25:37.883 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] aio-dio-cow-race 2024-04-20T10:25:38.183 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] aio-dio-cycle-write 2024-04-20T10:25:38.547 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] aio-dio-eof-race 2024-04-20T10:25:38.925 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] aio-dio-extend-stat 2024-04-20T10:25:39.262 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] aio-dio-fcntl-race 2024-04-20T10:25:39.591 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] aio-dio-hole-filling-race 2024-04-20T10:25:39.921 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] aio-dio-invalidate-failure 2024-04-20T10:25:40.257 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] aio-dio-invalidate-readahead 2024-04-20T10:25:40.597 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] aiodio_sparse2 2024-04-20T10:25:40.994 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] aio-dio-subblock-eof-read 2024-04-20T10:25:41.288 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] aio-dio-write-verify 2024-04-20T10:25:41.668 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2024-04-20T10:25:41.977 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2024-04-20T10:25:42.284 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] aio-last-ref-held-by-io 2024-04-20T10:25:42.597 INFO:teuthology.orchestra.run.smithi121.stdout:Building vfs 2024-04-20T10:25:42.610 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] vfstest 2024-04-20T10:25:46.582 INFO:teuthology.orchestra.run.smithi121.stdout: [CC] mount-idmapped 2024-04-20T10:25:47.144 INFO:teuthology.orchestra.run.smithi121.stdout:Building m4 2024-04-20T10:25:47.153 INFO:teuthology.orchestra.run.smithi121.stdout:Building common 2024-04-20T10:25:47.163 INFO:teuthology.orchestra.run.smithi121.stdout:Building tests 2024-04-20T10:25:47.178 INFO:teuthology.orchestra.run.smithi121.stdout:Building btrfs 2024-04-20T10:25:47.194 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/btrfs/group.list 2024-04-20T10:25:47.400 INFO:teuthology.orchestra.run.smithi121.stdout:Building ceph 2024-04-20T10:25:47.413 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/ceph/group.list 2024-04-20T10:25:47.465 INFO:teuthology.orchestra.run.smithi121.stdout:Building cifs 2024-04-20T10:25:47.480 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/cifs/group.list 2024-04-20T10:25:47.526 INFO:teuthology.orchestra.run.smithi121.stdout:Building ext4 2024-04-20T10:25:47.541 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/ext4/group.list 2024-04-20T10:25:47.679 INFO:teuthology.orchestra.run.smithi121.stdout:Building f2fs 2024-04-20T10:25:47.694 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/f2fs/group.list 2024-04-20T10:25:47.757 INFO:teuthology.orchestra.run.smithi121.stdout:Building generic 2024-04-20T10:25:47.775 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/generic/group.list 2024-04-20T10:25:48.129 INFO:teuthology.orchestra.run.smithi121.stdout:Building nfs 2024-04-20T10:25:48.144 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/nfs/group.list 2024-04-20T10:25:48.196 INFO:teuthology.orchestra.run.smithi121.stdout:Building ocfs2 2024-04-20T10:25:48.210 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/ocfs2/group.list 2024-04-20T10:25:48.258 INFO:teuthology.orchestra.run.smithi121.stdout:Building overlay 2024-04-20T10:25:48.272 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/overlay/group.list 2024-04-20T10:25:48.421 INFO:teuthology.orchestra.run.smithi121.stdout:Building perf 2024-04-20T10:25:48.434 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/perf/group.list 2024-04-20T10:25:48.478 INFO:teuthology.orchestra.run.smithi121.stdout:Building selftest 2024-04-20T10:25:48.493 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/selftest/group.list 2024-04-20T10:25:48.536 INFO:teuthology.orchestra.run.smithi121.stdout:Building shared 2024-04-20T10:25:48.549 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/shared/group.list 2024-04-20T10:25:48.611 INFO:teuthology.orchestra.run.smithi121.stdout:Building tmpfs 2024-04-20T10:25:48.625 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/tmpfs/group.list 2024-04-20T10:25:48.673 INFO:teuthology.orchestra.run.smithi121.stdout:Building udf 2024-04-20T10:25:48.687 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/udf/group.list 2024-04-20T10:25:48.748 INFO:teuthology.orchestra.run.smithi121.stdout:Building xfs 2024-04-20T10:25:48.761 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/xfs/group.list 2024-04-20T10:25:49.078 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo make install) 2024-04-20T10:25:49.191 INFO:teuthology.orchestra.run.smithi121.stdout:Building include 2024-04-20T10:25:49.202 INFO:teuthology.orchestra.run.smithi121.stdout:Building lib 2024-04-20T10:25:49.212 INFO:teuthology.orchestra.run.smithi121.stdout:Building ltp 2024-04-20T10:25:49.223 INFO:teuthology.orchestra.run.smithi121.stdout:Building src 2024-04-20T10:25:49.302 INFO:teuthology.orchestra.run.smithi121.stdout:Building log-writes 2024-04-20T10:25:49.312 INFO:teuthology.orchestra.run.smithi121.stdout:Building perf 2024-04-20T10:25:49.322 INFO:teuthology.orchestra.run.smithi121.stdout:Building aio-dio-regress 2024-04-20T10:25:49.334 INFO:teuthology.orchestra.run.smithi121.stdout:Building vfs 2024-04-20T10:25:49.345 INFO:teuthology.orchestra.run.smithi121.stdout:Building m4 2024-04-20T10:25:49.355 INFO:teuthology.orchestra.run.smithi121.stdout:Building common 2024-04-20T10:25:49.365 INFO:teuthology.orchestra.run.smithi121.stdout:Building tests 2024-04-20T10:25:49.379 INFO:teuthology.orchestra.run.smithi121.stdout:Building btrfs 2024-04-20T10:25:49.395 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/btrfs/group.list 2024-04-20T10:25:49.613 INFO:teuthology.orchestra.run.smithi121.stdout:Building ceph 2024-04-20T10:25:49.627 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/ceph/group.list 2024-04-20T10:25:49.679 INFO:teuthology.orchestra.run.smithi121.stdout:Building cifs 2024-04-20T10:25:49.694 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/cifs/group.list 2024-04-20T10:25:49.739 INFO:teuthology.orchestra.run.smithi121.stdout:Building ext4 2024-04-20T10:25:49.754 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/ext4/group.list 2024-04-20T10:25:49.890 INFO:teuthology.orchestra.run.smithi121.stdout:Building f2fs 2024-04-20T10:25:49.906 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/f2fs/group.list 2024-04-20T10:25:49.970 INFO:teuthology.orchestra.run.smithi121.stdout:Building generic 2024-04-20T10:25:49.988 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/generic/group.list 2024-04-20T10:25:50.329 INFO:teuthology.orchestra.run.smithi121.stdout:Building nfs 2024-04-20T10:25:50.343 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/nfs/group.list 2024-04-20T10:25:50.395 INFO:teuthology.orchestra.run.smithi121.stdout:Building ocfs2 2024-04-20T10:25:50.410 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/ocfs2/group.list 2024-04-20T10:25:50.458 INFO:teuthology.orchestra.run.smithi121.stdout:Building overlay 2024-04-20T10:25:50.473 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/overlay/group.list 2024-04-20T10:25:50.621 INFO:teuthology.orchestra.run.smithi121.stdout:Building perf 2024-04-20T10:25:50.636 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/perf/group.list 2024-04-20T10:25:50.676 INFO:teuthology.orchestra.run.smithi121.stdout:Building selftest 2024-04-20T10:25:50.689 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/selftest/group.list 2024-04-20T10:25:50.731 INFO:teuthology.orchestra.run.smithi121.stdout:Building shared 2024-04-20T10:25:50.746 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/shared/group.list 2024-04-20T10:25:50.809 INFO:teuthology.orchestra.run.smithi121.stdout:Building tmpfs 2024-04-20T10:25:50.824 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/tmpfs/group.list 2024-04-20T10:25:50.874 INFO:teuthology.orchestra.run.smithi121.stdout:Building udf 2024-04-20T10:25:50.889 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/udf/group.list 2024-04-20T10:25:50.931 INFO:teuthology.orchestra.run.smithi121.stdout:Building xfs 2024-04-20T10:25:50.947 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/xfs/group.list 2024-04-20T10:25:51.284 INFO:teuthology.orchestra.run.smithi121.stdout:/usr/bin/gmake --no-print-directory Q=@ -C include install 2024-04-20T10:25:51.287 INFO:teuthology.orchestra.run.smithi121.stdout:gmake[1]: Nothing to be done for 'install'. 2024-04-20T10:25:51.287 INFO:teuthology.orchestra.run.smithi121.stdout:/usr/bin/gmake --no-print-directory Q=@ -C lib install 2024-04-20T10:25:51.293 INFO:teuthology.orchestra.run.smithi121.stdout:gmake[1]: Nothing to be done for 'install'. 2024-04-20T10:25:51.293 INFO:teuthology.orchestra.run.smithi121.stdout:/usr/bin/gmake --no-print-directory Q=@ -C ltp install 2024-04-20T10:25:51.299 INFO:teuthology.orchestra.run.smithi121.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/ltp 2024-04-20T10:25:51.331 INFO:teuthology.orchestra.run.smithi121.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 doio fsstress fsx growfiles iogen aio-stress /var/lib/xfstests/ltp 2024-04-20T10:25:51.573 INFO:teuthology.orchestra.run.smithi121.stdout:../install-sh -o root -g root -m 755 rwtest.sh /var/lib/xfstests/ltp 2024-04-20T10:25:51.606 INFO:teuthology.orchestra.run.smithi121.stdout:/usr/bin/gmake --no-print-directory Q=@ -C src install 2024-04-20T10:25:51.650 INFO:teuthology.orchestra.run.smithi121.stdout:Building log-writes 2024-04-20T10:25:51.661 INFO:teuthology.orchestra.run.smithi121.stdout:Building perf 2024-04-20T10:25:51.671 INFO:teuthology.orchestra.run.smithi121.stdout:Building aio-dio-regress 2024-04-20T10:25:51.683 INFO:teuthology.orchestra.run.smithi121.stdout:Building vfs 2024-04-20T10:25:51.691 INFO:teuthology.orchestra.run.smithi121.stdout:/usr/bin/gmake -C log-writes install 2024-04-20T10:25:51.695 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/log-writes 2024-04-20T10:25:51.727 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 755 replay-log /var/lib/xfstests/src/log-writes 2024-04-20T10:25:51.760 INFO:teuthology.orchestra.run.smithi121.stdout:/usr/bin/gmake -C perf install 2024-04-20T10:25:51.763 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/perf 2024-04-20T10:25:51.795 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 755 FioCompare.py fio-insert-and-compare.py FioResultDecoder.py generate-schema.py ResultData.py fio-results.sql /var/lib/xfstests/src/perf 2024-04-20T10:25:51.877 INFO:teuthology.orchestra.run.smithi121.stdout:/usr/bin/gmake -C aio-dio-regress install 2024-04-20T10:25:51.882 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/aio-dio-regress 2024-04-20T10:25:51.913 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 755 aiocp aio-dio-append-write-fallocate-race aio-dio-append-write-read-race aio-dio-cow-race aio-dio-cycle-write aio-dio-eof-race aio-dio-extend-stat aio-dio-fcntl-race aio-dio-hole-filling-race aio-dio-invalidate-failure aio-dio-invalidate-readahead aiodio_sparse2 aio-dio-subblock-eof-read aio-dio-write-verify aio-free-ring-with-bogus-nr-pages aio-io-setup-with-nonwritable-context-pointer aio-last-ref-held-by-io /var/lib/xfstests/src/aio-dio-regress 2024-04-20T10:25:52.099 INFO:teuthology.orchestra.run.smithi121.stdout:/usr/bin/gmake -C vfs install 2024-04-20T10:25:52.102 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src/vfs 2024-04-20T10:25:52.132 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 755 vfstest mount-idmapped /var/lib/xfstests/src/vfs 2024-04-20T10:25:52.177 INFO:teuthology.orchestra.run.smithi121.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/src 2024-04-20T10:25:52.207 INFO:teuthology.orchestra.run.smithi121.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dirstress fill fill2 getpagesize holes lstat64 nametest permname randholes runas truncfile usemem mmapcat append_reader append_writer dirperf metaperf devzero feature alloc fault fstest t_access_root godown resvtest writemod writev_on_pagefault makeextents itrash rename multi_open_unlink unwritten_sync genhashnames t_holes t_mmap_writev t_truncate_cmtime dirhash_collide t_rename_overwrite holetest t_truncate_self af_unix t_mmap_stale_pmd t_mmap_cow_race t_mmap_fallocate fsync-err t_mmap_write_ro t_ext4_dax_journal_corruption t_ext4_dax_inline_corruption t_ofd_locks t_mmap_collision mmap-write-concurrent t_get_file_time t_create_short_dirs t_create_long_dirs t_enospc t_mmap_writev_overlap checkpoint_journal mmap-rw-fault allocstale t_mmap_cow_memory_failure fake-dump-rootino dio-buf-fault rewinddir-test readdir-while-renames xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader preallo_rw_pattern_writer ftrunc trunc fs_perms testx looptest locktest unwritten_mmap bulkstat_unlink_test deduperace bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test seek_copy_test t_readdir_1 t_readdir_2 fsync-tester nsexec cloner renameat2 t_getcwd e4compact test-nextquota punch-alternating attr-list-by-handle-cursor-test listxattr dio-interleaved t_dir_type dio-invalidate-cache stat_test t_encrypted_d_revalidate attr_replace_test swapon mkswap t_attr_corruption t_open_tmpfiles fscrypt-crypt-util bulkstat_null_ocount splice-test chprojid_fail detached_mounts_propagation ext4_resize t_readdir_3 splice2pipe uuid_ioctl t_snapshot_deleted_subvolume fiemap-fault fiemap-tester t_stripealign open_by_handle dbtest t_mmap_dio xfsfind t_reflink_read_race fssum /var/lib/xfstests/src 2024-04-20T10:25:56.504 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:25:56.509 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:25:56.648 INFO:teuthology.orchestra.run.smithi121.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 755 dmerror fill2attr fill2fs fill2fs_check scaleread.sh btrfs_crc32c_forged_name.py popdir.pl popattr.py soak_duration.awk /var/lib/xfstests/src 2024-04-20T10:25:56.966 INFO:teuthology.orchestra.run.smithi121.stdout:/bin/sh ../libtool --quiet --mode=install ../install-sh -o root -g root -m 644 dumpfile /var/lib/xfstests/src 2024-04-20T10:25:57.028 INFO:teuthology.orchestra.run.smithi121.stdout:/usr/bin/gmake --no-print-directory Q=@ -C m4 install 2024-04-20T10:25:57.031 INFO:teuthology.orchestra.run.smithi121.stdout:gmake[1]: Nothing to be done for 'install'. 2024-04-20T10:25:57.031 INFO:teuthology.orchestra.run.smithi121.stdout:/usr/bin/gmake --no-print-directory Q=@ -C common install 2024-04-20T10:25:57.035 INFO:teuthology.orchestra.run.smithi121.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/common 2024-04-20T10:25:57.066 INFO:teuthology.orchestra.run.smithi121.stdout:../install-sh -o root -g root -m 644 * /var/lib/xfstests/common 2024-04-20T10:25:57.641 INFO:teuthology.orchestra.run.smithi121.stdout:/usr/bin/gmake --no-print-directory Q=@ -C tests install 2024-04-20T10:25:57.645 INFO:teuthology.orchestra.run.smithi121.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.yONpfXTHudxfstests-dev/tests/btrfs/ install 2024-04-20T10:25:57.653 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/btrfs/group.list 2024-04-20T10:25:57.864 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/btrfs 2024-04-20T10:25:57.897 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 320 330 /var/lib/xfstests/tests/btrfs 2024-04-20T10:26:01.092 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/btrfs 2024-04-20T10:26:01.121 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 096.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 216.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 320.out 330.out /var/lib/xfstests/tests/btrfs 2024-04-20T10:26:04.594 INFO:teuthology.orchestra.run.smithi121.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.yONpfXTHudxfstests-dev/tests/ceph/ install 2024-04-20T10:26:04.602 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/ceph/group.list 2024-04-20T10:26:04.652 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ceph 2024-04-20T10:26:04.683 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 /var/lib/xfstests/tests/ceph 2024-04-20T10:26:04.755 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ceph 2024-04-20T10:26:04.787 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out /var/lib/xfstests/tests/ceph 2024-04-20T10:26:04.859 INFO:teuthology.orchestra.run.smithi121.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.yONpfXTHudxfstests-dev/tests/cifs/ install 2024-04-20T10:26:04.865 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/cifs/group.list 2024-04-20T10:26:04.906 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/cifs 2024-04-20T10:26:04.934 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/cifs 2024-04-20T10:26:04.964 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/cifs 2024-04-20T10:26:04.993 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/cifs 2024-04-20T10:26:05.022 INFO:teuthology.orchestra.run.smithi121.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.yONpfXTHudxfstests-dev/tests/ext4/ install 2024-04-20T10:26:05.030 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/ext4/group.list 2024-04-20T10:26:05.159 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ext4 2024-04-20T10:26:05.192 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 271 301 302 303 304 305 306 307 308 /var/lib/xfstests/tests/ext4 2024-04-20T10:26:05.890 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ext4 2024-04-20T10:26:05.922 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 644 001.out.default 001.out.nodelalloc 001.out.nozero 001.cfg 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 271.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out /var/lib/xfstests/tests/ext4 2024-04-20T10:26:06.706 INFO:teuthology.orchestra.run.smithi121.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.yONpfXTHudxfstests-dev/tests/f2fs/ install 2024-04-20T10:26:06.713 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/f2fs/group.list 2024-04-20T10:26:06.775 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/f2fs 2024-04-20T10:26:06.807 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 755 001 002 /var/lib/xfstests/tests/f2fs 2024-04-20T10:26:06.850 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/f2fs 2024-04-20T10:26:06.883 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 644 001.out 002.out /var/lib/xfstests/tests/f2fs 2024-04-20T10:26:06.928 INFO:teuthology.orchestra.run.smithi121.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.yONpfXTHudxfstests-dev/tests/generic/ install 2024-04-20T10:26:06.938 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/generic/group.list 2024-04-20T10:26:07.296 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/generic 2024-04-20T10:26:07.329 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 741 742 743 /var/lib/xfstests/tests/generic 2024-04-20T10:26:15.136 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/generic 2024-04-20T10:26:15.168 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 644 035.out.default 035.out.nfs 050.out.default 050.out.nojournal 050.out.xfsquota 035.cfg 050.cfg 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 096.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 216.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 318.out 319.out 320.out 321.out 322.out 323.out 324.out 325.out 326.out 327.out 328.out 329.out 330.out 331.out 332.out 333.out 334.out 335.out 336.out 337.out 338.out 339.out 340.out 341.out 342.out 343.out 344.out 345.out 346.out 347.out 348.out 349.out 350.out 351.out 352.out 353.out 354.out 355.out 356.out 357.out 358.out 359.out 360.out 361.out 362.out 363.out 364.out 365.out 366.out 367.out 368.out 369.out 370.out 371.out 372.out 373.out 374.out 375.out 376.out 377.out 378.out 379.out 380.out 381.out 382.out 383.out 384.out 385.out 386.out 387.out 388.out 389.out 390.out 391.out 392.out 393.out 394.out 395.out 396.out 397.out 398.out 399.out 400.out 401.out 402.out 403.out 404.out 405.out 406.out 407.out 408.out 409.out 410.out 411.out 412.out 413.out 414.out 415.out 416.out 417.out 418.out 419.out 420.out 421.out 422.out 423.out 424.out 425.out 426.out 427.out 428.out 429.out 430.out 431.out 432.out 433.out 434.out 435.out 436.out 437.out 438.out 439.out 440.out 441.out 442.out 443.out 444.out 445.out 446.out 447.out 448.out 449.out 450.out 451.out 452.out 453.out 454.out 455.out 456.out 457.out 458.out 459.out 460.out 461.out 462.out 463.out 464.out 465.out 466.out 467.out 468.out 469.out 470.out 471.out 472.out 473.out 474.out 475.out 476.out 477.out 478.out 479.out 480.out 481.out 482.out 483.out 484.out 485.out 486.out 487.out 488.out 489.out 490.out 491.out 492.out 493.out 494.out 495.out 496.out 497.out 498.out 499.out 500.out 501.out 502.out 503.out 504.out 505.out 506.out 507.out 508.out 509.out 510.out 511.out 512.out 513.out 514.out 515.out 516.out 517.out 518.out 519.out 520.out 521.out 522.out 523.out 524.out 525.out 526.out 527.out 528.out 529.out 530.out 531.out 532.out 533.out 534.out 535.out 536.out 537.out 538.out 539.out 540.out 541.out 542.out 543.out 544.out 545.out 546.out 547.out 548.out 549.out 550.out 551.out 552.out 553.out 554.out 555.out 556.out 557.out 558.out 559.out 560.out 561.out 562.out 563.out 564.out 565.out 566.out 567.out 568.out 569.out 570.out 571.out 572.out 573.out 574.out 575.out 576.out 577.out 578.out 579.out 580.out 581.out 582.out 583.out 584.out 585.out 586.out 587.out 588.out 589.out 590.out 591.out 592.out 593.out 594.out 595.out 596.out 597.out 598.out 599.out 600.out 601.out 602.out 603.out 604.out 605.out 606.out 607.out 608.out 609.out 610.out 611.out 612.out 613.out 614.out 615.out 616.out 617.out 618.out 619.out 620.out 621.out 622.out 623.out 624.out 625.out 626.out 627.out 628.out 629.out 630.out 631.out 632.out 633.out 634.out 635.out 636.out 637.out 638.out 639.out 640.out 641.out 642.out 643.out 644.out 645.out 646.out 647.out 648.out 649.out 650.out 651.out 652.out 653.out 654.out 655.out 656.out 657.out 658.out 659.out 660.out 661.out 662.out 663.out 664.out 665.out 666.out 667.out 668.out 669.out 670.out 671.out 672.out 673.out 674.out 675.out 676.out 677.out 678.out 679.out 680.out 681.out 682.out 683.out 684.out 685.out 686.out 687.out 688.out 689.out 690.out 691.out 692.out 693.out 694.out 695.out 696.out 697.out 698.out 699.out 700.out 701.out 702.out 703.out 704.out 705.out 706.out 707.out 708.out 709.out 710.out 711.out 712.out 713.out 714.out 715.out 716.out 717.out 718.out 719.out 720.out 721.out 722.out 723.out 724.out 725.out 726.out 727.out 728.out 729.out 730.out 731.out 732.out 733.out 734.out 735.out 736.out 737.out 738.out 739.out 741.out 742.out 743.out /var/lib/xfstests/tests/generic 2024-04-20T10:26:23.126 INFO:teuthology.orchestra.run.smithi121.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.yONpfXTHudxfstests-dev/tests/nfs/ install 2024-04-20T10:26:23.134 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/nfs/group.list 2024-04-20T10:26:23.185 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/nfs 2024-04-20T10:26:23.217 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/nfs 2024-04-20T10:26:23.249 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/nfs 2024-04-20T10:26:23.279 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/nfs 2024-04-20T10:26:23.312 INFO:teuthology.orchestra.run.smithi121.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.yONpfXTHudxfstests-dev/tests/ocfs2/ install 2024-04-20T10:26:23.319 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/ocfs2/group.list 2024-04-20T10:26:23.363 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/ocfs2 2024-04-20T10:26:23.392 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/ocfs2 2024-04-20T10:26:23.423 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/ocfs2 2024-04-20T10:26:23.456 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/ocfs2 2024-04-20T10:26:23.490 INFO:teuthology.orchestra.run.smithi121.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.yONpfXTHudxfstests-dev/tests/overlay/ install 2024-04-20T10:26:23.496 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/overlay/group.list 2024-04-20T10:26:23.642 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/overlay 2024-04-20T10:26:23.673 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 /var/lib/xfstests/tests/overlay 2024-04-20T10:26:24.783 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/overlay 2024-04-20T10:26:24.815 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 033.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 071.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out /var/lib/xfstests/tests/overlay 2024-04-20T10:26:25.993 INFO:teuthology.orchestra.run.smithi121.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.yONpfXTHudxfstests-dev/tests/perf/ install 2024-04-20T10:26:26.000 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/perf/group.list 2024-04-20T10:26:26.038 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/perf 2024-04-20T10:26:26.067 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/perf 2024-04-20T10:26:26.095 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/perf 2024-04-20T10:26:26.125 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/perf 2024-04-20T10:26:26.155 INFO:teuthology.orchestra.run.smithi121.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.yONpfXTHudxfstests-dev/tests/selftest/ install 2024-04-20T10:26:26.161 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/selftest/group.list 2024-04-20T10:26:26.203 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/selftest 2024-04-20T10:26:26.236 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 /var/lib/xfstests/tests/selftest 2024-04-20T10:26:26.319 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/selftest 2024-04-20T10:26:26.352 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 644 001.out 002.out 003.out 004.out 005.out 006.out /var/lib/xfstests/tests/selftest 2024-04-20T10:26:26.432 INFO:teuthology.orchestra.run.smithi121.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.yONpfXTHudxfstests-dev/tests/shared/ install 2024-04-20T10:26:26.440 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/shared/group.list 2024-04-20T10:26:26.499 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/shared 2024-04-20T10:26:26.528 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 755 002 032 298 /var/lib/xfstests/tests/shared 2024-04-20T10:26:26.578 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/shared 2024-04-20T10:26:26.610 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 644 002.out 032.out 298.out /var/lib/xfstests/tests/shared 2024-04-20T10:26:26.642 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:26:26.649 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:26:26.663 INFO:teuthology.orchestra.run.smithi121.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.yONpfXTHudxfstests-dev/tests/tmpfs/ install 2024-04-20T10:26:26.669 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/tmpfs/group.list 2024-04-20T10:26:26.708 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/tmpfs 2024-04-20T10:26:26.732 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 755 001 /var/lib/xfstests/tests/tmpfs 2024-04-20T10:26:26.759 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/tmpfs 2024-04-20T10:26:26.786 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 644 001.out /var/lib/xfstests/tests/tmpfs 2024-04-20T10:26:26.817 INFO:teuthology.orchestra.run.smithi121.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.yONpfXTHudxfstests-dev/tests/udf/ install 2024-04-20T10:26:26.823 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/udf/group.list 2024-04-20T10:26:26.856 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/udf 2024-04-20T10:26:26.883 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 755 102 /var/lib/xfstests/tests/udf 2024-04-20T10:26:26.914 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/udf 2024-04-20T10:26:26.946 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 644 102.out /var/lib/xfstests/tests/udf 2024-04-20T10:26:26.975 INFO:teuthology.orchestra.run.smithi121.stdout:/usr/bin/gmake --no-print-directory Q=@ -C /tmp/tmp.yONpfXTHudxfstests-dev/tests/xfs/ install 2024-04-20T10:26:26.983 INFO:teuthology.orchestra.run.smithi121.stdout: [GROUP] /tmp/tmp.yONpfXTHudxfstests-dev/tests/xfs/group.list 2024-04-20T10:26:27.355 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests/xfs 2024-04-20T10:26:27.391 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 755 001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 /var/lib/xfstests/tests/xfs 2024-04-20T10:26:34.960 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 644 group.list /var/lib/xfstests/tests/xfs 2024-04-20T10:26:34.995 INFO:teuthology.orchestra.run.smithi121.stdout:../../install-sh -o root -g root -m 644 033.out.crc 033.out.default 071.out.32 071.out.64 096.out.external 096.out.internal 216.out.64mblog 216.out.classic 033.cfg 216.cfg 001.out 002.out 003.out 004.out 005.out 006.out 007.out 008.out 009.out 010.out 011.out 012.out 013.out 014.out 015.out 016.out 017.out 018.out 019.out 020.out 021.out 022.out 023.out 024.out 025.out 026.out 027.out 028.out 029.out 030.out 031.out 032.out 034.out 035.out 036.out 037.out 038.out 039.out 040.out 041.out 042.out 043.out 044.out 045.out 046.out 047.out 048.out 049.out 050.out 051.out 052.out 053.out 054.out 055.out 056.out 057.out 058.out 059.out 060.out 061.out 062.out 063.out 064.out 065.out 066.out 067.out 068.out 069.out 070.out 072.out 073.out 074.out 075.out 076.out 077.out 078.out 079.out 080.out 081.out 082.out 083.out 084.out 085.out 086.out 087.out 088.out 089.out 090.out 091.out 092.out 093.out 094.out 095.out 097.out 098.out 099.out 100.out 101.out 102.out 103.out 104.out 105.out 106.out 107.out 108.out 109.out 110.out 111.out 112.out 113.out 114.out 115.out 116.out 117.out 118.out 119.out 120.out 121.out 122.out 123.out 124.out 125.out 126.out 127.out 128.out 129.out 130.out 131.out 132.out 133.out 134.out 135.out 136.out 137.out 138.out 139.out 140.out 141.out 142.out 143.out 144.out 145.out 146.out 147.out 148.out 149.out 150.out 151.out 152.out 153.out 154.out 155.out 156.out 157.out 158.out 159.out 160.out 161.out 162.out 163.out 164.out 165.out 166.out 167.out 168.out 169.out 170.out 171.out 172.out 173.out 174.out 175.out 176.out 177.out 178.out 179.out 180.out 181.out 182.out 183.out 184.out 185.out 186.out 187.out 188.out 189.out 190.out 191.out 192.out 193.out 194.out 195.out 196.out 197.out 198.out 199.out 200.out 201.out 202.out 203.out 204.out 205.out 206.out 207.out 208.out 209.out 210.out 211.out 212.out 213.out 214.out 215.out 217.out 218.out 219.out 220.out 221.out 222.out 223.out 224.out 225.out 226.out 227.out 228.out 229.out 230.out 231.out 232.out 233.out 234.out 235.out 236.out 237.out 238.out 239.out 240.out 241.out 242.out 243.out 244.out 245.out 246.out 247.out 248.out 249.out 250.out 251.out 252.out 253.out 254.out 255.out 256.out 257.out 258.out 259.out 260.out 261.out 262.out 263.out 264.out 265.out 266.out 267.out 268.out 269.out 270.out 271.out 272.out 273.out 274.out 275.out 276.out 277.out 278.out 279.out 280.out 281.out 282.out 283.out 284.out 285.out 286.out 287.out 288.out 289.out 290.out 291.out 292.out 293.out 294.out 295.out 296.out 297.out 298.out 299.out 300.out 301.out 302.out 303.out 304.out 305.out 306.out 307.out 308.out 309.out 310.out 311.out 312.out 313.out 314.out 315.out 316.out 317.out 318.out 319.out 320.out 321.out 322.out 323.out 324.out 325.out 326.out 327.out 328.out 329.out 330.out 331.out 332.out 333.out 334.out 335.out 336.out 337.out 338.out 339.out 340.out 341.out 342.out 343.out 344.out 345.out 346.out 347.out 348.out 349.out 350.out 351.out 352.out 353.out 354.out 355.out 356.out 357.out 358.out 359.out 360.out 361.out 362.out 363.out 364.out 365.out 366.out 367.out 368.out 369.out 370.out 371.out 372.out 373.out 374.out 375.out 376.out 377.out 378.out 379.out 380.out 381.out 382.out 383.out 384.out 385.out 386.out 387.out 388.out 389.out 390.out 391.out 392.out 393.out 394.out 395.out 396.out 397.out 398.out 399.out 400.out 401.out 402.out 403.out 404.out 405.out 406.out 407.out 408.out 409.out 410.out 411.out 412.out 413.out 414.out 415.out 416.out 417.out 418.out 419.out 420.out 421.out 422.out 423.out 424.out 425.out 426.out 427.out 428.out 429.out 430.out 431.out 432.out 433.out 434.out 435.out 436.out 437.out 438.out 439.out 440.out 441.out 442.out 443.out 444.out 445.out 446.out 447.out 448.out 449.out 450.out 451.out 452.out 453.out 454.out 455.out 456.out 457.out 458.out 459.out 460.out 461.out 462.out 463.out 464.out 465.out 466.out 467.out 468.out 469.out 470.out 471.out 472.out 473.out 474.out 475.out 476.out 477.out 478.out 479.out 480.out 481.out 482.out 483.out 484.out 485.out 486.out 487.out 488.out 489.out 490.out 491.out 492.out 493.out 494.out 495.out 496.out 497.out 498.out 499.out 500.out 501.out 502.out 503.out 504.out 505.out 506.out 507.out 508.out 509.out 510.out 511.out 512.out 513.out 514.out 515.out 516.out 517.out 518.out 519.out 520.out 521.out 522.out 523.out 524.out 525.out 526.out 527.out 528.out 529.out 530.out 531.out 532.out 533.out 534.out 535.out 536.out 537.out 538.out 539.out 540.out 541.out 542.out 543.out 544.out 545.out 546.out 547.out 548.out 549.out 550.out 551.out 552.out 553.out 554.out 555.out 556.out 557.out 558.out 559.out 560.out 561.out 562.out 563.out 564.out 565.out 566.out 567.out 568.out 569.out 570.out 571.out 572.out 573.out 574.out 575.out 576.out 577.out 578.out 579.out 580.out 581.out 582.out 583.out 584.out 585.out 586.out 587.out 588.out 589.out 590.out 591.out 592.out 593.out 594.out 595.out 596.out 597.out 598.out 599.out 600.out 601.out 602.out 603.out 604.out 605.out 606.out 607.out 708.out 709.out 710.out 711.out 712.out 713.out 714.out 715.out 716.out 717.out 718.out 719.out 720.out 721.out 722.out 723.out 724.out 725.out 726.out 727.out 728.out 729.out 730.out 731.out 732.out 733.out 734.out 735.out 736.out 737.out 738.out 739.out 740.out 741.out 742.out 743.out 744.out 745.out 746.out 747.out 748.out 749.out 750.out 751.out 752.out 753.out 754.out 755.out 756.out 757.out 758.out 759.out 760.out 761.out 762.out 763.out 764.out 765.out 766.out 767.out 768.out 769.out 770.out 771.out 772.out 773.out 774.out 775.out 776.out 777.out 778.out 779.out 780.out 781.out 782.out 783.out 784.out 785.out 786.out 787.out 788.out 789.out 790.out 791.out 792.out 793.out 794.out 795.out 796.out 797.out 798.out 799.out 800.out /var/lib/xfstests/tests/xfs 2024-04-20T10:26:42.317 INFO:teuthology.orchestra.run.smithi121.stdout:../install-sh -o root -g root -m 755 -d /var/lib/xfstests/tests 2024-04-20T10:26:42.352 INFO:teuthology.orchestra.run.smithi121.stdout:./install-sh -o root -g root -m 755 -d /var/lib/xfstests 2024-04-20T10:26:42.383 INFO:teuthology.orchestra.run.smithi121.stdout:./install-sh -o root -g root -m 755 check /var/lib/xfstests 2024-04-20T10:26:42.415 INFO:teuthology.orchestra.run.smithi121.stdout:./install-sh -o root -g root -m 644 randomize.awk /var/lib/xfstests 2024-04-20T10:26:42.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:26:42.453 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec ls tests/generic) 2024-04-20T10:26:42.552 INFO:teuthology.orchestra.run.smithi121.stdout:001 2024-04-20T10:26:42.552 INFO:teuthology.orchestra.run.smithi121.stdout:001.out 2024-04-20T10:26:42.552 INFO:teuthology.orchestra.run.smithi121.stdout:002 2024-04-20T10:26:42.552 INFO:teuthology.orchestra.run.smithi121.stdout:002.out 2024-04-20T10:26:42.552 INFO:teuthology.orchestra.run.smithi121.stdout:003 2024-04-20T10:26:42.552 INFO:teuthology.orchestra.run.smithi121.stdout:003.out 2024-04-20T10:26:42.552 INFO:teuthology.orchestra.run.smithi121.stdout:004 2024-04-20T10:26:42.552 INFO:teuthology.orchestra.run.smithi121.stdout:004.out 2024-04-20T10:26:42.552 INFO:teuthology.orchestra.run.smithi121.stdout:005 2024-04-20T10:26:42.552 INFO:teuthology.orchestra.run.smithi121.stdout:005.out 2024-04-20T10:26:42.552 INFO:teuthology.orchestra.run.smithi121.stdout:006 2024-04-20T10:26:42.553 INFO:teuthology.orchestra.run.smithi121.stdout:006.out 2024-04-20T10:26:42.553 INFO:teuthology.orchestra.run.smithi121.stdout:007 2024-04-20T10:26:42.553 INFO:teuthology.orchestra.run.smithi121.stdout:007.out 2024-04-20T10:26:42.553 INFO:teuthology.orchestra.run.smithi121.stdout:008 2024-04-20T10:26:42.553 INFO:teuthology.orchestra.run.smithi121.stdout:008.out 2024-04-20T10:26:42.553 INFO:teuthology.orchestra.run.smithi121.stdout:009 2024-04-20T10:26:42.553 INFO:teuthology.orchestra.run.smithi121.stdout:009.out 2024-04-20T10:26:42.553 INFO:teuthology.orchestra.run.smithi121.stdout:010 2024-04-20T10:26:42.553 INFO:teuthology.orchestra.run.smithi121.stdout:010.out 2024-04-20T10:26:42.553 INFO:teuthology.orchestra.run.smithi121.stdout:011 2024-04-20T10:26:42.553 INFO:teuthology.orchestra.run.smithi121.stdout:011.out 2024-04-20T10:26:42.553 INFO:teuthology.orchestra.run.smithi121.stdout:012 2024-04-20T10:26:42.553 INFO:teuthology.orchestra.run.smithi121.stdout:012.out 2024-04-20T10:26:42.553 INFO:teuthology.orchestra.run.smithi121.stdout:013 2024-04-20T10:26:42.553 INFO:teuthology.orchestra.run.smithi121.stdout:013.out 2024-04-20T10:26:42.554 INFO:teuthology.orchestra.run.smithi121.stdout:014 2024-04-20T10:26:42.554 INFO:teuthology.orchestra.run.smithi121.stdout:014.out 2024-04-20T10:26:42.554 INFO:teuthology.orchestra.run.smithi121.stdout:015 2024-04-20T10:26:42.554 INFO:teuthology.orchestra.run.smithi121.stdout:015.out 2024-04-20T10:26:42.554 INFO:teuthology.orchestra.run.smithi121.stdout:016 2024-04-20T10:26:42.554 INFO:teuthology.orchestra.run.smithi121.stdout:016.out 2024-04-20T10:26:42.554 INFO:teuthology.orchestra.run.smithi121.stdout:017 2024-04-20T10:26:42.554 INFO:teuthology.orchestra.run.smithi121.stdout:017.out 2024-04-20T10:26:42.554 INFO:teuthology.orchestra.run.smithi121.stdout:018 2024-04-20T10:26:42.554 INFO:teuthology.orchestra.run.smithi121.stdout:018.out 2024-04-20T10:26:42.554 INFO:teuthology.orchestra.run.smithi121.stdout:019 2024-04-20T10:26:42.554 INFO:teuthology.orchestra.run.smithi121.stdout:019.out 2024-04-20T10:26:42.554 INFO:teuthology.orchestra.run.smithi121.stdout:020 2024-04-20T10:26:42.555 INFO:teuthology.orchestra.run.smithi121.stdout:020.out 2024-04-20T10:26:42.555 INFO:teuthology.orchestra.run.smithi121.stdout:021 2024-04-20T10:26:42.555 INFO:teuthology.orchestra.run.smithi121.stdout:021.out 2024-04-20T10:26:42.555 INFO:teuthology.orchestra.run.smithi121.stdout:022 2024-04-20T10:26:42.555 INFO:teuthology.orchestra.run.smithi121.stdout:022.out 2024-04-20T10:26:42.555 INFO:teuthology.orchestra.run.smithi121.stdout:023 2024-04-20T10:26:42.555 INFO:teuthology.orchestra.run.smithi121.stdout:023.out 2024-04-20T10:26:42.555 INFO:teuthology.orchestra.run.smithi121.stdout:024 2024-04-20T10:26:42.555 INFO:teuthology.orchestra.run.smithi121.stdout:024.out 2024-04-20T10:26:42.555 INFO:teuthology.orchestra.run.smithi121.stdout:025 2024-04-20T10:26:42.555 INFO:teuthology.orchestra.run.smithi121.stdout:025.out 2024-04-20T10:26:42.555 INFO:teuthology.orchestra.run.smithi121.stdout:026 2024-04-20T10:26:42.555 INFO:teuthology.orchestra.run.smithi121.stdout:026.out 2024-04-20T10:26:42.555 INFO:teuthology.orchestra.run.smithi121.stdout:027 2024-04-20T10:26:42.555 INFO:teuthology.orchestra.run.smithi121.stdout:027.out 2024-04-20T10:26:42.556 INFO:teuthology.orchestra.run.smithi121.stdout:028 2024-04-20T10:26:42.556 INFO:teuthology.orchestra.run.smithi121.stdout:028.out 2024-04-20T10:26:42.556 INFO:teuthology.orchestra.run.smithi121.stdout:029 2024-04-20T10:26:42.556 INFO:teuthology.orchestra.run.smithi121.stdout:029.out 2024-04-20T10:26:42.556 INFO:teuthology.orchestra.run.smithi121.stdout:030 2024-04-20T10:26:42.556 INFO:teuthology.orchestra.run.smithi121.stdout:030.out 2024-04-20T10:26:42.556 INFO:teuthology.orchestra.run.smithi121.stdout:031 2024-04-20T10:26:42.556 INFO:teuthology.orchestra.run.smithi121.stdout:031.out 2024-04-20T10:26:42.556 INFO:teuthology.orchestra.run.smithi121.stdout:032 2024-04-20T10:26:42.556 INFO:teuthology.orchestra.run.smithi121.stdout:032.out 2024-04-20T10:26:42.556 INFO:teuthology.orchestra.run.smithi121.stdout:033 2024-04-20T10:26:42.556 INFO:teuthology.orchestra.run.smithi121.stdout:033.out 2024-04-20T10:26:42.556 INFO:teuthology.orchestra.run.smithi121.stdout:034 2024-04-20T10:26:42.556 INFO:teuthology.orchestra.run.smithi121.stdout:034.out 2024-04-20T10:26:42.556 INFO:teuthology.orchestra.run.smithi121.stdout:035 2024-04-20T10:26:42.557 INFO:teuthology.orchestra.run.smithi121.stdout:035.cfg 2024-04-20T10:26:42.557 INFO:teuthology.orchestra.run.smithi121.stdout:035.out.default 2024-04-20T10:26:42.557 INFO:teuthology.orchestra.run.smithi121.stdout:035.out.nfs 2024-04-20T10:26:42.557 INFO:teuthology.orchestra.run.smithi121.stdout:036 2024-04-20T10:26:42.557 INFO:teuthology.orchestra.run.smithi121.stdout:036.out 2024-04-20T10:26:42.557 INFO:teuthology.orchestra.run.smithi121.stdout:037 2024-04-20T10:26:42.557 INFO:teuthology.orchestra.run.smithi121.stdout:037.out 2024-04-20T10:26:42.557 INFO:teuthology.orchestra.run.smithi121.stdout:038 2024-04-20T10:26:42.557 INFO:teuthology.orchestra.run.smithi121.stdout:038.out 2024-04-20T10:26:42.557 INFO:teuthology.orchestra.run.smithi121.stdout:039 2024-04-20T10:26:42.557 INFO:teuthology.orchestra.run.smithi121.stdout:039.out 2024-04-20T10:26:42.557 INFO:teuthology.orchestra.run.smithi121.stdout:040 2024-04-20T10:26:42.557 INFO:teuthology.orchestra.run.smithi121.stdout:040.out 2024-04-20T10:26:42.557 INFO:teuthology.orchestra.run.smithi121.stdout:041 2024-04-20T10:26:42.558 INFO:teuthology.orchestra.run.smithi121.stdout:041.out 2024-04-20T10:26:42.558 INFO:teuthology.orchestra.run.smithi121.stdout:042 2024-04-20T10:26:42.558 INFO:teuthology.orchestra.run.smithi121.stdout:042.out 2024-04-20T10:26:42.558 INFO:teuthology.orchestra.run.smithi121.stdout:043 2024-04-20T10:26:42.558 INFO:teuthology.orchestra.run.smithi121.stdout:043.out 2024-04-20T10:26:42.558 INFO:teuthology.orchestra.run.smithi121.stdout:044 2024-04-20T10:26:42.558 INFO:teuthology.orchestra.run.smithi121.stdout:044.out 2024-04-20T10:26:42.558 INFO:teuthology.orchestra.run.smithi121.stdout:045 2024-04-20T10:26:42.558 INFO:teuthology.orchestra.run.smithi121.stdout:045.out 2024-04-20T10:26:42.558 INFO:teuthology.orchestra.run.smithi121.stdout:046 2024-04-20T10:26:42.558 INFO:teuthology.orchestra.run.smithi121.stdout:046.out 2024-04-20T10:26:42.558 INFO:teuthology.orchestra.run.smithi121.stdout:047 2024-04-20T10:26:42.558 INFO:teuthology.orchestra.run.smithi121.stdout:047.out 2024-04-20T10:26:42.558 INFO:teuthology.orchestra.run.smithi121.stdout:048 2024-04-20T10:26:42.559 INFO:teuthology.orchestra.run.smithi121.stdout:048.out 2024-04-20T10:26:42.559 INFO:teuthology.orchestra.run.smithi121.stdout:049 2024-04-20T10:26:42.559 INFO:teuthology.orchestra.run.smithi121.stdout:049.out 2024-04-20T10:26:42.559 INFO:teuthology.orchestra.run.smithi121.stdout:050 2024-04-20T10:26:42.559 INFO:teuthology.orchestra.run.smithi121.stdout:050.cfg 2024-04-20T10:26:42.559 INFO:teuthology.orchestra.run.smithi121.stdout:050.out.default 2024-04-20T10:26:42.559 INFO:teuthology.orchestra.run.smithi121.stdout:050.out.nojournal 2024-04-20T10:26:42.559 INFO:teuthology.orchestra.run.smithi121.stdout:050.out.xfsquota 2024-04-20T10:26:42.559 INFO:teuthology.orchestra.run.smithi121.stdout:051 2024-04-20T10:26:42.559 INFO:teuthology.orchestra.run.smithi121.stdout:051.out 2024-04-20T10:26:42.559 INFO:teuthology.orchestra.run.smithi121.stdout:052 2024-04-20T10:26:42.559 INFO:teuthology.orchestra.run.smithi121.stdout:052.out 2024-04-20T10:26:42.559 INFO:teuthology.orchestra.run.smithi121.stdout:053 2024-04-20T10:26:42.559 INFO:teuthology.orchestra.run.smithi121.stdout:053.out 2024-04-20T10:26:42.559 INFO:teuthology.orchestra.run.smithi121.stdout:054 2024-04-20T10:26:42.560 INFO:teuthology.orchestra.run.smithi121.stdout:054.out 2024-04-20T10:26:42.560 INFO:teuthology.orchestra.run.smithi121.stdout:055 2024-04-20T10:26:42.560 INFO:teuthology.orchestra.run.smithi121.stdout:055.out 2024-04-20T10:26:42.560 INFO:teuthology.orchestra.run.smithi121.stdout:056 2024-04-20T10:26:42.560 INFO:teuthology.orchestra.run.smithi121.stdout:056.out 2024-04-20T10:26:42.560 INFO:teuthology.orchestra.run.smithi121.stdout:057 2024-04-20T10:26:42.560 INFO:teuthology.orchestra.run.smithi121.stdout:057.out 2024-04-20T10:26:42.560 INFO:teuthology.orchestra.run.smithi121.stdout:058 2024-04-20T10:26:42.560 INFO:teuthology.orchestra.run.smithi121.stdout:058.out 2024-04-20T10:26:42.560 INFO:teuthology.orchestra.run.smithi121.stdout:059 2024-04-20T10:26:42.560 INFO:teuthology.orchestra.run.smithi121.stdout:059.out 2024-04-20T10:26:42.560 INFO:teuthology.orchestra.run.smithi121.stdout:060 2024-04-20T10:26:42.561 INFO:teuthology.orchestra.run.smithi121.stdout:060.out 2024-04-20T10:26:42.561 INFO:teuthology.orchestra.run.smithi121.stdout:061 2024-04-20T10:26:42.561 INFO:teuthology.orchestra.run.smithi121.stdout:061.out 2024-04-20T10:26:42.561 INFO:teuthology.orchestra.run.smithi121.stdout:062 2024-04-20T10:26:42.561 INFO:teuthology.orchestra.run.smithi121.stdout:062.out 2024-04-20T10:26:42.561 INFO:teuthology.orchestra.run.smithi121.stdout:063 2024-04-20T10:26:42.561 INFO:teuthology.orchestra.run.smithi121.stdout:063.out 2024-04-20T10:26:42.561 INFO:teuthology.orchestra.run.smithi121.stdout:064 2024-04-20T10:26:42.561 INFO:teuthology.orchestra.run.smithi121.stdout:064.out 2024-04-20T10:26:42.561 INFO:teuthology.orchestra.run.smithi121.stdout:065 2024-04-20T10:26:42.561 INFO:teuthology.orchestra.run.smithi121.stdout:065.out 2024-04-20T10:26:42.562 INFO:teuthology.orchestra.run.smithi121.stdout:066 2024-04-20T10:26:42.562 INFO:teuthology.orchestra.run.smithi121.stdout:066.out 2024-04-20T10:26:42.562 INFO:teuthology.orchestra.run.smithi121.stdout:067 2024-04-20T10:26:42.562 INFO:teuthology.orchestra.run.smithi121.stdout:067.out 2024-04-20T10:26:42.562 INFO:teuthology.orchestra.run.smithi121.stdout:068 2024-04-20T10:26:42.562 INFO:teuthology.orchestra.run.smithi121.stdout:068.out 2024-04-20T10:26:42.562 INFO:teuthology.orchestra.run.smithi121.stdout:069 2024-04-20T10:26:42.562 INFO:teuthology.orchestra.run.smithi121.stdout:069.out 2024-04-20T10:26:42.562 INFO:teuthology.orchestra.run.smithi121.stdout:070 2024-04-20T10:26:42.562 INFO:teuthology.orchestra.run.smithi121.stdout:070.out 2024-04-20T10:26:42.562 INFO:teuthology.orchestra.run.smithi121.stdout:071 2024-04-20T10:26:42.562 INFO:teuthology.orchestra.run.smithi121.stdout:071.out 2024-04-20T10:26:42.562 INFO:teuthology.orchestra.run.smithi121.stdout:072 2024-04-20T10:26:42.563 INFO:teuthology.orchestra.run.smithi121.stdout:072.out 2024-04-20T10:26:42.563 INFO:teuthology.orchestra.run.smithi121.stdout:073 2024-04-20T10:26:42.563 INFO:teuthology.orchestra.run.smithi121.stdout:073.out 2024-04-20T10:26:42.563 INFO:teuthology.orchestra.run.smithi121.stdout:074 2024-04-20T10:26:42.563 INFO:teuthology.orchestra.run.smithi121.stdout:074.out 2024-04-20T10:26:42.563 INFO:teuthology.orchestra.run.smithi121.stdout:075 2024-04-20T10:26:42.563 INFO:teuthology.orchestra.run.smithi121.stdout:075.out 2024-04-20T10:26:42.563 INFO:teuthology.orchestra.run.smithi121.stdout:076 2024-04-20T10:26:42.563 INFO:teuthology.orchestra.run.smithi121.stdout:076.out 2024-04-20T10:26:42.563 INFO:teuthology.orchestra.run.smithi121.stdout:077 2024-04-20T10:26:42.563 INFO:teuthology.orchestra.run.smithi121.stdout:077.out 2024-04-20T10:26:42.563 INFO:teuthology.orchestra.run.smithi121.stdout:078 2024-04-20T10:26:42.563 INFO:teuthology.orchestra.run.smithi121.stdout:078.out 2024-04-20T10:26:42.563 INFO:teuthology.orchestra.run.smithi121.stdout:079 2024-04-20T10:26:42.564 INFO:teuthology.orchestra.run.smithi121.stdout:079.out 2024-04-20T10:26:42.564 INFO:teuthology.orchestra.run.smithi121.stdout:080 2024-04-20T10:26:42.564 INFO:teuthology.orchestra.run.smithi121.stdout:080.out 2024-04-20T10:26:42.564 INFO:teuthology.orchestra.run.smithi121.stdout:081 2024-04-20T10:26:42.564 INFO:teuthology.orchestra.run.smithi121.stdout:081.out 2024-04-20T10:26:42.564 INFO:teuthology.orchestra.run.smithi121.stdout:082 2024-04-20T10:26:42.564 INFO:teuthology.orchestra.run.smithi121.stdout:082.out 2024-04-20T10:26:42.564 INFO:teuthology.orchestra.run.smithi121.stdout:083 2024-04-20T10:26:42.564 INFO:teuthology.orchestra.run.smithi121.stdout:083.out 2024-04-20T10:26:42.564 INFO:teuthology.orchestra.run.smithi121.stdout:084 2024-04-20T10:26:42.564 INFO:teuthology.orchestra.run.smithi121.stdout:084.out 2024-04-20T10:26:42.564 INFO:teuthology.orchestra.run.smithi121.stdout:085 2024-04-20T10:26:42.564 INFO:teuthology.orchestra.run.smithi121.stdout:085.out 2024-04-20T10:26:42.565 INFO:teuthology.orchestra.run.smithi121.stdout:086 2024-04-20T10:26:42.565 INFO:teuthology.orchestra.run.smithi121.stdout:086.out 2024-04-20T10:26:42.565 INFO:teuthology.orchestra.run.smithi121.stdout:087 2024-04-20T10:26:42.565 INFO:teuthology.orchestra.run.smithi121.stdout:087.out 2024-04-20T10:26:42.565 INFO:teuthology.orchestra.run.smithi121.stdout:088 2024-04-20T10:26:42.565 INFO:teuthology.orchestra.run.smithi121.stdout:088.out 2024-04-20T10:26:42.565 INFO:teuthology.orchestra.run.smithi121.stdout:089 2024-04-20T10:26:42.565 INFO:teuthology.orchestra.run.smithi121.stdout:089.out 2024-04-20T10:26:42.565 INFO:teuthology.orchestra.run.smithi121.stdout:090 2024-04-20T10:26:42.565 INFO:teuthology.orchestra.run.smithi121.stdout:090.out 2024-04-20T10:26:42.565 INFO:teuthology.orchestra.run.smithi121.stdout:091 2024-04-20T10:26:42.565 INFO:teuthology.orchestra.run.smithi121.stdout:091.out 2024-04-20T10:26:42.565 INFO:teuthology.orchestra.run.smithi121.stdout:092 2024-04-20T10:26:42.565 INFO:teuthology.orchestra.run.smithi121.stdout:092.out 2024-04-20T10:26:42.566 INFO:teuthology.orchestra.run.smithi121.stdout:093 2024-04-20T10:26:42.566 INFO:teuthology.orchestra.run.smithi121.stdout:093.out 2024-04-20T10:26:42.566 INFO:teuthology.orchestra.run.smithi121.stdout:094 2024-04-20T10:26:42.566 INFO:teuthology.orchestra.run.smithi121.stdout:094.out 2024-04-20T10:26:42.566 INFO:teuthology.orchestra.run.smithi121.stdout:095 2024-04-20T10:26:42.566 INFO:teuthology.orchestra.run.smithi121.stdout:095.out 2024-04-20T10:26:42.566 INFO:teuthology.orchestra.run.smithi121.stdout:096 2024-04-20T10:26:42.566 INFO:teuthology.orchestra.run.smithi121.stdout:096.out 2024-04-20T10:26:42.566 INFO:teuthology.orchestra.run.smithi121.stdout:097 2024-04-20T10:26:42.566 INFO:teuthology.orchestra.run.smithi121.stdout:097.out 2024-04-20T10:26:42.567 INFO:teuthology.orchestra.run.smithi121.stdout:098 2024-04-20T10:26:42.567 INFO:teuthology.orchestra.run.smithi121.stdout:098.out 2024-04-20T10:26:42.567 INFO:teuthology.orchestra.run.smithi121.stdout:099 2024-04-20T10:26:42.567 INFO:teuthology.orchestra.run.smithi121.stdout:099.out 2024-04-20T10:26:42.567 INFO:teuthology.orchestra.run.smithi121.stdout:100 2024-04-20T10:26:42.567 INFO:teuthology.orchestra.run.smithi121.stdout:100.out 2024-04-20T10:26:42.567 INFO:teuthology.orchestra.run.smithi121.stdout:101 2024-04-20T10:26:42.567 INFO:teuthology.orchestra.run.smithi121.stdout:101.out 2024-04-20T10:26:42.567 INFO:teuthology.orchestra.run.smithi121.stdout:102 2024-04-20T10:26:42.567 INFO:teuthology.orchestra.run.smithi121.stdout:102.out 2024-04-20T10:26:42.567 INFO:teuthology.orchestra.run.smithi121.stdout:103 2024-04-20T10:26:42.567 INFO:teuthology.orchestra.run.smithi121.stdout:103.out 2024-04-20T10:26:42.567 INFO:teuthology.orchestra.run.smithi121.stdout:104 2024-04-20T10:26:42.567 INFO:teuthology.orchestra.run.smithi121.stdout:104.out 2024-04-20T10:26:42.568 INFO:teuthology.orchestra.run.smithi121.stdout:105 2024-04-20T10:26:42.568 INFO:teuthology.orchestra.run.smithi121.stdout:105.out 2024-04-20T10:26:42.568 INFO:teuthology.orchestra.run.smithi121.stdout:106 2024-04-20T10:26:42.568 INFO:teuthology.orchestra.run.smithi121.stdout:106.out 2024-04-20T10:26:42.568 INFO:teuthology.orchestra.run.smithi121.stdout:107 2024-04-20T10:26:42.568 INFO:teuthology.orchestra.run.smithi121.stdout:107.out 2024-04-20T10:26:42.568 INFO:teuthology.orchestra.run.smithi121.stdout:108 2024-04-20T10:26:42.568 INFO:teuthology.orchestra.run.smithi121.stdout:108.out 2024-04-20T10:26:42.568 INFO:teuthology.orchestra.run.smithi121.stdout:109 2024-04-20T10:26:42.568 INFO:teuthology.orchestra.run.smithi121.stdout:109.out 2024-04-20T10:26:42.568 INFO:teuthology.orchestra.run.smithi121.stdout:110 2024-04-20T10:26:42.568 INFO:teuthology.orchestra.run.smithi121.stdout:110.out 2024-04-20T10:26:42.568 INFO:teuthology.orchestra.run.smithi121.stdout:111 2024-04-20T10:26:42.569 INFO:teuthology.orchestra.run.smithi121.stdout:111.out 2024-04-20T10:26:42.569 INFO:teuthology.orchestra.run.smithi121.stdout:112 2024-04-20T10:26:42.569 INFO:teuthology.orchestra.run.smithi121.stdout:112.out 2024-04-20T10:26:42.569 INFO:teuthology.orchestra.run.smithi121.stdout:113 2024-04-20T10:26:42.569 INFO:teuthology.orchestra.run.smithi121.stdout:113.out 2024-04-20T10:26:42.569 INFO:teuthology.orchestra.run.smithi121.stdout:114 2024-04-20T10:26:42.569 INFO:teuthology.orchestra.run.smithi121.stdout:114.out 2024-04-20T10:26:42.569 INFO:teuthology.orchestra.run.smithi121.stdout:115 2024-04-20T10:26:42.569 INFO:teuthology.orchestra.run.smithi121.stdout:115.out 2024-04-20T10:26:42.569 INFO:teuthology.orchestra.run.smithi121.stdout:116 2024-04-20T10:26:42.569 INFO:teuthology.orchestra.run.smithi121.stdout:116.out 2024-04-20T10:26:42.570 INFO:teuthology.orchestra.run.smithi121.stdout:117 2024-04-20T10:26:42.570 INFO:teuthology.orchestra.run.smithi121.stdout:117.out 2024-04-20T10:26:42.570 INFO:teuthology.orchestra.run.smithi121.stdout:118 2024-04-20T10:26:42.570 INFO:teuthology.orchestra.run.smithi121.stdout:118.out 2024-04-20T10:26:42.570 INFO:teuthology.orchestra.run.smithi121.stdout:119 2024-04-20T10:26:42.570 INFO:teuthology.orchestra.run.smithi121.stdout:119.out 2024-04-20T10:26:42.570 INFO:teuthology.orchestra.run.smithi121.stdout:120 2024-04-20T10:26:42.570 INFO:teuthology.orchestra.run.smithi121.stdout:120.out 2024-04-20T10:26:42.570 INFO:teuthology.orchestra.run.smithi121.stdout:121 2024-04-20T10:26:42.570 INFO:teuthology.orchestra.run.smithi121.stdout:121.out 2024-04-20T10:26:42.570 INFO:teuthology.orchestra.run.smithi121.stdout:122 2024-04-20T10:26:42.570 INFO:teuthology.orchestra.run.smithi121.stdout:122.out 2024-04-20T10:26:42.570 INFO:teuthology.orchestra.run.smithi121.stdout:123 2024-04-20T10:26:42.570 INFO:teuthology.orchestra.run.smithi121.stdout:123.out 2024-04-20T10:26:42.571 INFO:teuthology.orchestra.run.smithi121.stdout:124 2024-04-20T10:26:42.571 INFO:teuthology.orchestra.run.smithi121.stdout:124.out 2024-04-20T10:26:42.571 INFO:teuthology.orchestra.run.smithi121.stdout:125 2024-04-20T10:26:42.571 INFO:teuthology.orchestra.run.smithi121.stdout:125.out 2024-04-20T10:26:42.571 INFO:teuthology.orchestra.run.smithi121.stdout:126 2024-04-20T10:26:42.571 INFO:teuthology.orchestra.run.smithi121.stdout:126.out 2024-04-20T10:26:42.571 INFO:teuthology.orchestra.run.smithi121.stdout:127 2024-04-20T10:26:42.571 INFO:teuthology.orchestra.run.smithi121.stdout:127.out 2024-04-20T10:26:42.571 INFO:teuthology.orchestra.run.smithi121.stdout:128 2024-04-20T10:26:42.571 INFO:teuthology.orchestra.run.smithi121.stdout:128.out 2024-04-20T10:26:42.572 INFO:teuthology.orchestra.run.smithi121.stdout:129 2024-04-20T10:26:42.572 INFO:teuthology.orchestra.run.smithi121.stdout:129.out 2024-04-20T10:26:42.572 INFO:teuthology.orchestra.run.smithi121.stdout:130 2024-04-20T10:26:42.572 INFO:teuthology.orchestra.run.smithi121.stdout:130.out 2024-04-20T10:26:42.572 INFO:teuthology.orchestra.run.smithi121.stdout:131 2024-04-20T10:26:42.572 INFO:teuthology.orchestra.run.smithi121.stdout:131.out 2024-04-20T10:26:42.572 INFO:teuthology.orchestra.run.smithi121.stdout:132 2024-04-20T10:26:42.572 INFO:teuthology.orchestra.run.smithi121.stdout:132.out 2024-04-20T10:26:42.572 INFO:teuthology.orchestra.run.smithi121.stdout:133 2024-04-20T10:26:42.572 INFO:teuthology.orchestra.run.smithi121.stdout:133.out 2024-04-20T10:26:42.572 INFO:teuthology.orchestra.run.smithi121.stdout:134 2024-04-20T10:26:42.572 INFO:teuthology.orchestra.run.smithi121.stdout:134.out 2024-04-20T10:26:42.572 INFO:teuthology.orchestra.run.smithi121.stdout:135 2024-04-20T10:26:42.573 INFO:teuthology.orchestra.run.smithi121.stdout:135.out 2024-04-20T10:26:42.573 INFO:teuthology.orchestra.run.smithi121.stdout:136 2024-04-20T10:26:42.573 INFO:teuthology.orchestra.run.smithi121.stdout:136.out 2024-04-20T10:26:42.573 INFO:teuthology.orchestra.run.smithi121.stdout:137 2024-04-20T10:26:42.573 INFO:teuthology.orchestra.run.smithi121.stdout:137.out 2024-04-20T10:26:42.573 INFO:teuthology.orchestra.run.smithi121.stdout:138 2024-04-20T10:26:42.573 INFO:teuthology.orchestra.run.smithi121.stdout:138.out 2024-04-20T10:26:42.573 INFO:teuthology.orchestra.run.smithi121.stdout:139 2024-04-20T10:26:42.573 INFO:teuthology.orchestra.run.smithi121.stdout:139.out 2024-04-20T10:26:42.573 INFO:teuthology.orchestra.run.smithi121.stdout:140 2024-04-20T10:26:42.573 INFO:teuthology.orchestra.run.smithi121.stdout:140.out 2024-04-20T10:26:42.573 INFO:teuthology.orchestra.run.smithi121.stdout:141 2024-04-20T10:26:42.573 INFO:teuthology.orchestra.run.smithi121.stdout:141.out 2024-04-20T10:26:42.574 INFO:teuthology.orchestra.run.smithi121.stdout:142 2024-04-20T10:26:42.574 INFO:teuthology.orchestra.run.smithi121.stdout:142.out 2024-04-20T10:26:42.574 INFO:teuthology.orchestra.run.smithi121.stdout:143 2024-04-20T10:26:42.574 INFO:teuthology.orchestra.run.smithi121.stdout:143.out 2024-04-20T10:26:42.574 INFO:teuthology.orchestra.run.smithi121.stdout:144 2024-04-20T10:26:42.574 INFO:teuthology.orchestra.run.smithi121.stdout:144.out 2024-04-20T10:26:42.574 INFO:teuthology.orchestra.run.smithi121.stdout:145 2024-04-20T10:26:42.574 INFO:teuthology.orchestra.run.smithi121.stdout:145.out 2024-04-20T10:26:42.574 INFO:teuthology.orchestra.run.smithi121.stdout:146 2024-04-20T10:26:42.574 INFO:teuthology.orchestra.run.smithi121.stdout:146.out 2024-04-20T10:26:42.574 INFO:teuthology.orchestra.run.smithi121.stdout:147 2024-04-20T10:26:42.574 INFO:teuthology.orchestra.run.smithi121.stdout:147.out 2024-04-20T10:26:42.574 INFO:teuthology.orchestra.run.smithi121.stdout:148 2024-04-20T10:26:42.575 INFO:teuthology.orchestra.run.smithi121.stdout:148.out 2024-04-20T10:26:42.575 INFO:teuthology.orchestra.run.smithi121.stdout:149 2024-04-20T10:26:42.575 INFO:teuthology.orchestra.run.smithi121.stdout:149.out 2024-04-20T10:26:42.575 INFO:teuthology.orchestra.run.smithi121.stdout:150 2024-04-20T10:26:42.575 INFO:teuthology.orchestra.run.smithi121.stdout:150.out 2024-04-20T10:26:42.575 INFO:teuthology.orchestra.run.smithi121.stdout:151 2024-04-20T10:26:42.575 INFO:teuthology.orchestra.run.smithi121.stdout:151.out 2024-04-20T10:26:42.575 INFO:teuthology.orchestra.run.smithi121.stdout:152 2024-04-20T10:26:42.575 INFO:teuthology.orchestra.run.smithi121.stdout:152.out 2024-04-20T10:26:42.575 INFO:teuthology.orchestra.run.smithi121.stdout:153 2024-04-20T10:26:42.575 INFO:teuthology.orchestra.run.smithi121.stdout:153.out 2024-04-20T10:26:42.575 INFO:teuthology.orchestra.run.smithi121.stdout:154 2024-04-20T10:26:42.575 INFO:teuthology.orchestra.run.smithi121.stdout:154.out 2024-04-20T10:26:42.575 INFO:teuthology.orchestra.run.smithi121.stdout:155 2024-04-20T10:26:42.575 INFO:teuthology.orchestra.run.smithi121.stdout:155.out 2024-04-20T10:26:42.576 INFO:teuthology.orchestra.run.smithi121.stdout:156 2024-04-20T10:26:42.576 INFO:teuthology.orchestra.run.smithi121.stdout:156.out 2024-04-20T10:26:42.576 INFO:teuthology.orchestra.run.smithi121.stdout:157 2024-04-20T10:26:42.576 INFO:teuthology.orchestra.run.smithi121.stdout:157.out 2024-04-20T10:26:42.576 INFO:teuthology.orchestra.run.smithi121.stdout:158 2024-04-20T10:26:42.576 INFO:teuthology.orchestra.run.smithi121.stdout:158.out 2024-04-20T10:26:42.576 INFO:teuthology.orchestra.run.smithi121.stdout:159 2024-04-20T10:26:42.576 INFO:teuthology.orchestra.run.smithi121.stdout:159.out 2024-04-20T10:26:42.576 INFO:teuthology.orchestra.run.smithi121.stdout:160 2024-04-20T10:26:42.576 INFO:teuthology.orchestra.run.smithi121.stdout:160.out 2024-04-20T10:26:42.576 INFO:teuthology.orchestra.run.smithi121.stdout:161 2024-04-20T10:26:42.576 INFO:teuthology.orchestra.run.smithi121.stdout:161.out 2024-04-20T10:26:42.576 INFO:teuthology.orchestra.run.smithi121.stdout:162 2024-04-20T10:26:42.576 INFO:teuthology.orchestra.run.smithi121.stdout:162.out 2024-04-20T10:26:42.576 INFO:teuthology.orchestra.run.smithi121.stdout:163 2024-04-20T10:26:42.577 INFO:teuthology.orchestra.run.smithi121.stdout:163.out 2024-04-20T10:26:42.577 INFO:teuthology.orchestra.run.smithi121.stdout:164 2024-04-20T10:26:42.577 INFO:teuthology.orchestra.run.smithi121.stdout:164.out 2024-04-20T10:26:42.577 INFO:teuthology.orchestra.run.smithi121.stdout:165 2024-04-20T10:26:42.577 INFO:teuthology.orchestra.run.smithi121.stdout:165.out 2024-04-20T10:26:42.577 INFO:teuthology.orchestra.run.smithi121.stdout:166 2024-04-20T10:26:42.577 INFO:teuthology.orchestra.run.smithi121.stdout:166.out 2024-04-20T10:26:42.577 INFO:teuthology.orchestra.run.smithi121.stdout:167 2024-04-20T10:26:42.577 INFO:teuthology.orchestra.run.smithi121.stdout:167.out 2024-04-20T10:26:42.577 INFO:teuthology.orchestra.run.smithi121.stdout:168 2024-04-20T10:26:42.577 INFO:teuthology.orchestra.run.smithi121.stdout:168.out 2024-04-20T10:26:42.577 INFO:teuthology.orchestra.run.smithi121.stdout:169 2024-04-20T10:26:42.577 INFO:teuthology.orchestra.run.smithi121.stdout:169.out 2024-04-20T10:26:42.577 INFO:teuthology.orchestra.run.smithi121.stdout:170 2024-04-20T10:26:42.577 INFO:teuthology.orchestra.run.smithi121.stdout:170.out 2024-04-20T10:26:42.578 INFO:teuthology.orchestra.run.smithi121.stdout:171 2024-04-20T10:26:42.578 INFO:teuthology.orchestra.run.smithi121.stdout:171.out 2024-04-20T10:26:42.578 INFO:teuthology.orchestra.run.smithi121.stdout:172 2024-04-20T10:26:42.578 INFO:teuthology.orchestra.run.smithi121.stdout:172.out 2024-04-20T10:26:42.578 INFO:teuthology.orchestra.run.smithi121.stdout:173 2024-04-20T10:26:42.578 INFO:teuthology.orchestra.run.smithi121.stdout:173.out 2024-04-20T10:26:42.578 INFO:teuthology.orchestra.run.smithi121.stdout:174 2024-04-20T10:26:42.578 INFO:teuthology.orchestra.run.smithi121.stdout:174.out 2024-04-20T10:26:42.578 INFO:teuthology.orchestra.run.smithi121.stdout:175 2024-04-20T10:26:42.578 INFO:teuthology.orchestra.run.smithi121.stdout:175.out 2024-04-20T10:26:42.578 INFO:teuthology.orchestra.run.smithi121.stdout:176 2024-04-20T10:26:42.578 INFO:teuthology.orchestra.run.smithi121.stdout:176.out 2024-04-20T10:26:42.578 INFO:teuthology.orchestra.run.smithi121.stdout:177 2024-04-20T10:26:42.578 INFO:teuthology.orchestra.run.smithi121.stdout:177.out 2024-04-20T10:26:42.579 INFO:teuthology.orchestra.run.smithi121.stdout:178 2024-04-20T10:26:42.579 INFO:teuthology.orchestra.run.smithi121.stdout:178.out 2024-04-20T10:26:42.579 INFO:teuthology.orchestra.run.smithi121.stdout:179 2024-04-20T10:26:42.579 INFO:teuthology.orchestra.run.smithi121.stdout:179.out 2024-04-20T10:26:42.579 INFO:teuthology.orchestra.run.smithi121.stdout:180 2024-04-20T10:26:42.579 INFO:teuthology.orchestra.run.smithi121.stdout:180.out 2024-04-20T10:26:42.579 INFO:teuthology.orchestra.run.smithi121.stdout:181 2024-04-20T10:26:42.579 INFO:teuthology.orchestra.run.smithi121.stdout:181.out 2024-04-20T10:26:42.579 INFO:teuthology.orchestra.run.smithi121.stdout:182 2024-04-20T10:26:42.579 INFO:teuthology.orchestra.run.smithi121.stdout:182.out 2024-04-20T10:26:42.579 INFO:teuthology.orchestra.run.smithi121.stdout:183 2024-04-20T10:26:42.579 INFO:teuthology.orchestra.run.smithi121.stdout:183.out 2024-04-20T10:26:42.579 INFO:teuthology.orchestra.run.smithi121.stdout:184 2024-04-20T10:26:42.579 INFO:teuthology.orchestra.run.smithi121.stdout:184.out 2024-04-20T10:26:42.580 INFO:teuthology.orchestra.run.smithi121.stdout:185 2024-04-20T10:26:42.580 INFO:teuthology.orchestra.run.smithi121.stdout:185.out 2024-04-20T10:26:42.580 INFO:teuthology.orchestra.run.smithi121.stdout:186 2024-04-20T10:26:42.580 INFO:teuthology.orchestra.run.smithi121.stdout:186.out 2024-04-20T10:26:42.580 INFO:teuthology.orchestra.run.smithi121.stdout:187 2024-04-20T10:26:42.580 INFO:teuthology.orchestra.run.smithi121.stdout:187.out 2024-04-20T10:26:42.580 INFO:teuthology.orchestra.run.smithi121.stdout:188 2024-04-20T10:26:42.580 INFO:teuthology.orchestra.run.smithi121.stdout:188.out 2024-04-20T10:26:42.580 INFO:teuthology.orchestra.run.smithi121.stdout:189 2024-04-20T10:26:42.580 INFO:teuthology.orchestra.run.smithi121.stdout:189.out 2024-04-20T10:26:42.580 INFO:teuthology.orchestra.run.smithi121.stdout:190 2024-04-20T10:26:42.580 INFO:teuthology.orchestra.run.smithi121.stdout:190.out 2024-04-20T10:26:42.580 INFO:teuthology.orchestra.run.smithi121.stdout:191 2024-04-20T10:26:42.580 INFO:teuthology.orchestra.run.smithi121.stdout:191.out 2024-04-20T10:26:42.580 INFO:teuthology.orchestra.run.smithi121.stdout:192 2024-04-20T10:26:42.581 INFO:teuthology.orchestra.run.smithi121.stdout:192.out 2024-04-20T10:26:42.581 INFO:teuthology.orchestra.run.smithi121.stdout:193 2024-04-20T10:26:42.581 INFO:teuthology.orchestra.run.smithi121.stdout:193.out 2024-04-20T10:26:42.581 INFO:teuthology.orchestra.run.smithi121.stdout:194 2024-04-20T10:26:42.581 INFO:teuthology.orchestra.run.smithi121.stdout:194.out 2024-04-20T10:26:42.581 INFO:teuthology.orchestra.run.smithi121.stdout:195 2024-04-20T10:26:42.581 INFO:teuthology.orchestra.run.smithi121.stdout:195.out 2024-04-20T10:26:42.581 INFO:teuthology.orchestra.run.smithi121.stdout:196 2024-04-20T10:26:42.581 INFO:teuthology.orchestra.run.smithi121.stdout:196.out 2024-04-20T10:26:42.581 INFO:teuthology.orchestra.run.smithi121.stdout:197 2024-04-20T10:26:42.581 INFO:teuthology.orchestra.run.smithi121.stdout:197.out 2024-04-20T10:26:42.581 INFO:teuthology.orchestra.run.smithi121.stdout:198 2024-04-20T10:26:42.581 INFO:teuthology.orchestra.run.smithi121.stdout:198.out 2024-04-20T10:26:42.581 INFO:teuthology.orchestra.run.smithi121.stdout:199 2024-04-20T10:26:42.581 INFO:teuthology.orchestra.run.smithi121.stdout:199.out 2024-04-20T10:26:42.582 INFO:teuthology.orchestra.run.smithi121.stdout:200 2024-04-20T10:26:42.582 INFO:teuthology.orchestra.run.smithi121.stdout:200.out 2024-04-20T10:26:42.582 INFO:teuthology.orchestra.run.smithi121.stdout:201 2024-04-20T10:26:42.582 INFO:teuthology.orchestra.run.smithi121.stdout:201.out 2024-04-20T10:26:42.582 INFO:teuthology.orchestra.run.smithi121.stdout:202 2024-04-20T10:26:42.582 INFO:teuthology.orchestra.run.smithi121.stdout:202.out 2024-04-20T10:26:42.582 INFO:teuthology.orchestra.run.smithi121.stdout:203 2024-04-20T10:26:42.582 INFO:teuthology.orchestra.run.smithi121.stdout:203.out 2024-04-20T10:26:42.582 INFO:teuthology.orchestra.run.smithi121.stdout:204 2024-04-20T10:26:42.583 INFO:teuthology.orchestra.run.smithi121.stdout:204.out 2024-04-20T10:26:42.583 INFO:teuthology.orchestra.run.smithi121.stdout:205 2024-04-20T10:26:42.583 INFO:teuthology.orchestra.run.smithi121.stdout:205.out 2024-04-20T10:26:42.583 INFO:teuthology.orchestra.run.smithi121.stdout:206 2024-04-20T10:26:42.583 INFO:teuthology.orchestra.run.smithi121.stdout:206.out 2024-04-20T10:26:42.583 INFO:teuthology.orchestra.run.smithi121.stdout:207 2024-04-20T10:26:42.583 INFO:teuthology.orchestra.run.smithi121.stdout:207.out 2024-04-20T10:26:42.583 INFO:teuthology.orchestra.run.smithi121.stdout:208 2024-04-20T10:26:42.583 INFO:teuthology.orchestra.run.smithi121.stdout:208.out 2024-04-20T10:26:42.583 INFO:teuthology.orchestra.run.smithi121.stdout:209 2024-04-20T10:26:42.583 INFO:teuthology.orchestra.run.smithi121.stdout:209.out 2024-04-20T10:26:42.583 INFO:teuthology.orchestra.run.smithi121.stdout:210 2024-04-20T10:26:42.583 INFO:teuthology.orchestra.run.smithi121.stdout:210.out 2024-04-20T10:26:42.583 INFO:teuthology.orchestra.run.smithi121.stdout:211 2024-04-20T10:26:42.583 INFO:teuthology.orchestra.run.smithi121.stdout:211.out 2024-04-20T10:26:42.584 INFO:teuthology.orchestra.run.smithi121.stdout:212 2024-04-20T10:26:42.584 INFO:teuthology.orchestra.run.smithi121.stdout:212.out 2024-04-20T10:26:42.584 INFO:teuthology.orchestra.run.smithi121.stdout:213 2024-04-20T10:26:42.584 INFO:teuthology.orchestra.run.smithi121.stdout:213.out 2024-04-20T10:26:42.584 INFO:teuthology.orchestra.run.smithi121.stdout:214 2024-04-20T10:26:42.584 INFO:teuthology.orchestra.run.smithi121.stdout:214.out 2024-04-20T10:26:42.584 INFO:teuthology.orchestra.run.smithi121.stdout:215 2024-04-20T10:26:42.584 INFO:teuthology.orchestra.run.smithi121.stdout:215.out 2024-04-20T10:26:42.584 INFO:teuthology.orchestra.run.smithi121.stdout:216 2024-04-20T10:26:42.584 INFO:teuthology.orchestra.run.smithi121.stdout:216.out 2024-04-20T10:26:42.584 INFO:teuthology.orchestra.run.smithi121.stdout:217 2024-04-20T10:26:42.584 INFO:teuthology.orchestra.run.smithi121.stdout:217.out 2024-04-20T10:26:42.585 INFO:teuthology.orchestra.run.smithi121.stdout:218 2024-04-20T10:26:42.585 INFO:teuthology.orchestra.run.smithi121.stdout:218.out 2024-04-20T10:26:42.585 INFO:teuthology.orchestra.run.smithi121.stdout:219 2024-04-20T10:26:42.585 INFO:teuthology.orchestra.run.smithi121.stdout:219.out 2024-04-20T10:26:42.585 INFO:teuthology.orchestra.run.smithi121.stdout:220 2024-04-20T10:26:42.585 INFO:teuthology.orchestra.run.smithi121.stdout:220.out 2024-04-20T10:26:42.585 INFO:teuthology.orchestra.run.smithi121.stdout:221 2024-04-20T10:26:42.585 INFO:teuthology.orchestra.run.smithi121.stdout:221.out 2024-04-20T10:26:42.585 INFO:teuthology.orchestra.run.smithi121.stdout:222 2024-04-20T10:26:42.585 INFO:teuthology.orchestra.run.smithi121.stdout:222.out 2024-04-20T10:26:42.585 INFO:teuthology.orchestra.run.smithi121.stdout:223 2024-04-20T10:26:42.586 INFO:teuthology.orchestra.run.smithi121.stdout:223.out 2024-04-20T10:26:42.586 INFO:teuthology.orchestra.run.smithi121.stdout:224 2024-04-20T10:26:42.586 INFO:teuthology.orchestra.run.smithi121.stdout:224.out 2024-04-20T10:26:42.586 INFO:teuthology.orchestra.run.smithi121.stdout:225 2024-04-20T10:26:42.586 INFO:teuthology.orchestra.run.smithi121.stdout:225.out 2024-04-20T10:26:42.586 INFO:teuthology.orchestra.run.smithi121.stdout:226 2024-04-20T10:26:42.586 INFO:teuthology.orchestra.run.smithi121.stdout:226.out 2024-04-20T10:26:42.586 INFO:teuthology.orchestra.run.smithi121.stdout:227 2024-04-20T10:26:42.586 INFO:teuthology.orchestra.run.smithi121.stdout:227.out 2024-04-20T10:26:42.586 INFO:teuthology.orchestra.run.smithi121.stdout:228 2024-04-20T10:26:42.586 INFO:teuthology.orchestra.run.smithi121.stdout:228.out 2024-04-20T10:26:42.586 INFO:teuthology.orchestra.run.smithi121.stdout:229 2024-04-20T10:26:42.586 INFO:teuthology.orchestra.run.smithi121.stdout:229.out 2024-04-20T10:26:42.587 INFO:teuthology.orchestra.run.smithi121.stdout:230 2024-04-20T10:26:42.587 INFO:teuthology.orchestra.run.smithi121.stdout:230.out 2024-04-20T10:26:42.587 INFO:teuthology.orchestra.run.smithi121.stdout:231 2024-04-20T10:26:42.587 INFO:teuthology.orchestra.run.smithi121.stdout:231.out 2024-04-20T10:26:42.587 INFO:teuthology.orchestra.run.smithi121.stdout:232 2024-04-20T10:26:42.587 INFO:teuthology.orchestra.run.smithi121.stdout:232.out 2024-04-20T10:26:42.587 INFO:teuthology.orchestra.run.smithi121.stdout:233 2024-04-20T10:26:42.587 INFO:teuthology.orchestra.run.smithi121.stdout:233.out 2024-04-20T10:26:42.587 INFO:teuthology.orchestra.run.smithi121.stdout:234 2024-04-20T10:26:42.587 INFO:teuthology.orchestra.run.smithi121.stdout:234.out 2024-04-20T10:26:42.587 INFO:teuthology.orchestra.run.smithi121.stdout:235 2024-04-20T10:26:42.587 INFO:teuthology.orchestra.run.smithi121.stdout:235.out 2024-04-20T10:26:42.587 INFO:teuthology.orchestra.run.smithi121.stdout:236 2024-04-20T10:26:42.587 INFO:teuthology.orchestra.run.smithi121.stdout:236.out 2024-04-20T10:26:42.587 INFO:teuthology.orchestra.run.smithi121.stdout:237 2024-04-20T10:26:42.588 INFO:teuthology.orchestra.run.smithi121.stdout:237.out 2024-04-20T10:26:42.588 INFO:teuthology.orchestra.run.smithi121.stdout:238 2024-04-20T10:26:42.588 INFO:teuthology.orchestra.run.smithi121.stdout:238.out 2024-04-20T10:26:42.588 INFO:teuthology.orchestra.run.smithi121.stdout:239 2024-04-20T10:26:42.588 INFO:teuthology.orchestra.run.smithi121.stdout:239.out 2024-04-20T10:26:42.588 INFO:teuthology.orchestra.run.smithi121.stdout:240 2024-04-20T10:26:42.588 INFO:teuthology.orchestra.run.smithi121.stdout:240.out 2024-04-20T10:26:42.588 INFO:teuthology.orchestra.run.smithi121.stdout:241 2024-04-20T10:26:42.588 INFO:teuthology.orchestra.run.smithi121.stdout:241.out 2024-04-20T10:26:42.588 INFO:teuthology.orchestra.run.smithi121.stdout:242 2024-04-20T10:26:42.588 INFO:teuthology.orchestra.run.smithi121.stdout:242.out 2024-04-20T10:26:42.588 INFO:teuthology.orchestra.run.smithi121.stdout:243 2024-04-20T10:26:42.588 INFO:teuthology.orchestra.run.smithi121.stdout:243.out 2024-04-20T10:26:42.588 INFO:teuthology.orchestra.run.smithi121.stdout:244 2024-04-20T10:26:42.588 INFO:teuthology.orchestra.run.smithi121.stdout:244.out 2024-04-20T10:26:42.589 INFO:teuthology.orchestra.run.smithi121.stdout:245 2024-04-20T10:26:42.589 INFO:teuthology.orchestra.run.smithi121.stdout:245.out 2024-04-20T10:26:42.589 INFO:teuthology.orchestra.run.smithi121.stdout:246 2024-04-20T10:26:42.589 INFO:teuthology.orchestra.run.smithi121.stdout:246.out 2024-04-20T10:26:42.589 INFO:teuthology.orchestra.run.smithi121.stdout:247 2024-04-20T10:26:42.589 INFO:teuthology.orchestra.run.smithi121.stdout:247.out 2024-04-20T10:26:42.589 INFO:teuthology.orchestra.run.smithi121.stdout:248 2024-04-20T10:26:42.589 INFO:teuthology.orchestra.run.smithi121.stdout:248.out 2024-04-20T10:26:42.589 INFO:teuthology.orchestra.run.smithi121.stdout:249 2024-04-20T10:26:42.589 INFO:teuthology.orchestra.run.smithi121.stdout:249.out 2024-04-20T10:26:42.589 INFO:teuthology.orchestra.run.smithi121.stdout:250 2024-04-20T10:26:42.589 INFO:teuthology.orchestra.run.smithi121.stdout:250.out 2024-04-20T10:26:42.589 INFO:teuthology.orchestra.run.smithi121.stdout:251 2024-04-20T10:26:42.589 INFO:teuthology.orchestra.run.smithi121.stdout:251.out 2024-04-20T10:26:42.589 INFO:teuthology.orchestra.run.smithi121.stdout:252 2024-04-20T10:26:42.590 INFO:teuthology.orchestra.run.smithi121.stdout:252.out 2024-04-20T10:26:42.590 INFO:teuthology.orchestra.run.smithi121.stdout:253 2024-04-20T10:26:42.590 INFO:teuthology.orchestra.run.smithi121.stdout:253.out 2024-04-20T10:26:42.590 INFO:teuthology.orchestra.run.smithi121.stdout:254 2024-04-20T10:26:42.590 INFO:teuthology.orchestra.run.smithi121.stdout:254.out 2024-04-20T10:26:42.590 INFO:teuthology.orchestra.run.smithi121.stdout:255 2024-04-20T10:26:42.590 INFO:teuthology.orchestra.run.smithi121.stdout:255.out 2024-04-20T10:26:42.590 INFO:teuthology.orchestra.run.smithi121.stdout:256 2024-04-20T10:26:42.590 INFO:teuthology.orchestra.run.smithi121.stdout:256.out 2024-04-20T10:26:42.590 INFO:teuthology.orchestra.run.smithi121.stdout:257 2024-04-20T10:26:42.590 INFO:teuthology.orchestra.run.smithi121.stdout:257.out 2024-04-20T10:26:42.591 INFO:teuthology.orchestra.run.smithi121.stdout:258 2024-04-20T10:26:42.591 INFO:teuthology.orchestra.run.smithi121.stdout:258.out 2024-04-20T10:26:42.591 INFO:teuthology.orchestra.run.smithi121.stdout:259 2024-04-20T10:26:42.591 INFO:teuthology.orchestra.run.smithi121.stdout:259.out 2024-04-20T10:26:42.591 INFO:teuthology.orchestra.run.smithi121.stdout:260 2024-04-20T10:26:42.591 INFO:teuthology.orchestra.run.smithi121.stdout:260.out 2024-04-20T10:26:42.591 INFO:teuthology.orchestra.run.smithi121.stdout:261 2024-04-20T10:26:42.591 INFO:teuthology.orchestra.run.smithi121.stdout:261.out 2024-04-20T10:26:42.591 INFO:teuthology.orchestra.run.smithi121.stdout:262 2024-04-20T10:26:42.591 INFO:teuthology.orchestra.run.smithi121.stdout:262.out 2024-04-20T10:26:42.591 INFO:teuthology.orchestra.run.smithi121.stdout:263 2024-04-20T10:26:42.591 INFO:teuthology.orchestra.run.smithi121.stdout:263.out 2024-04-20T10:26:42.591 INFO:teuthology.orchestra.run.smithi121.stdout:264 2024-04-20T10:26:42.591 INFO:teuthology.orchestra.run.smithi121.stdout:264.out 2024-04-20T10:26:42.592 INFO:teuthology.orchestra.run.smithi121.stdout:265 2024-04-20T10:26:42.592 INFO:teuthology.orchestra.run.smithi121.stdout:265.out 2024-04-20T10:26:42.592 INFO:teuthology.orchestra.run.smithi121.stdout:266 2024-04-20T10:26:42.592 INFO:teuthology.orchestra.run.smithi121.stdout:266.out 2024-04-20T10:26:42.592 INFO:teuthology.orchestra.run.smithi121.stdout:267 2024-04-20T10:26:42.592 INFO:teuthology.orchestra.run.smithi121.stdout:267.out 2024-04-20T10:26:42.592 INFO:teuthology.orchestra.run.smithi121.stdout:268 2024-04-20T10:26:42.592 INFO:teuthology.orchestra.run.smithi121.stdout:268.out 2024-04-20T10:26:42.592 INFO:teuthology.orchestra.run.smithi121.stdout:269 2024-04-20T10:26:42.592 INFO:teuthology.orchestra.run.smithi121.stdout:269.out 2024-04-20T10:26:42.592 INFO:teuthology.orchestra.run.smithi121.stdout:270 2024-04-20T10:26:42.593 INFO:teuthology.orchestra.run.smithi121.stdout:270.out 2024-04-20T10:26:42.593 INFO:teuthology.orchestra.run.smithi121.stdout:271 2024-04-20T10:26:42.593 INFO:teuthology.orchestra.run.smithi121.stdout:271.out 2024-04-20T10:26:42.593 INFO:teuthology.orchestra.run.smithi121.stdout:272 2024-04-20T10:26:42.593 INFO:teuthology.orchestra.run.smithi121.stdout:272.out 2024-04-20T10:26:42.593 INFO:teuthology.orchestra.run.smithi121.stdout:273 2024-04-20T10:26:42.593 INFO:teuthology.orchestra.run.smithi121.stdout:273.out 2024-04-20T10:26:42.593 INFO:teuthology.orchestra.run.smithi121.stdout:274 2024-04-20T10:26:42.593 INFO:teuthology.orchestra.run.smithi121.stdout:274.out 2024-04-20T10:26:42.593 INFO:teuthology.orchestra.run.smithi121.stdout:275 2024-04-20T10:26:42.593 INFO:teuthology.orchestra.run.smithi121.stdout:275.out 2024-04-20T10:26:42.593 INFO:teuthology.orchestra.run.smithi121.stdout:276 2024-04-20T10:26:42.593 INFO:teuthology.orchestra.run.smithi121.stdout:276.out 2024-04-20T10:26:42.594 INFO:teuthology.orchestra.run.smithi121.stdout:277 2024-04-20T10:26:42.594 INFO:teuthology.orchestra.run.smithi121.stdout:277.out 2024-04-20T10:26:42.594 INFO:teuthology.orchestra.run.smithi121.stdout:278 2024-04-20T10:26:42.594 INFO:teuthology.orchestra.run.smithi121.stdout:278.out 2024-04-20T10:26:42.594 INFO:teuthology.orchestra.run.smithi121.stdout:279 2024-04-20T10:26:42.594 INFO:teuthology.orchestra.run.smithi121.stdout:279.out 2024-04-20T10:26:42.594 INFO:teuthology.orchestra.run.smithi121.stdout:280 2024-04-20T10:26:42.594 INFO:teuthology.orchestra.run.smithi121.stdout:280.out 2024-04-20T10:26:42.594 INFO:teuthology.orchestra.run.smithi121.stdout:281 2024-04-20T10:26:42.594 INFO:teuthology.orchestra.run.smithi121.stdout:281.out 2024-04-20T10:26:42.594 INFO:teuthology.orchestra.run.smithi121.stdout:282 2024-04-20T10:26:42.594 INFO:teuthology.orchestra.run.smithi121.stdout:282.out 2024-04-20T10:26:42.594 INFO:teuthology.orchestra.run.smithi121.stdout:283 2024-04-20T10:26:42.594 INFO:teuthology.orchestra.run.smithi121.stdout:283.out 2024-04-20T10:26:42.595 INFO:teuthology.orchestra.run.smithi121.stdout:284 2024-04-20T10:26:42.595 INFO:teuthology.orchestra.run.smithi121.stdout:284.out 2024-04-20T10:26:42.595 INFO:teuthology.orchestra.run.smithi121.stdout:285 2024-04-20T10:26:42.595 INFO:teuthology.orchestra.run.smithi121.stdout:285.out 2024-04-20T10:26:42.595 INFO:teuthology.orchestra.run.smithi121.stdout:286 2024-04-20T10:26:42.595 INFO:teuthology.orchestra.run.smithi121.stdout:286.out 2024-04-20T10:26:42.596 INFO:teuthology.orchestra.run.smithi121.stdout:287 2024-04-20T10:26:42.596 INFO:teuthology.orchestra.run.smithi121.stdout:287.out 2024-04-20T10:26:42.596 INFO:teuthology.orchestra.run.smithi121.stdout:288 2024-04-20T10:26:42.596 INFO:teuthology.orchestra.run.smithi121.stdout:288.out 2024-04-20T10:26:42.596 INFO:teuthology.orchestra.run.smithi121.stdout:289 2024-04-20T10:26:42.596 INFO:teuthology.orchestra.run.smithi121.stdout:289.out 2024-04-20T10:26:42.596 INFO:teuthology.orchestra.run.smithi121.stdout:290 2024-04-20T10:26:42.596 INFO:teuthology.orchestra.run.smithi121.stdout:290.out 2024-04-20T10:26:42.596 INFO:teuthology.orchestra.run.smithi121.stdout:291 2024-04-20T10:26:42.597 INFO:teuthology.orchestra.run.smithi121.stdout:291.out 2024-04-20T10:26:42.597 INFO:teuthology.orchestra.run.smithi121.stdout:292 2024-04-20T10:26:42.597 INFO:teuthology.orchestra.run.smithi121.stdout:292.out 2024-04-20T10:26:42.597 INFO:teuthology.orchestra.run.smithi121.stdout:293 2024-04-20T10:26:42.597 INFO:teuthology.orchestra.run.smithi121.stdout:293.out 2024-04-20T10:26:42.597 INFO:teuthology.orchestra.run.smithi121.stdout:294 2024-04-20T10:26:42.597 INFO:teuthology.orchestra.run.smithi121.stdout:294.out 2024-04-20T10:26:42.597 INFO:teuthology.orchestra.run.smithi121.stdout:295 2024-04-20T10:26:42.597 INFO:teuthology.orchestra.run.smithi121.stdout:295.out 2024-04-20T10:26:42.597 INFO:teuthology.orchestra.run.smithi121.stdout:296 2024-04-20T10:26:42.597 INFO:teuthology.orchestra.run.smithi121.stdout:296.out 2024-04-20T10:26:42.597 INFO:teuthology.orchestra.run.smithi121.stdout:297 2024-04-20T10:26:42.597 INFO:teuthology.orchestra.run.smithi121.stdout:297.out 2024-04-20T10:26:42.597 INFO:teuthology.orchestra.run.smithi121.stdout:298 2024-04-20T10:26:42.598 INFO:teuthology.orchestra.run.smithi121.stdout:298.out 2024-04-20T10:26:42.598 INFO:teuthology.orchestra.run.smithi121.stdout:299 2024-04-20T10:26:42.598 INFO:teuthology.orchestra.run.smithi121.stdout:299.out 2024-04-20T10:26:42.598 INFO:teuthology.orchestra.run.smithi121.stdout:300 2024-04-20T10:26:42.598 INFO:teuthology.orchestra.run.smithi121.stdout:300.out 2024-04-20T10:26:42.598 INFO:teuthology.orchestra.run.smithi121.stdout:301 2024-04-20T10:26:42.598 INFO:teuthology.orchestra.run.smithi121.stdout:301.out 2024-04-20T10:26:42.598 INFO:teuthology.orchestra.run.smithi121.stdout:302 2024-04-20T10:26:42.598 INFO:teuthology.orchestra.run.smithi121.stdout:302.out 2024-04-20T10:26:42.598 INFO:teuthology.orchestra.run.smithi121.stdout:303 2024-04-20T10:26:42.598 INFO:teuthology.orchestra.run.smithi121.stdout:303.out 2024-04-20T10:26:42.599 INFO:teuthology.orchestra.run.smithi121.stdout:304 2024-04-20T10:26:42.599 INFO:teuthology.orchestra.run.smithi121.stdout:304.out 2024-04-20T10:26:42.599 INFO:teuthology.orchestra.run.smithi121.stdout:305 2024-04-20T10:26:42.599 INFO:teuthology.orchestra.run.smithi121.stdout:305.out 2024-04-20T10:26:42.599 INFO:teuthology.orchestra.run.smithi121.stdout:306 2024-04-20T10:26:42.599 INFO:teuthology.orchestra.run.smithi121.stdout:306.out 2024-04-20T10:26:42.599 INFO:teuthology.orchestra.run.smithi121.stdout:307 2024-04-20T10:26:42.599 INFO:teuthology.orchestra.run.smithi121.stdout:307.out 2024-04-20T10:26:42.599 INFO:teuthology.orchestra.run.smithi121.stdout:308 2024-04-20T10:26:42.599 INFO:teuthology.orchestra.run.smithi121.stdout:308.out 2024-04-20T10:26:42.599 INFO:teuthology.orchestra.run.smithi121.stdout:309 2024-04-20T10:26:42.599 INFO:teuthology.orchestra.run.smithi121.stdout:309.out 2024-04-20T10:26:42.599 INFO:teuthology.orchestra.run.smithi121.stdout:310 2024-04-20T10:26:42.599 INFO:teuthology.orchestra.run.smithi121.stdout:310.out 2024-04-20T10:26:42.599 INFO:teuthology.orchestra.run.smithi121.stdout:311 2024-04-20T10:26:42.600 INFO:teuthology.orchestra.run.smithi121.stdout:311.out 2024-04-20T10:26:42.600 INFO:teuthology.orchestra.run.smithi121.stdout:312 2024-04-20T10:26:42.600 INFO:teuthology.orchestra.run.smithi121.stdout:312.out 2024-04-20T10:26:42.600 INFO:teuthology.orchestra.run.smithi121.stdout:313 2024-04-20T10:26:42.600 INFO:teuthology.orchestra.run.smithi121.stdout:313.out 2024-04-20T10:26:42.600 INFO:teuthology.orchestra.run.smithi121.stdout:314 2024-04-20T10:26:42.600 INFO:teuthology.orchestra.run.smithi121.stdout:314.out 2024-04-20T10:26:42.600 INFO:teuthology.orchestra.run.smithi121.stdout:315 2024-04-20T10:26:42.600 INFO:teuthology.orchestra.run.smithi121.stdout:315.out 2024-04-20T10:26:42.600 INFO:teuthology.orchestra.run.smithi121.stdout:316 2024-04-20T10:26:42.600 INFO:teuthology.orchestra.run.smithi121.stdout:316.out 2024-04-20T10:26:42.600 INFO:teuthology.orchestra.run.smithi121.stdout:317 2024-04-20T10:26:42.600 INFO:teuthology.orchestra.run.smithi121.stdout:317.out 2024-04-20T10:26:42.600 INFO:teuthology.orchestra.run.smithi121.stdout:318 2024-04-20T10:26:42.601 INFO:teuthology.orchestra.run.smithi121.stdout:318.out 2024-04-20T10:26:42.601 INFO:teuthology.orchestra.run.smithi121.stdout:319 2024-04-20T10:26:42.601 INFO:teuthology.orchestra.run.smithi121.stdout:319.out 2024-04-20T10:26:42.601 INFO:teuthology.orchestra.run.smithi121.stdout:320 2024-04-20T10:26:42.601 INFO:teuthology.orchestra.run.smithi121.stdout:320.out 2024-04-20T10:26:42.601 INFO:teuthology.orchestra.run.smithi121.stdout:321 2024-04-20T10:26:42.601 INFO:teuthology.orchestra.run.smithi121.stdout:321.out 2024-04-20T10:26:42.601 INFO:teuthology.orchestra.run.smithi121.stdout:322 2024-04-20T10:26:42.601 INFO:teuthology.orchestra.run.smithi121.stdout:322.out 2024-04-20T10:26:42.601 INFO:teuthology.orchestra.run.smithi121.stdout:323 2024-04-20T10:26:42.601 INFO:teuthology.orchestra.run.smithi121.stdout:323.out 2024-04-20T10:26:42.601 INFO:teuthology.orchestra.run.smithi121.stdout:324 2024-04-20T10:26:42.601 INFO:teuthology.orchestra.run.smithi121.stdout:324.out 2024-04-20T10:26:42.601 INFO:teuthology.orchestra.run.smithi121.stdout:325 2024-04-20T10:26:42.601 INFO:teuthology.orchestra.run.smithi121.stdout:325.out 2024-04-20T10:26:42.601 INFO:teuthology.orchestra.run.smithi121.stdout:326 2024-04-20T10:26:42.602 INFO:teuthology.orchestra.run.smithi121.stdout:326.out 2024-04-20T10:26:42.602 INFO:teuthology.orchestra.run.smithi121.stdout:327 2024-04-20T10:26:42.602 INFO:teuthology.orchestra.run.smithi121.stdout:327.out 2024-04-20T10:26:42.602 INFO:teuthology.orchestra.run.smithi121.stdout:328 2024-04-20T10:26:42.602 INFO:teuthology.orchestra.run.smithi121.stdout:328.out 2024-04-20T10:26:42.602 INFO:teuthology.orchestra.run.smithi121.stdout:329 2024-04-20T10:26:42.602 INFO:teuthology.orchestra.run.smithi121.stdout:329.out 2024-04-20T10:26:42.602 INFO:teuthology.orchestra.run.smithi121.stdout:330 2024-04-20T10:26:42.602 INFO:teuthology.orchestra.run.smithi121.stdout:330.out 2024-04-20T10:26:42.602 INFO:teuthology.orchestra.run.smithi121.stdout:331 2024-04-20T10:26:42.602 INFO:teuthology.orchestra.run.smithi121.stdout:331.out 2024-04-20T10:26:42.602 INFO:teuthology.orchestra.run.smithi121.stdout:332 2024-04-20T10:26:42.602 INFO:teuthology.orchestra.run.smithi121.stdout:332.out 2024-04-20T10:26:42.602 INFO:teuthology.orchestra.run.smithi121.stdout:333 2024-04-20T10:26:42.603 INFO:teuthology.orchestra.run.smithi121.stdout:333.out 2024-04-20T10:26:42.603 INFO:teuthology.orchestra.run.smithi121.stdout:334 2024-04-20T10:26:42.603 INFO:teuthology.orchestra.run.smithi121.stdout:334.out 2024-04-20T10:26:42.603 INFO:teuthology.orchestra.run.smithi121.stdout:335 2024-04-20T10:26:42.603 INFO:teuthology.orchestra.run.smithi121.stdout:335.out 2024-04-20T10:26:42.603 INFO:teuthology.orchestra.run.smithi121.stdout:336 2024-04-20T10:26:42.603 INFO:teuthology.orchestra.run.smithi121.stdout:336.out 2024-04-20T10:26:42.603 INFO:teuthology.orchestra.run.smithi121.stdout:337 2024-04-20T10:26:42.603 INFO:teuthology.orchestra.run.smithi121.stdout:337.out 2024-04-20T10:26:42.603 INFO:teuthology.orchestra.run.smithi121.stdout:338 2024-04-20T10:26:42.604 INFO:teuthology.orchestra.run.smithi121.stdout:338.out 2024-04-20T10:26:42.604 INFO:teuthology.orchestra.run.smithi121.stdout:339 2024-04-20T10:26:42.604 INFO:teuthology.orchestra.run.smithi121.stdout:339.out 2024-04-20T10:26:42.604 INFO:teuthology.orchestra.run.smithi121.stdout:340 2024-04-20T10:26:42.604 INFO:teuthology.orchestra.run.smithi121.stdout:340.out 2024-04-20T10:26:42.604 INFO:teuthology.orchestra.run.smithi121.stdout:341 2024-04-20T10:26:42.604 INFO:teuthology.orchestra.run.smithi121.stdout:341.out 2024-04-20T10:26:42.604 INFO:teuthology.orchestra.run.smithi121.stdout:342 2024-04-20T10:26:42.604 INFO:teuthology.orchestra.run.smithi121.stdout:342.out 2024-04-20T10:26:42.604 INFO:teuthology.orchestra.run.smithi121.stdout:343 2024-04-20T10:26:42.604 INFO:teuthology.orchestra.run.smithi121.stdout:343.out 2024-04-20T10:26:42.604 INFO:teuthology.orchestra.run.smithi121.stdout:344 2024-04-20T10:26:42.604 INFO:teuthology.orchestra.run.smithi121.stdout:344.out 2024-04-20T10:26:42.604 INFO:teuthology.orchestra.run.smithi121.stdout:345 2024-04-20T10:26:42.605 INFO:teuthology.orchestra.run.smithi121.stdout:345.out 2024-04-20T10:26:42.605 INFO:teuthology.orchestra.run.smithi121.stdout:346 2024-04-20T10:26:42.605 INFO:teuthology.orchestra.run.smithi121.stdout:346.out 2024-04-20T10:26:42.605 INFO:teuthology.orchestra.run.smithi121.stdout:347 2024-04-20T10:26:42.605 INFO:teuthology.orchestra.run.smithi121.stdout:347.out 2024-04-20T10:26:42.605 INFO:teuthology.orchestra.run.smithi121.stdout:348 2024-04-20T10:26:42.605 INFO:teuthology.orchestra.run.smithi121.stdout:348.out 2024-04-20T10:26:42.605 INFO:teuthology.orchestra.run.smithi121.stdout:349 2024-04-20T10:26:42.605 INFO:teuthology.orchestra.run.smithi121.stdout:349.out 2024-04-20T10:26:42.605 INFO:teuthology.orchestra.run.smithi121.stdout:350 2024-04-20T10:26:42.605 INFO:teuthology.orchestra.run.smithi121.stdout:350.out 2024-04-20T10:26:42.605 INFO:teuthology.orchestra.run.smithi121.stdout:351 2024-04-20T10:26:42.605 INFO:teuthology.orchestra.run.smithi121.stdout:351.out 2024-04-20T10:26:42.605 INFO:teuthology.orchestra.run.smithi121.stdout:352 2024-04-20T10:26:42.605 INFO:teuthology.orchestra.run.smithi121.stdout:352.out 2024-04-20T10:26:42.606 INFO:teuthology.orchestra.run.smithi121.stdout:353 2024-04-20T10:26:42.606 INFO:teuthology.orchestra.run.smithi121.stdout:353.out 2024-04-20T10:26:42.606 INFO:teuthology.orchestra.run.smithi121.stdout:354 2024-04-20T10:26:42.606 INFO:teuthology.orchestra.run.smithi121.stdout:354.out 2024-04-20T10:26:42.606 INFO:teuthology.orchestra.run.smithi121.stdout:355 2024-04-20T10:26:42.606 INFO:teuthology.orchestra.run.smithi121.stdout:355.out 2024-04-20T10:26:42.606 INFO:teuthology.orchestra.run.smithi121.stdout:356 2024-04-20T10:26:42.606 INFO:teuthology.orchestra.run.smithi121.stdout:356.out 2024-04-20T10:26:42.606 INFO:teuthology.orchestra.run.smithi121.stdout:357 2024-04-20T10:26:42.606 INFO:teuthology.orchestra.run.smithi121.stdout:357.out 2024-04-20T10:26:42.607 INFO:teuthology.orchestra.run.smithi121.stdout:358 2024-04-20T10:26:42.607 INFO:teuthology.orchestra.run.smithi121.stdout:358.out 2024-04-20T10:26:42.607 INFO:teuthology.orchestra.run.smithi121.stdout:359 2024-04-20T10:26:42.607 INFO:teuthology.orchestra.run.smithi121.stdout:359.out 2024-04-20T10:26:42.607 INFO:teuthology.orchestra.run.smithi121.stdout:360 2024-04-20T10:26:42.607 INFO:teuthology.orchestra.run.smithi121.stdout:360.out 2024-04-20T10:26:42.607 INFO:teuthology.orchestra.run.smithi121.stdout:361 2024-04-20T10:26:42.607 INFO:teuthology.orchestra.run.smithi121.stdout:361.out 2024-04-20T10:26:42.607 INFO:teuthology.orchestra.run.smithi121.stdout:362 2024-04-20T10:26:42.607 INFO:teuthology.orchestra.run.smithi121.stdout:362.out 2024-04-20T10:26:42.607 INFO:teuthology.orchestra.run.smithi121.stdout:363 2024-04-20T10:26:42.607 INFO:teuthology.orchestra.run.smithi121.stdout:363.out 2024-04-20T10:26:42.607 INFO:teuthology.orchestra.run.smithi121.stdout:364 2024-04-20T10:26:42.607 INFO:teuthology.orchestra.run.smithi121.stdout:364.out 2024-04-20T10:26:42.607 INFO:teuthology.orchestra.run.smithi121.stdout:365 2024-04-20T10:26:42.607 INFO:teuthology.orchestra.run.smithi121.stdout:365.out 2024-04-20T10:26:42.608 INFO:teuthology.orchestra.run.smithi121.stdout:366 2024-04-20T10:26:42.608 INFO:teuthology.orchestra.run.smithi121.stdout:366.out 2024-04-20T10:26:42.608 INFO:teuthology.orchestra.run.smithi121.stdout:367 2024-04-20T10:26:42.608 INFO:teuthology.orchestra.run.smithi121.stdout:367.out 2024-04-20T10:26:42.608 INFO:teuthology.orchestra.run.smithi121.stdout:368 2024-04-20T10:26:42.608 INFO:teuthology.orchestra.run.smithi121.stdout:368.out 2024-04-20T10:26:42.608 INFO:teuthology.orchestra.run.smithi121.stdout:369 2024-04-20T10:26:42.608 INFO:teuthology.orchestra.run.smithi121.stdout:369.out 2024-04-20T10:26:42.608 INFO:teuthology.orchestra.run.smithi121.stdout:370 2024-04-20T10:26:42.608 INFO:teuthology.orchestra.run.smithi121.stdout:370.out 2024-04-20T10:26:42.608 INFO:teuthology.orchestra.run.smithi121.stdout:371 2024-04-20T10:26:42.608 INFO:teuthology.orchestra.run.smithi121.stdout:371.out 2024-04-20T10:26:42.608 INFO:teuthology.orchestra.run.smithi121.stdout:372 2024-04-20T10:26:42.608 INFO:teuthology.orchestra.run.smithi121.stdout:372.out 2024-04-20T10:26:42.608 INFO:teuthology.orchestra.run.smithi121.stdout:373 2024-04-20T10:26:42.608 INFO:teuthology.orchestra.run.smithi121.stdout:373.out 2024-04-20T10:26:42.609 INFO:teuthology.orchestra.run.smithi121.stdout:374 2024-04-20T10:26:42.609 INFO:teuthology.orchestra.run.smithi121.stdout:374.out 2024-04-20T10:26:42.609 INFO:teuthology.orchestra.run.smithi121.stdout:375 2024-04-20T10:26:42.609 INFO:teuthology.orchestra.run.smithi121.stdout:375.out 2024-04-20T10:26:42.609 INFO:teuthology.orchestra.run.smithi121.stdout:376 2024-04-20T10:26:42.609 INFO:teuthology.orchestra.run.smithi121.stdout:376.out 2024-04-20T10:26:42.609 INFO:teuthology.orchestra.run.smithi121.stdout:377 2024-04-20T10:26:42.609 INFO:teuthology.orchestra.run.smithi121.stdout:377.out 2024-04-20T10:26:42.609 INFO:teuthology.orchestra.run.smithi121.stdout:378 2024-04-20T10:26:42.609 INFO:teuthology.orchestra.run.smithi121.stdout:378.out 2024-04-20T10:26:42.609 INFO:teuthology.orchestra.run.smithi121.stdout:379 2024-04-20T10:26:42.609 INFO:teuthology.orchestra.run.smithi121.stdout:379.out 2024-04-20T10:26:42.609 INFO:teuthology.orchestra.run.smithi121.stdout:380 2024-04-20T10:26:42.609 INFO:teuthology.orchestra.run.smithi121.stdout:380.out 2024-04-20T10:26:42.609 INFO:teuthology.orchestra.run.smithi121.stdout:381 2024-04-20T10:26:42.610 INFO:teuthology.orchestra.run.smithi121.stdout:381.out 2024-04-20T10:26:42.610 INFO:teuthology.orchestra.run.smithi121.stdout:382 2024-04-20T10:26:42.610 INFO:teuthology.orchestra.run.smithi121.stdout:382.out 2024-04-20T10:26:42.610 INFO:teuthology.orchestra.run.smithi121.stdout:383 2024-04-20T10:26:42.610 INFO:teuthology.orchestra.run.smithi121.stdout:383.out 2024-04-20T10:26:42.610 INFO:teuthology.orchestra.run.smithi121.stdout:384 2024-04-20T10:26:42.610 INFO:teuthology.orchestra.run.smithi121.stdout:384.out 2024-04-20T10:26:42.610 INFO:teuthology.orchestra.run.smithi121.stdout:385 2024-04-20T10:26:42.610 INFO:teuthology.orchestra.run.smithi121.stdout:385.out 2024-04-20T10:26:42.610 INFO:teuthology.orchestra.run.smithi121.stdout:386 2024-04-20T10:26:42.610 INFO:teuthology.orchestra.run.smithi121.stdout:386.out 2024-04-20T10:26:42.610 INFO:teuthology.orchestra.run.smithi121.stdout:387 2024-04-20T10:26:42.610 INFO:teuthology.orchestra.run.smithi121.stdout:387.out 2024-04-20T10:26:42.611 INFO:teuthology.orchestra.run.smithi121.stdout:388 2024-04-20T10:26:42.611 INFO:teuthology.orchestra.run.smithi121.stdout:388.out 2024-04-20T10:26:42.611 INFO:teuthology.orchestra.run.smithi121.stdout:389 2024-04-20T10:26:42.611 INFO:teuthology.orchestra.run.smithi121.stdout:389.out 2024-04-20T10:26:42.611 INFO:teuthology.orchestra.run.smithi121.stdout:390 2024-04-20T10:26:42.611 INFO:teuthology.orchestra.run.smithi121.stdout:390.out 2024-04-20T10:26:42.611 INFO:teuthology.orchestra.run.smithi121.stdout:391 2024-04-20T10:26:42.611 INFO:teuthology.orchestra.run.smithi121.stdout:391.out 2024-04-20T10:26:42.611 INFO:teuthology.orchestra.run.smithi121.stdout:392 2024-04-20T10:26:42.611 INFO:teuthology.orchestra.run.smithi121.stdout:392.out 2024-04-20T10:26:42.611 INFO:teuthology.orchestra.run.smithi121.stdout:393 2024-04-20T10:26:42.611 INFO:teuthology.orchestra.run.smithi121.stdout:393.out 2024-04-20T10:26:42.611 INFO:teuthology.orchestra.run.smithi121.stdout:394 2024-04-20T10:26:42.611 INFO:teuthology.orchestra.run.smithi121.stdout:394.out 2024-04-20T10:26:42.611 INFO:teuthology.orchestra.run.smithi121.stdout:395 2024-04-20T10:26:42.611 INFO:teuthology.orchestra.run.smithi121.stdout:395.out 2024-04-20T10:26:42.611 INFO:teuthology.orchestra.run.smithi121.stdout:396 2024-04-20T10:26:42.612 INFO:teuthology.orchestra.run.smithi121.stdout:396.out 2024-04-20T10:26:42.612 INFO:teuthology.orchestra.run.smithi121.stdout:397 2024-04-20T10:26:42.612 INFO:teuthology.orchestra.run.smithi121.stdout:397.out 2024-04-20T10:26:42.612 INFO:teuthology.orchestra.run.smithi121.stdout:398 2024-04-20T10:26:42.612 INFO:teuthology.orchestra.run.smithi121.stdout:398.out 2024-04-20T10:26:42.612 INFO:teuthology.orchestra.run.smithi121.stdout:399 2024-04-20T10:26:42.612 INFO:teuthology.orchestra.run.smithi121.stdout:399.out 2024-04-20T10:26:42.612 INFO:teuthology.orchestra.run.smithi121.stdout:400 2024-04-20T10:26:42.612 INFO:teuthology.orchestra.run.smithi121.stdout:400.out 2024-04-20T10:26:42.612 INFO:teuthology.orchestra.run.smithi121.stdout:401 2024-04-20T10:26:42.612 INFO:teuthology.orchestra.run.smithi121.stdout:401.out 2024-04-20T10:26:42.612 INFO:teuthology.orchestra.run.smithi121.stdout:402 2024-04-20T10:26:42.612 INFO:teuthology.orchestra.run.smithi121.stdout:402.out 2024-04-20T10:26:42.613 INFO:teuthology.orchestra.run.smithi121.stdout:403 2024-04-20T10:26:42.613 INFO:teuthology.orchestra.run.smithi121.stdout:403.out 2024-04-20T10:26:42.613 INFO:teuthology.orchestra.run.smithi121.stdout:404 2024-04-20T10:26:42.613 INFO:teuthology.orchestra.run.smithi121.stdout:404.out 2024-04-20T10:26:42.613 INFO:teuthology.orchestra.run.smithi121.stdout:405 2024-04-20T10:26:42.613 INFO:teuthology.orchestra.run.smithi121.stdout:405.out 2024-04-20T10:26:42.613 INFO:teuthology.orchestra.run.smithi121.stdout:406 2024-04-20T10:26:42.613 INFO:teuthology.orchestra.run.smithi121.stdout:406.out 2024-04-20T10:26:42.613 INFO:teuthology.orchestra.run.smithi121.stdout:407 2024-04-20T10:26:42.613 INFO:teuthology.orchestra.run.smithi121.stdout:407.out 2024-04-20T10:26:42.613 INFO:teuthology.orchestra.run.smithi121.stdout:408 2024-04-20T10:26:42.613 INFO:teuthology.orchestra.run.smithi121.stdout:408.out 2024-04-20T10:26:42.613 INFO:teuthology.orchestra.run.smithi121.stdout:409 2024-04-20T10:26:42.613 INFO:teuthology.orchestra.run.smithi121.stdout:409.out 2024-04-20T10:26:42.613 INFO:teuthology.orchestra.run.smithi121.stdout:410 2024-04-20T10:26:42.614 INFO:teuthology.orchestra.run.smithi121.stdout:410.out 2024-04-20T10:26:42.614 INFO:teuthology.orchestra.run.smithi121.stdout:411 2024-04-20T10:26:42.614 INFO:teuthology.orchestra.run.smithi121.stdout:411.out 2024-04-20T10:26:42.614 INFO:teuthology.orchestra.run.smithi121.stdout:412 2024-04-20T10:26:42.614 INFO:teuthology.orchestra.run.smithi121.stdout:412.out 2024-04-20T10:26:42.614 INFO:teuthology.orchestra.run.smithi121.stdout:413 2024-04-20T10:26:42.614 INFO:teuthology.orchestra.run.smithi121.stdout:413.out 2024-04-20T10:26:42.614 INFO:teuthology.orchestra.run.smithi121.stdout:414 2024-04-20T10:26:42.614 INFO:teuthology.orchestra.run.smithi121.stdout:414.out 2024-04-20T10:26:42.614 INFO:teuthology.orchestra.run.smithi121.stdout:415 2024-04-20T10:26:42.614 INFO:teuthology.orchestra.run.smithi121.stdout:415.out 2024-04-20T10:26:42.614 INFO:teuthology.orchestra.run.smithi121.stdout:416 2024-04-20T10:26:42.615 INFO:teuthology.orchestra.run.smithi121.stdout:416.out 2024-04-20T10:26:42.615 INFO:teuthology.orchestra.run.smithi121.stdout:417 2024-04-20T10:26:42.615 INFO:teuthology.orchestra.run.smithi121.stdout:417.out 2024-04-20T10:26:42.615 INFO:teuthology.orchestra.run.smithi121.stdout:418 2024-04-20T10:26:42.615 INFO:teuthology.orchestra.run.smithi121.stdout:418.out 2024-04-20T10:26:42.615 INFO:teuthology.orchestra.run.smithi121.stdout:419 2024-04-20T10:26:42.615 INFO:teuthology.orchestra.run.smithi121.stdout:419.out 2024-04-20T10:26:42.615 INFO:teuthology.orchestra.run.smithi121.stdout:420 2024-04-20T10:26:42.615 INFO:teuthology.orchestra.run.smithi121.stdout:420.out 2024-04-20T10:26:42.615 INFO:teuthology.orchestra.run.smithi121.stdout:421 2024-04-20T10:26:42.615 INFO:teuthology.orchestra.run.smithi121.stdout:421.out 2024-04-20T10:26:42.615 INFO:teuthology.orchestra.run.smithi121.stdout:422 2024-04-20T10:26:42.615 INFO:teuthology.orchestra.run.smithi121.stdout:422.out 2024-04-20T10:26:42.615 INFO:teuthology.orchestra.run.smithi121.stdout:423 2024-04-20T10:26:42.616 INFO:teuthology.orchestra.run.smithi121.stdout:423.out 2024-04-20T10:26:42.616 INFO:teuthology.orchestra.run.smithi121.stdout:424 2024-04-20T10:26:42.616 INFO:teuthology.orchestra.run.smithi121.stdout:424.out 2024-04-20T10:26:42.616 INFO:teuthology.orchestra.run.smithi121.stdout:425 2024-04-20T10:26:42.616 INFO:teuthology.orchestra.run.smithi121.stdout:425.out 2024-04-20T10:26:42.616 INFO:teuthology.orchestra.run.smithi121.stdout:426 2024-04-20T10:26:42.616 INFO:teuthology.orchestra.run.smithi121.stdout:426.out 2024-04-20T10:26:42.616 INFO:teuthology.orchestra.run.smithi121.stdout:427 2024-04-20T10:26:42.616 INFO:teuthology.orchestra.run.smithi121.stdout:427.out 2024-04-20T10:26:42.616 INFO:teuthology.orchestra.run.smithi121.stdout:428 2024-04-20T10:26:42.616 INFO:teuthology.orchestra.run.smithi121.stdout:428.out 2024-04-20T10:26:42.616 INFO:teuthology.orchestra.run.smithi121.stdout:429 2024-04-20T10:26:42.616 INFO:teuthology.orchestra.run.smithi121.stdout:429.out 2024-04-20T10:26:42.616 INFO:teuthology.orchestra.run.smithi121.stdout:430 2024-04-20T10:26:42.616 INFO:teuthology.orchestra.run.smithi121.stdout:430.out 2024-04-20T10:26:42.616 INFO:teuthology.orchestra.run.smithi121.stdout:431 2024-04-20T10:26:42.617 INFO:teuthology.orchestra.run.smithi121.stdout:431.out 2024-04-20T10:26:42.617 INFO:teuthology.orchestra.run.smithi121.stdout:432 2024-04-20T10:26:42.617 INFO:teuthology.orchestra.run.smithi121.stdout:432.out 2024-04-20T10:26:42.617 INFO:teuthology.orchestra.run.smithi121.stdout:433 2024-04-20T10:26:42.617 INFO:teuthology.orchestra.run.smithi121.stdout:433.out 2024-04-20T10:26:42.617 INFO:teuthology.orchestra.run.smithi121.stdout:434 2024-04-20T10:26:42.617 INFO:teuthology.orchestra.run.smithi121.stdout:434.out 2024-04-20T10:26:42.617 INFO:teuthology.orchestra.run.smithi121.stdout:435 2024-04-20T10:26:42.617 INFO:teuthology.orchestra.run.smithi121.stdout:435.out 2024-04-20T10:26:42.617 INFO:teuthology.orchestra.run.smithi121.stdout:436 2024-04-20T10:26:42.617 INFO:teuthology.orchestra.run.smithi121.stdout:436.out 2024-04-20T10:26:42.617 INFO:teuthology.orchestra.run.smithi121.stdout:437 2024-04-20T10:26:42.617 INFO:teuthology.orchestra.run.smithi121.stdout:437.out 2024-04-20T10:26:42.617 INFO:teuthology.orchestra.run.smithi121.stdout:438 2024-04-20T10:26:42.617 INFO:teuthology.orchestra.run.smithi121.stdout:438.out 2024-04-20T10:26:42.617 INFO:teuthology.orchestra.run.smithi121.stdout:439 2024-04-20T10:26:42.618 INFO:teuthology.orchestra.run.smithi121.stdout:439.out 2024-04-20T10:26:42.618 INFO:teuthology.orchestra.run.smithi121.stdout:440 2024-04-20T10:26:42.618 INFO:teuthology.orchestra.run.smithi121.stdout:440.out 2024-04-20T10:26:42.618 INFO:teuthology.orchestra.run.smithi121.stdout:441 2024-04-20T10:26:42.618 INFO:teuthology.orchestra.run.smithi121.stdout:441.out 2024-04-20T10:26:42.618 INFO:teuthology.orchestra.run.smithi121.stdout:442 2024-04-20T10:26:42.618 INFO:teuthology.orchestra.run.smithi121.stdout:442.out 2024-04-20T10:26:42.618 INFO:teuthology.orchestra.run.smithi121.stdout:443 2024-04-20T10:26:42.618 INFO:teuthology.orchestra.run.smithi121.stdout:443.out 2024-04-20T10:26:42.618 INFO:teuthology.orchestra.run.smithi121.stdout:444 2024-04-20T10:26:42.618 INFO:teuthology.orchestra.run.smithi121.stdout:444.out 2024-04-20T10:26:42.618 INFO:teuthology.orchestra.run.smithi121.stdout:445 2024-04-20T10:26:42.618 INFO:teuthology.orchestra.run.smithi121.stdout:445.out 2024-04-20T10:26:42.618 INFO:teuthology.orchestra.run.smithi121.stdout:446 2024-04-20T10:26:42.618 INFO:teuthology.orchestra.run.smithi121.stdout:446.out 2024-04-20T10:26:42.619 INFO:teuthology.orchestra.run.smithi121.stdout:447 2024-04-20T10:26:42.619 INFO:teuthology.orchestra.run.smithi121.stdout:447.out 2024-04-20T10:26:42.619 INFO:teuthology.orchestra.run.smithi121.stdout:448 2024-04-20T10:26:42.619 INFO:teuthology.orchestra.run.smithi121.stdout:448.out 2024-04-20T10:26:42.619 INFO:teuthology.orchestra.run.smithi121.stdout:449 2024-04-20T10:26:42.619 INFO:teuthology.orchestra.run.smithi121.stdout:449.out 2024-04-20T10:26:42.619 INFO:teuthology.orchestra.run.smithi121.stdout:450 2024-04-20T10:26:42.619 INFO:teuthology.orchestra.run.smithi121.stdout:450.out 2024-04-20T10:26:42.619 INFO:teuthology.orchestra.run.smithi121.stdout:451 2024-04-20T10:26:42.619 INFO:teuthology.orchestra.run.smithi121.stdout:451.out 2024-04-20T10:26:42.619 INFO:teuthology.orchestra.run.smithi121.stdout:452 2024-04-20T10:26:42.619 INFO:teuthology.orchestra.run.smithi121.stdout:452.out 2024-04-20T10:26:42.619 INFO:teuthology.orchestra.run.smithi121.stdout:453 2024-04-20T10:26:42.619 INFO:teuthology.orchestra.run.smithi121.stdout:453.out 2024-04-20T10:26:42.620 INFO:teuthology.orchestra.run.smithi121.stdout:454 2024-04-20T10:26:42.620 INFO:teuthology.orchestra.run.smithi121.stdout:454.out 2024-04-20T10:26:42.620 INFO:teuthology.orchestra.run.smithi121.stdout:455 2024-04-20T10:26:42.620 INFO:teuthology.orchestra.run.smithi121.stdout:455.out 2024-04-20T10:26:42.620 INFO:teuthology.orchestra.run.smithi121.stdout:456 2024-04-20T10:26:42.620 INFO:teuthology.orchestra.run.smithi121.stdout:456.out 2024-04-20T10:26:42.620 INFO:teuthology.orchestra.run.smithi121.stdout:457 2024-04-20T10:26:42.620 INFO:teuthology.orchestra.run.smithi121.stdout:457.out 2024-04-20T10:26:42.620 INFO:teuthology.orchestra.run.smithi121.stdout:458 2024-04-20T10:26:42.620 INFO:teuthology.orchestra.run.smithi121.stdout:458.out 2024-04-20T10:26:42.620 INFO:teuthology.orchestra.run.smithi121.stdout:459 2024-04-20T10:26:42.620 INFO:teuthology.orchestra.run.smithi121.stdout:459.out 2024-04-20T10:26:42.620 INFO:teuthology.orchestra.run.smithi121.stdout:460 2024-04-20T10:26:42.620 INFO:teuthology.orchestra.run.smithi121.stdout:460.out 2024-04-20T10:26:42.620 INFO:teuthology.orchestra.run.smithi121.stdout:461 2024-04-20T10:26:42.621 INFO:teuthology.orchestra.run.smithi121.stdout:461.out 2024-04-20T10:26:42.621 INFO:teuthology.orchestra.run.smithi121.stdout:462 2024-04-20T10:26:42.621 INFO:teuthology.orchestra.run.smithi121.stdout:462.out 2024-04-20T10:26:42.621 INFO:teuthology.orchestra.run.smithi121.stdout:463 2024-04-20T10:26:42.621 INFO:teuthology.orchestra.run.smithi121.stdout:463.out 2024-04-20T10:26:42.621 INFO:teuthology.orchestra.run.smithi121.stdout:464 2024-04-20T10:26:42.621 INFO:teuthology.orchestra.run.smithi121.stdout:464.out 2024-04-20T10:26:42.621 INFO:teuthology.orchestra.run.smithi121.stdout:465 2024-04-20T10:26:42.621 INFO:teuthology.orchestra.run.smithi121.stdout:465.out 2024-04-20T10:26:42.621 INFO:teuthology.orchestra.run.smithi121.stdout:466 2024-04-20T10:26:42.621 INFO:teuthology.orchestra.run.smithi121.stdout:466.out 2024-04-20T10:26:42.621 INFO:teuthology.orchestra.run.smithi121.stdout:467 2024-04-20T10:26:42.621 INFO:teuthology.orchestra.run.smithi121.stdout:467.out 2024-04-20T10:26:42.621 INFO:teuthology.orchestra.run.smithi121.stdout:468 2024-04-20T10:26:42.621 INFO:teuthology.orchestra.run.smithi121.stdout:468.out 2024-04-20T10:26:42.621 INFO:teuthology.orchestra.run.smithi121.stdout:469 2024-04-20T10:26:42.622 INFO:teuthology.orchestra.run.smithi121.stdout:469.out 2024-04-20T10:26:42.622 INFO:teuthology.orchestra.run.smithi121.stdout:470 2024-04-20T10:26:42.622 INFO:teuthology.orchestra.run.smithi121.stdout:470.out 2024-04-20T10:26:42.622 INFO:teuthology.orchestra.run.smithi121.stdout:471 2024-04-20T10:26:42.622 INFO:teuthology.orchestra.run.smithi121.stdout:471.out 2024-04-20T10:26:42.622 INFO:teuthology.orchestra.run.smithi121.stdout:472 2024-04-20T10:26:42.622 INFO:teuthology.orchestra.run.smithi121.stdout:472.out 2024-04-20T10:26:42.622 INFO:teuthology.orchestra.run.smithi121.stdout:473 2024-04-20T10:26:42.622 INFO:teuthology.orchestra.run.smithi121.stdout:473.out 2024-04-20T10:26:42.622 INFO:teuthology.orchestra.run.smithi121.stdout:474 2024-04-20T10:26:42.622 INFO:teuthology.orchestra.run.smithi121.stdout:474.out 2024-04-20T10:26:42.622 INFO:teuthology.orchestra.run.smithi121.stdout:475 2024-04-20T10:26:42.622 INFO:teuthology.orchestra.run.smithi121.stdout:475.out 2024-04-20T10:26:42.622 INFO:teuthology.orchestra.run.smithi121.stdout:476 2024-04-20T10:26:42.622 INFO:teuthology.orchestra.run.smithi121.stdout:476.out 2024-04-20T10:26:42.623 INFO:teuthology.orchestra.run.smithi121.stdout:477 2024-04-20T10:26:42.623 INFO:teuthology.orchestra.run.smithi121.stdout:477.out 2024-04-20T10:26:42.623 INFO:teuthology.orchestra.run.smithi121.stdout:478 2024-04-20T10:26:42.623 INFO:teuthology.orchestra.run.smithi121.stdout:478.out 2024-04-20T10:26:42.623 INFO:teuthology.orchestra.run.smithi121.stdout:479 2024-04-20T10:26:42.623 INFO:teuthology.orchestra.run.smithi121.stdout:479.out 2024-04-20T10:26:42.623 INFO:teuthology.orchestra.run.smithi121.stdout:480 2024-04-20T10:26:42.623 INFO:teuthology.orchestra.run.smithi121.stdout:480.out 2024-04-20T10:26:42.623 INFO:teuthology.orchestra.run.smithi121.stdout:481 2024-04-20T10:26:42.623 INFO:teuthology.orchestra.run.smithi121.stdout:481.out 2024-04-20T10:26:42.623 INFO:teuthology.orchestra.run.smithi121.stdout:482 2024-04-20T10:26:42.623 INFO:teuthology.orchestra.run.smithi121.stdout:482.out 2024-04-20T10:26:42.623 INFO:teuthology.orchestra.run.smithi121.stdout:483 2024-04-20T10:26:42.623 INFO:teuthology.orchestra.run.smithi121.stdout:483.out 2024-04-20T10:26:42.623 INFO:teuthology.orchestra.run.smithi121.stdout:484 2024-04-20T10:26:42.623 INFO:teuthology.orchestra.run.smithi121.stdout:484.out 2024-04-20T10:26:42.624 INFO:teuthology.orchestra.run.smithi121.stdout:485 2024-04-20T10:26:42.624 INFO:teuthology.orchestra.run.smithi121.stdout:485.out 2024-04-20T10:26:42.624 INFO:teuthology.orchestra.run.smithi121.stdout:486 2024-04-20T10:26:42.624 INFO:teuthology.orchestra.run.smithi121.stdout:486.out 2024-04-20T10:26:42.624 INFO:teuthology.orchestra.run.smithi121.stdout:487 2024-04-20T10:26:42.624 INFO:teuthology.orchestra.run.smithi121.stdout:487.out 2024-04-20T10:26:42.624 INFO:teuthology.orchestra.run.smithi121.stdout:488 2024-04-20T10:26:42.624 INFO:teuthology.orchestra.run.smithi121.stdout:488.out 2024-04-20T10:26:42.624 INFO:teuthology.orchestra.run.smithi121.stdout:489 2024-04-20T10:26:42.624 INFO:teuthology.orchestra.run.smithi121.stdout:489.out 2024-04-20T10:26:42.624 INFO:teuthology.orchestra.run.smithi121.stdout:490 2024-04-20T10:26:42.624 INFO:teuthology.orchestra.run.smithi121.stdout:490.out 2024-04-20T10:26:42.624 INFO:teuthology.orchestra.run.smithi121.stdout:491 2024-04-20T10:26:42.624 INFO:teuthology.orchestra.run.smithi121.stdout:491.out 2024-04-20T10:26:42.624 INFO:teuthology.orchestra.run.smithi121.stdout:492 2024-04-20T10:26:42.624 INFO:teuthology.orchestra.run.smithi121.stdout:492.out 2024-04-20T10:26:42.625 INFO:teuthology.orchestra.run.smithi121.stdout:493 2024-04-20T10:26:42.625 INFO:teuthology.orchestra.run.smithi121.stdout:493.out 2024-04-20T10:26:42.625 INFO:teuthology.orchestra.run.smithi121.stdout:494 2024-04-20T10:26:42.625 INFO:teuthology.orchestra.run.smithi121.stdout:494.out 2024-04-20T10:26:42.625 INFO:teuthology.orchestra.run.smithi121.stdout:495 2024-04-20T10:26:42.625 INFO:teuthology.orchestra.run.smithi121.stdout:495.out 2024-04-20T10:26:42.625 INFO:teuthology.orchestra.run.smithi121.stdout:496 2024-04-20T10:26:42.625 INFO:teuthology.orchestra.run.smithi121.stdout:496.out 2024-04-20T10:26:42.625 INFO:teuthology.orchestra.run.smithi121.stdout:497 2024-04-20T10:26:42.625 INFO:teuthology.orchestra.run.smithi121.stdout:497.out 2024-04-20T10:26:42.625 INFO:teuthology.orchestra.run.smithi121.stdout:498 2024-04-20T10:26:42.625 INFO:teuthology.orchestra.run.smithi121.stdout:498.out 2024-04-20T10:26:42.625 INFO:teuthology.orchestra.run.smithi121.stdout:499 2024-04-20T10:26:42.625 INFO:teuthology.orchestra.run.smithi121.stdout:499.out 2024-04-20T10:26:42.625 INFO:teuthology.orchestra.run.smithi121.stdout:500 2024-04-20T10:26:42.626 INFO:teuthology.orchestra.run.smithi121.stdout:500.out 2024-04-20T10:26:42.626 INFO:teuthology.orchestra.run.smithi121.stdout:501 2024-04-20T10:26:42.626 INFO:teuthology.orchestra.run.smithi121.stdout:501.out 2024-04-20T10:26:42.626 INFO:teuthology.orchestra.run.smithi121.stdout:502 2024-04-20T10:26:42.626 INFO:teuthology.orchestra.run.smithi121.stdout:502.out 2024-04-20T10:26:42.626 INFO:teuthology.orchestra.run.smithi121.stdout:503 2024-04-20T10:26:42.626 INFO:teuthology.orchestra.run.smithi121.stdout:503.out 2024-04-20T10:26:42.626 INFO:teuthology.orchestra.run.smithi121.stdout:504 2024-04-20T10:26:42.626 INFO:teuthology.orchestra.run.smithi121.stdout:504.out 2024-04-20T10:26:42.626 INFO:teuthology.orchestra.run.smithi121.stdout:505 2024-04-20T10:26:42.626 INFO:teuthology.orchestra.run.smithi121.stdout:505.out 2024-04-20T10:26:42.626 INFO:teuthology.orchestra.run.smithi121.stdout:506 2024-04-20T10:26:42.626 INFO:teuthology.orchestra.run.smithi121.stdout:506.out 2024-04-20T10:26:42.626 INFO:teuthology.orchestra.run.smithi121.stdout:507 2024-04-20T10:26:42.627 INFO:teuthology.orchestra.run.smithi121.stdout:507.out 2024-04-20T10:26:42.627 INFO:teuthology.orchestra.run.smithi121.stdout:508 2024-04-20T10:26:42.627 INFO:teuthology.orchestra.run.smithi121.stdout:508.out 2024-04-20T10:26:42.627 INFO:teuthology.orchestra.run.smithi121.stdout:509 2024-04-20T10:26:42.627 INFO:teuthology.orchestra.run.smithi121.stdout:509.out 2024-04-20T10:26:42.627 INFO:teuthology.orchestra.run.smithi121.stdout:510 2024-04-20T10:26:42.627 INFO:teuthology.orchestra.run.smithi121.stdout:510.out 2024-04-20T10:26:42.627 INFO:teuthology.orchestra.run.smithi121.stdout:511 2024-04-20T10:26:42.627 INFO:teuthology.orchestra.run.smithi121.stdout:511.out 2024-04-20T10:26:42.627 INFO:teuthology.orchestra.run.smithi121.stdout:512 2024-04-20T10:26:42.627 INFO:teuthology.orchestra.run.smithi121.stdout:512.out 2024-04-20T10:26:42.627 INFO:teuthology.orchestra.run.smithi121.stdout:513 2024-04-20T10:26:42.627 INFO:teuthology.orchestra.run.smithi121.stdout:513.out 2024-04-20T10:26:42.627 INFO:teuthology.orchestra.run.smithi121.stdout:514 2024-04-20T10:26:42.627 INFO:teuthology.orchestra.run.smithi121.stdout:514.out 2024-04-20T10:26:42.628 INFO:teuthology.orchestra.run.smithi121.stdout:515 2024-04-20T10:26:42.628 INFO:teuthology.orchestra.run.smithi121.stdout:515.out 2024-04-20T10:26:42.628 INFO:teuthology.orchestra.run.smithi121.stdout:516 2024-04-20T10:26:42.628 INFO:teuthology.orchestra.run.smithi121.stdout:516.out 2024-04-20T10:26:42.628 INFO:teuthology.orchestra.run.smithi121.stdout:517 2024-04-20T10:26:42.628 INFO:teuthology.orchestra.run.smithi121.stdout:517.out 2024-04-20T10:26:42.628 INFO:teuthology.orchestra.run.smithi121.stdout:518 2024-04-20T10:26:42.628 INFO:teuthology.orchestra.run.smithi121.stdout:518.out 2024-04-20T10:26:42.628 INFO:teuthology.orchestra.run.smithi121.stdout:519 2024-04-20T10:26:42.628 INFO:teuthology.orchestra.run.smithi121.stdout:519.out 2024-04-20T10:26:42.628 INFO:teuthology.orchestra.run.smithi121.stdout:520 2024-04-20T10:26:42.628 INFO:teuthology.orchestra.run.smithi121.stdout:520.out 2024-04-20T10:26:42.628 INFO:teuthology.orchestra.run.smithi121.stdout:521 2024-04-20T10:26:42.629 INFO:teuthology.orchestra.run.smithi121.stdout:521.out 2024-04-20T10:26:42.629 INFO:teuthology.orchestra.run.smithi121.stdout:522 2024-04-20T10:26:42.629 INFO:teuthology.orchestra.run.smithi121.stdout:522.out 2024-04-20T10:26:42.629 INFO:teuthology.orchestra.run.smithi121.stdout:523 2024-04-20T10:26:42.629 INFO:teuthology.orchestra.run.smithi121.stdout:523.out 2024-04-20T10:26:42.629 INFO:teuthology.orchestra.run.smithi121.stdout:524 2024-04-20T10:26:42.629 INFO:teuthology.orchestra.run.smithi121.stdout:524.out 2024-04-20T10:26:42.629 INFO:teuthology.orchestra.run.smithi121.stdout:525 2024-04-20T10:26:42.629 INFO:teuthology.orchestra.run.smithi121.stdout:525.out 2024-04-20T10:26:42.629 INFO:teuthology.orchestra.run.smithi121.stdout:526 2024-04-20T10:26:42.629 INFO:teuthology.orchestra.run.smithi121.stdout:526.out 2024-04-20T10:26:42.629 INFO:teuthology.orchestra.run.smithi121.stdout:527 2024-04-20T10:26:42.629 INFO:teuthology.orchestra.run.smithi121.stdout:527.out 2024-04-20T10:26:42.629 INFO:teuthology.orchestra.run.smithi121.stdout:528 2024-04-20T10:26:42.630 INFO:teuthology.orchestra.run.smithi121.stdout:528.out 2024-04-20T10:26:42.630 INFO:teuthology.orchestra.run.smithi121.stdout:529 2024-04-20T10:26:42.630 INFO:teuthology.orchestra.run.smithi121.stdout:529.out 2024-04-20T10:26:42.630 INFO:teuthology.orchestra.run.smithi121.stdout:530 2024-04-20T10:26:42.630 INFO:teuthology.orchestra.run.smithi121.stdout:530.out 2024-04-20T10:26:42.630 INFO:teuthology.orchestra.run.smithi121.stdout:531 2024-04-20T10:26:42.630 INFO:teuthology.orchestra.run.smithi121.stdout:531.out 2024-04-20T10:26:42.630 INFO:teuthology.orchestra.run.smithi121.stdout:532 2024-04-20T10:26:42.630 INFO:teuthology.orchestra.run.smithi121.stdout:532.out 2024-04-20T10:26:42.630 INFO:teuthology.orchestra.run.smithi121.stdout:533 2024-04-20T10:26:42.630 INFO:teuthology.orchestra.run.smithi121.stdout:533.out 2024-04-20T10:26:42.630 INFO:teuthology.orchestra.run.smithi121.stdout:534 2024-04-20T10:26:42.630 INFO:teuthology.orchestra.run.smithi121.stdout:534.out 2024-04-20T10:26:42.630 INFO:teuthology.orchestra.run.smithi121.stdout:535 2024-04-20T10:26:42.631 INFO:teuthology.orchestra.run.smithi121.stdout:535.out 2024-04-20T10:26:42.631 INFO:teuthology.orchestra.run.smithi121.stdout:536 2024-04-20T10:26:42.631 INFO:teuthology.orchestra.run.smithi121.stdout:536.out 2024-04-20T10:26:42.631 INFO:teuthology.orchestra.run.smithi121.stdout:537 2024-04-20T10:26:42.631 INFO:teuthology.orchestra.run.smithi121.stdout:537.out 2024-04-20T10:26:42.631 INFO:teuthology.orchestra.run.smithi121.stdout:538 2024-04-20T10:26:42.631 INFO:teuthology.orchestra.run.smithi121.stdout:538.out 2024-04-20T10:26:42.631 INFO:teuthology.orchestra.run.smithi121.stdout:539 2024-04-20T10:26:42.631 INFO:teuthology.orchestra.run.smithi121.stdout:539.out 2024-04-20T10:26:42.631 INFO:teuthology.orchestra.run.smithi121.stdout:540 2024-04-20T10:26:42.631 INFO:teuthology.orchestra.run.smithi121.stdout:540.out 2024-04-20T10:26:42.631 INFO:teuthology.orchestra.run.smithi121.stdout:541 2024-04-20T10:26:42.631 INFO:teuthology.orchestra.run.smithi121.stdout:541.out 2024-04-20T10:26:42.631 INFO:teuthology.orchestra.run.smithi121.stdout:542 2024-04-20T10:26:42.631 INFO:teuthology.orchestra.run.smithi121.stdout:542.out 2024-04-20T10:26:42.631 INFO:teuthology.orchestra.run.smithi121.stdout:543 2024-04-20T10:26:42.632 INFO:teuthology.orchestra.run.smithi121.stdout:543.out 2024-04-20T10:26:42.632 INFO:teuthology.orchestra.run.smithi121.stdout:544 2024-04-20T10:26:42.632 INFO:teuthology.orchestra.run.smithi121.stdout:544.out 2024-04-20T10:26:42.632 INFO:teuthology.orchestra.run.smithi121.stdout:545 2024-04-20T10:26:42.632 INFO:teuthology.orchestra.run.smithi121.stdout:545.out 2024-04-20T10:26:42.632 INFO:teuthology.orchestra.run.smithi121.stdout:546 2024-04-20T10:26:42.632 INFO:teuthology.orchestra.run.smithi121.stdout:546.out 2024-04-20T10:26:42.632 INFO:teuthology.orchestra.run.smithi121.stdout:547 2024-04-20T10:26:42.632 INFO:teuthology.orchestra.run.smithi121.stdout:547.out 2024-04-20T10:26:42.632 INFO:teuthology.orchestra.run.smithi121.stdout:548 2024-04-20T10:26:42.632 INFO:teuthology.orchestra.run.smithi121.stdout:548.out 2024-04-20T10:26:42.632 INFO:teuthology.orchestra.run.smithi121.stdout:549 2024-04-20T10:26:42.632 INFO:teuthology.orchestra.run.smithi121.stdout:549.out 2024-04-20T10:26:42.632 INFO:teuthology.orchestra.run.smithi121.stdout:550 2024-04-20T10:26:42.632 INFO:teuthology.orchestra.run.smithi121.stdout:550.out 2024-04-20T10:26:42.632 INFO:teuthology.orchestra.run.smithi121.stdout:551 2024-04-20T10:26:42.633 INFO:teuthology.orchestra.run.smithi121.stdout:551.out 2024-04-20T10:26:42.633 INFO:teuthology.orchestra.run.smithi121.stdout:552 2024-04-20T10:26:42.633 INFO:teuthology.orchestra.run.smithi121.stdout:552.out 2024-04-20T10:26:42.633 INFO:teuthology.orchestra.run.smithi121.stdout:553 2024-04-20T10:26:42.633 INFO:teuthology.orchestra.run.smithi121.stdout:553.out 2024-04-20T10:26:42.633 INFO:teuthology.orchestra.run.smithi121.stdout:554 2024-04-20T10:26:42.633 INFO:teuthology.orchestra.run.smithi121.stdout:554.out 2024-04-20T10:26:42.633 INFO:teuthology.orchestra.run.smithi121.stdout:555 2024-04-20T10:26:42.633 INFO:teuthology.orchestra.run.smithi121.stdout:555.out 2024-04-20T10:26:42.633 INFO:teuthology.orchestra.run.smithi121.stdout:556 2024-04-20T10:26:42.633 INFO:teuthology.orchestra.run.smithi121.stdout:556.out 2024-04-20T10:26:42.633 INFO:teuthology.orchestra.run.smithi121.stdout:557 2024-04-20T10:26:42.633 INFO:teuthology.orchestra.run.smithi121.stdout:557.out 2024-04-20T10:26:42.633 INFO:teuthology.orchestra.run.smithi121.stdout:558 2024-04-20T10:26:42.633 INFO:teuthology.orchestra.run.smithi121.stdout:558.out 2024-04-20T10:26:42.634 INFO:teuthology.orchestra.run.smithi121.stdout:559 2024-04-20T10:26:42.634 INFO:teuthology.orchestra.run.smithi121.stdout:559.out 2024-04-20T10:26:42.634 INFO:teuthology.orchestra.run.smithi121.stdout:560 2024-04-20T10:26:42.634 INFO:teuthology.orchestra.run.smithi121.stdout:560.out 2024-04-20T10:26:42.634 INFO:teuthology.orchestra.run.smithi121.stdout:561 2024-04-20T10:26:42.634 INFO:teuthology.orchestra.run.smithi121.stdout:561.out 2024-04-20T10:26:42.634 INFO:teuthology.orchestra.run.smithi121.stdout:562 2024-04-20T10:26:42.634 INFO:teuthology.orchestra.run.smithi121.stdout:562.out 2024-04-20T10:26:42.634 INFO:teuthology.orchestra.run.smithi121.stdout:563 2024-04-20T10:26:42.634 INFO:teuthology.orchestra.run.smithi121.stdout:563.out 2024-04-20T10:26:42.634 INFO:teuthology.orchestra.run.smithi121.stdout:564 2024-04-20T10:26:42.634 INFO:teuthology.orchestra.run.smithi121.stdout:564.out 2024-04-20T10:26:42.634 INFO:teuthology.orchestra.run.smithi121.stdout:565 2024-04-20T10:26:42.634 INFO:teuthology.orchestra.run.smithi121.stdout:565.out 2024-04-20T10:26:42.635 INFO:teuthology.orchestra.run.smithi121.stdout:566 2024-04-20T10:26:42.635 INFO:teuthology.orchestra.run.smithi121.stdout:566.out 2024-04-20T10:26:42.635 INFO:teuthology.orchestra.run.smithi121.stdout:567 2024-04-20T10:26:42.635 INFO:teuthology.orchestra.run.smithi121.stdout:567.out 2024-04-20T10:26:42.635 INFO:teuthology.orchestra.run.smithi121.stdout:568 2024-04-20T10:26:42.635 INFO:teuthology.orchestra.run.smithi121.stdout:568.out 2024-04-20T10:26:42.635 INFO:teuthology.orchestra.run.smithi121.stdout:569 2024-04-20T10:26:42.635 INFO:teuthology.orchestra.run.smithi121.stdout:569.out 2024-04-20T10:26:42.635 INFO:teuthology.orchestra.run.smithi121.stdout:570 2024-04-20T10:26:42.635 INFO:teuthology.orchestra.run.smithi121.stdout:570.out 2024-04-20T10:26:42.635 INFO:teuthology.orchestra.run.smithi121.stdout:571 2024-04-20T10:26:42.635 INFO:teuthology.orchestra.run.smithi121.stdout:571.out 2024-04-20T10:26:42.635 INFO:teuthology.orchestra.run.smithi121.stdout:572 2024-04-20T10:26:42.635 INFO:teuthology.orchestra.run.smithi121.stdout:572.out 2024-04-20T10:26:42.636 INFO:teuthology.orchestra.run.smithi121.stdout:573 2024-04-20T10:26:42.636 INFO:teuthology.orchestra.run.smithi121.stdout:573.out 2024-04-20T10:26:42.636 INFO:teuthology.orchestra.run.smithi121.stdout:574 2024-04-20T10:26:42.636 INFO:teuthology.orchestra.run.smithi121.stdout:574.out 2024-04-20T10:26:42.636 INFO:teuthology.orchestra.run.smithi121.stdout:575 2024-04-20T10:26:42.636 INFO:teuthology.orchestra.run.smithi121.stdout:575.out 2024-04-20T10:26:42.636 INFO:teuthology.orchestra.run.smithi121.stdout:576 2024-04-20T10:26:42.636 INFO:teuthology.orchestra.run.smithi121.stdout:576.out 2024-04-20T10:26:42.636 INFO:teuthology.orchestra.run.smithi121.stdout:577 2024-04-20T10:26:42.636 INFO:teuthology.orchestra.run.smithi121.stdout:577.out 2024-04-20T10:26:42.636 INFO:teuthology.orchestra.run.smithi121.stdout:578 2024-04-20T10:26:42.636 INFO:teuthology.orchestra.run.smithi121.stdout:578.out 2024-04-20T10:26:42.636 INFO:teuthology.orchestra.run.smithi121.stdout:579 2024-04-20T10:26:42.636 INFO:teuthology.orchestra.run.smithi121.stdout:579.out 2024-04-20T10:26:42.637 INFO:teuthology.orchestra.run.smithi121.stdout:580 2024-04-20T10:26:42.637 INFO:teuthology.orchestra.run.smithi121.stdout:580.out 2024-04-20T10:26:42.637 INFO:teuthology.orchestra.run.smithi121.stdout:581 2024-04-20T10:26:42.637 INFO:teuthology.orchestra.run.smithi121.stdout:581.out 2024-04-20T10:26:42.637 INFO:teuthology.orchestra.run.smithi121.stdout:582 2024-04-20T10:26:42.637 INFO:teuthology.orchestra.run.smithi121.stdout:582.out 2024-04-20T10:26:42.637 INFO:teuthology.orchestra.run.smithi121.stdout:583 2024-04-20T10:26:42.637 INFO:teuthology.orchestra.run.smithi121.stdout:583.out 2024-04-20T10:26:42.637 INFO:teuthology.orchestra.run.smithi121.stdout:584 2024-04-20T10:26:42.637 INFO:teuthology.orchestra.run.smithi121.stdout:584.out 2024-04-20T10:26:42.637 INFO:teuthology.orchestra.run.smithi121.stdout:585 2024-04-20T10:26:42.637 INFO:teuthology.orchestra.run.smithi121.stdout:585.out 2024-04-20T10:26:42.637 INFO:teuthology.orchestra.run.smithi121.stdout:586 2024-04-20T10:26:42.637 INFO:teuthology.orchestra.run.smithi121.stdout:586.out 2024-04-20T10:26:42.638 INFO:teuthology.orchestra.run.smithi121.stdout:587 2024-04-20T10:26:42.638 INFO:teuthology.orchestra.run.smithi121.stdout:587.out 2024-04-20T10:26:42.638 INFO:teuthology.orchestra.run.smithi121.stdout:588 2024-04-20T10:26:42.638 INFO:teuthology.orchestra.run.smithi121.stdout:588.out 2024-04-20T10:26:42.638 INFO:teuthology.orchestra.run.smithi121.stdout:589 2024-04-20T10:26:42.638 INFO:teuthology.orchestra.run.smithi121.stdout:589.out 2024-04-20T10:26:42.638 INFO:teuthology.orchestra.run.smithi121.stdout:590 2024-04-20T10:26:42.638 INFO:teuthology.orchestra.run.smithi121.stdout:590.out 2024-04-20T10:26:42.638 INFO:teuthology.orchestra.run.smithi121.stdout:591 2024-04-20T10:26:42.638 INFO:teuthology.orchestra.run.smithi121.stdout:591.out 2024-04-20T10:26:42.638 INFO:teuthology.orchestra.run.smithi121.stdout:592 2024-04-20T10:26:42.638 INFO:teuthology.orchestra.run.smithi121.stdout:592.out 2024-04-20T10:26:42.638 INFO:teuthology.orchestra.run.smithi121.stdout:593 2024-04-20T10:26:42.638 INFO:teuthology.orchestra.run.smithi121.stdout:593.out 2024-04-20T10:26:42.638 INFO:teuthology.orchestra.run.smithi121.stdout:594 2024-04-20T10:26:42.639 INFO:teuthology.orchestra.run.smithi121.stdout:594.out 2024-04-20T10:26:42.639 INFO:teuthology.orchestra.run.smithi121.stdout:595 2024-04-20T10:26:42.639 INFO:teuthology.orchestra.run.smithi121.stdout:595.out 2024-04-20T10:26:42.639 INFO:teuthology.orchestra.run.smithi121.stdout:596 2024-04-20T10:26:42.639 INFO:teuthology.orchestra.run.smithi121.stdout:596.out 2024-04-20T10:26:42.639 INFO:teuthology.orchestra.run.smithi121.stdout:597 2024-04-20T10:26:42.639 INFO:teuthology.orchestra.run.smithi121.stdout:597.out 2024-04-20T10:26:42.639 INFO:teuthology.orchestra.run.smithi121.stdout:598 2024-04-20T10:26:42.639 INFO:teuthology.orchestra.run.smithi121.stdout:598.out 2024-04-20T10:26:42.639 INFO:teuthology.orchestra.run.smithi121.stdout:599 2024-04-20T10:26:42.639 INFO:teuthology.orchestra.run.smithi121.stdout:599.out 2024-04-20T10:26:42.639 INFO:teuthology.orchestra.run.smithi121.stdout:600 2024-04-20T10:26:42.639 INFO:teuthology.orchestra.run.smithi121.stdout:600.out 2024-04-20T10:26:42.639 INFO:teuthology.orchestra.run.smithi121.stdout:601 2024-04-20T10:26:42.640 INFO:teuthology.orchestra.run.smithi121.stdout:601.out 2024-04-20T10:26:42.640 INFO:teuthology.orchestra.run.smithi121.stdout:602 2024-04-20T10:26:42.640 INFO:teuthology.orchestra.run.smithi121.stdout:602.out 2024-04-20T10:26:42.640 INFO:teuthology.orchestra.run.smithi121.stdout:603 2024-04-20T10:26:42.640 INFO:teuthology.orchestra.run.smithi121.stdout:603.out 2024-04-20T10:26:42.640 INFO:teuthology.orchestra.run.smithi121.stdout:604 2024-04-20T10:26:42.640 INFO:teuthology.orchestra.run.smithi121.stdout:604.out 2024-04-20T10:26:42.640 INFO:teuthology.orchestra.run.smithi121.stdout:605 2024-04-20T10:26:42.640 INFO:teuthology.orchestra.run.smithi121.stdout:605.out 2024-04-20T10:26:42.640 INFO:teuthology.orchestra.run.smithi121.stdout:606 2024-04-20T10:26:42.640 INFO:teuthology.orchestra.run.smithi121.stdout:606.out 2024-04-20T10:26:42.640 INFO:teuthology.orchestra.run.smithi121.stdout:607 2024-04-20T10:26:42.640 INFO:teuthology.orchestra.run.smithi121.stdout:607.out 2024-04-20T10:26:42.640 INFO:teuthology.orchestra.run.smithi121.stdout:608 2024-04-20T10:26:42.640 INFO:teuthology.orchestra.run.smithi121.stdout:608.out 2024-04-20T10:26:42.640 INFO:teuthology.orchestra.run.smithi121.stdout:609 2024-04-20T10:26:42.641 INFO:teuthology.orchestra.run.smithi121.stdout:609.out 2024-04-20T10:26:42.641 INFO:teuthology.orchestra.run.smithi121.stdout:610 2024-04-20T10:26:42.641 INFO:teuthology.orchestra.run.smithi121.stdout:610.out 2024-04-20T10:26:42.641 INFO:teuthology.orchestra.run.smithi121.stdout:611 2024-04-20T10:26:42.641 INFO:teuthology.orchestra.run.smithi121.stdout:611.out 2024-04-20T10:26:42.641 INFO:teuthology.orchestra.run.smithi121.stdout:612 2024-04-20T10:26:42.641 INFO:teuthology.orchestra.run.smithi121.stdout:612.out 2024-04-20T10:26:42.641 INFO:teuthology.orchestra.run.smithi121.stdout:613 2024-04-20T10:26:42.641 INFO:teuthology.orchestra.run.smithi121.stdout:613.out 2024-04-20T10:26:42.641 INFO:teuthology.orchestra.run.smithi121.stdout:614 2024-04-20T10:26:42.641 INFO:teuthology.orchestra.run.smithi121.stdout:614.out 2024-04-20T10:26:42.641 INFO:teuthology.orchestra.run.smithi121.stdout:615 2024-04-20T10:26:42.641 INFO:teuthology.orchestra.run.smithi121.stdout:615.out 2024-04-20T10:26:42.641 INFO:teuthology.orchestra.run.smithi121.stdout:616 2024-04-20T10:26:42.641 INFO:teuthology.orchestra.run.smithi121.stdout:616.out 2024-04-20T10:26:42.642 INFO:teuthology.orchestra.run.smithi121.stdout:617 2024-04-20T10:26:42.642 INFO:teuthology.orchestra.run.smithi121.stdout:617.out 2024-04-20T10:26:42.642 INFO:teuthology.orchestra.run.smithi121.stdout:618 2024-04-20T10:26:42.642 INFO:teuthology.orchestra.run.smithi121.stdout:618.out 2024-04-20T10:26:42.642 INFO:teuthology.orchestra.run.smithi121.stdout:619 2024-04-20T10:26:42.642 INFO:teuthology.orchestra.run.smithi121.stdout:619.out 2024-04-20T10:26:42.642 INFO:teuthology.orchestra.run.smithi121.stdout:620 2024-04-20T10:26:42.642 INFO:teuthology.orchestra.run.smithi121.stdout:620.out 2024-04-20T10:26:42.642 INFO:teuthology.orchestra.run.smithi121.stdout:621 2024-04-20T10:26:42.642 INFO:teuthology.orchestra.run.smithi121.stdout:621.out 2024-04-20T10:26:42.642 INFO:teuthology.orchestra.run.smithi121.stdout:622 2024-04-20T10:26:42.642 INFO:teuthology.orchestra.run.smithi121.stdout:622.out 2024-04-20T10:26:42.642 INFO:teuthology.orchestra.run.smithi121.stdout:623 2024-04-20T10:26:42.642 INFO:teuthology.orchestra.run.smithi121.stdout:623.out 2024-04-20T10:26:42.642 INFO:teuthology.orchestra.run.smithi121.stdout:624 2024-04-20T10:26:42.643 INFO:teuthology.orchestra.run.smithi121.stdout:624.out 2024-04-20T10:26:42.643 INFO:teuthology.orchestra.run.smithi121.stdout:625 2024-04-20T10:26:42.643 INFO:teuthology.orchestra.run.smithi121.stdout:625.out 2024-04-20T10:26:42.643 INFO:teuthology.orchestra.run.smithi121.stdout:626 2024-04-20T10:26:42.643 INFO:teuthology.orchestra.run.smithi121.stdout:626.out 2024-04-20T10:26:42.643 INFO:teuthology.orchestra.run.smithi121.stdout:627 2024-04-20T10:26:42.643 INFO:teuthology.orchestra.run.smithi121.stdout:627.out 2024-04-20T10:26:42.643 INFO:teuthology.orchestra.run.smithi121.stdout:628 2024-04-20T10:26:42.643 INFO:teuthology.orchestra.run.smithi121.stdout:628.out 2024-04-20T10:26:42.643 INFO:teuthology.orchestra.run.smithi121.stdout:629 2024-04-20T10:26:42.643 INFO:teuthology.orchestra.run.smithi121.stdout:629.out 2024-04-20T10:26:42.643 INFO:teuthology.orchestra.run.smithi121.stdout:630 2024-04-20T10:26:42.643 INFO:teuthology.orchestra.run.smithi121.stdout:630.out 2024-04-20T10:26:42.643 INFO:teuthology.orchestra.run.smithi121.stdout:631 2024-04-20T10:26:42.643 INFO:teuthology.orchestra.run.smithi121.stdout:631.out 2024-04-20T10:26:42.644 INFO:teuthology.orchestra.run.smithi121.stdout:632 2024-04-20T10:26:42.644 INFO:teuthology.orchestra.run.smithi121.stdout:632.out 2024-04-20T10:26:42.644 INFO:teuthology.orchestra.run.smithi121.stdout:633 2024-04-20T10:26:42.644 INFO:teuthology.orchestra.run.smithi121.stdout:633.out 2024-04-20T10:26:42.644 INFO:teuthology.orchestra.run.smithi121.stdout:634 2024-04-20T10:26:42.644 INFO:teuthology.orchestra.run.smithi121.stdout:634.out 2024-04-20T10:26:42.644 INFO:teuthology.orchestra.run.smithi121.stdout:635 2024-04-20T10:26:42.644 INFO:teuthology.orchestra.run.smithi121.stdout:635.out 2024-04-20T10:26:42.644 INFO:teuthology.orchestra.run.smithi121.stdout:636 2024-04-20T10:26:42.644 INFO:teuthology.orchestra.run.smithi121.stdout:636.out 2024-04-20T10:26:42.644 INFO:teuthology.orchestra.run.smithi121.stdout:637 2024-04-20T10:26:42.644 INFO:teuthology.orchestra.run.smithi121.stdout:637.out 2024-04-20T10:26:42.644 INFO:teuthology.orchestra.run.smithi121.stdout:638 2024-04-20T10:26:42.644 INFO:teuthology.orchestra.run.smithi121.stdout:638.out 2024-04-20T10:26:42.644 INFO:teuthology.orchestra.run.smithi121.stdout:639 2024-04-20T10:26:42.644 INFO:teuthology.orchestra.run.smithi121.stdout:639.out 2024-04-20T10:26:42.645 INFO:teuthology.orchestra.run.smithi121.stdout:640 2024-04-20T10:26:42.645 INFO:teuthology.orchestra.run.smithi121.stdout:640.out 2024-04-20T10:26:42.645 INFO:teuthology.orchestra.run.smithi121.stdout:641 2024-04-20T10:26:42.645 INFO:teuthology.orchestra.run.smithi121.stdout:641.out 2024-04-20T10:26:42.645 INFO:teuthology.orchestra.run.smithi121.stdout:642 2024-04-20T10:26:42.645 INFO:teuthology.orchestra.run.smithi121.stdout:642.out 2024-04-20T10:26:42.645 INFO:teuthology.orchestra.run.smithi121.stdout:643 2024-04-20T10:26:42.645 INFO:teuthology.orchestra.run.smithi121.stdout:643.out 2024-04-20T10:26:42.645 INFO:teuthology.orchestra.run.smithi121.stdout:644 2024-04-20T10:26:42.645 INFO:teuthology.orchestra.run.smithi121.stdout:644.out 2024-04-20T10:26:42.645 INFO:teuthology.orchestra.run.smithi121.stdout:645 2024-04-20T10:26:42.645 INFO:teuthology.orchestra.run.smithi121.stdout:645.out 2024-04-20T10:26:42.645 INFO:teuthology.orchestra.run.smithi121.stdout:646 2024-04-20T10:26:42.645 INFO:teuthology.orchestra.run.smithi121.stdout:646.out 2024-04-20T10:26:42.645 INFO:teuthology.orchestra.run.smithi121.stdout:647 2024-04-20T10:26:42.645 INFO:teuthology.orchestra.run.smithi121.stdout:647.out 2024-04-20T10:26:42.645 INFO:teuthology.orchestra.run.smithi121.stdout:648 2024-04-20T10:26:42.646 INFO:teuthology.orchestra.run.smithi121.stdout:648.out 2024-04-20T10:26:42.646 INFO:teuthology.orchestra.run.smithi121.stdout:649 2024-04-20T10:26:42.646 INFO:teuthology.orchestra.run.smithi121.stdout:649.out 2024-04-20T10:26:42.646 INFO:teuthology.orchestra.run.smithi121.stdout:650 2024-04-20T10:26:42.646 INFO:teuthology.orchestra.run.smithi121.stdout:650.out 2024-04-20T10:26:42.646 INFO:teuthology.orchestra.run.smithi121.stdout:651 2024-04-20T10:26:42.646 INFO:teuthology.orchestra.run.smithi121.stdout:651.out 2024-04-20T10:26:42.646 INFO:teuthology.orchestra.run.smithi121.stdout:652 2024-04-20T10:26:42.646 INFO:teuthology.orchestra.run.smithi121.stdout:652.out 2024-04-20T10:26:42.646 INFO:teuthology.orchestra.run.smithi121.stdout:653 2024-04-20T10:26:42.646 INFO:teuthology.orchestra.run.smithi121.stdout:653.out 2024-04-20T10:26:42.646 INFO:teuthology.orchestra.run.smithi121.stdout:654 2024-04-20T10:26:42.646 INFO:teuthology.orchestra.run.smithi121.stdout:654.out 2024-04-20T10:26:42.646 INFO:teuthology.orchestra.run.smithi121.stdout:655 2024-04-20T10:26:42.647 INFO:teuthology.orchestra.run.smithi121.stdout:655.out 2024-04-20T10:26:42.647 INFO:teuthology.orchestra.run.smithi121.stdout:656 2024-04-20T10:26:42.647 INFO:teuthology.orchestra.run.smithi121.stdout:656.out 2024-04-20T10:26:42.647 INFO:teuthology.orchestra.run.smithi121.stdout:657 2024-04-20T10:26:42.647 INFO:teuthology.orchestra.run.smithi121.stdout:657.out 2024-04-20T10:26:42.647 INFO:teuthology.orchestra.run.smithi121.stdout:658 2024-04-20T10:26:42.647 INFO:teuthology.orchestra.run.smithi121.stdout:658.out 2024-04-20T10:26:42.647 INFO:teuthology.orchestra.run.smithi121.stdout:659 2024-04-20T10:26:42.647 INFO:teuthology.orchestra.run.smithi121.stdout:659.out 2024-04-20T10:26:42.647 INFO:teuthology.orchestra.run.smithi121.stdout:660 2024-04-20T10:26:42.647 INFO:teuthology.orchestra.run.smithi121.stdout:660.out 2024-04-20T10:26:42.647 INFO:teuthology.orchestra.run.smithi121.stdout:661 2024-04-20T10:26:42.647 INFO:teuthology.orchestra.run.smithi121.stdout:661.out 2024-04-20T10:26:42.647 INFO:teuthology.orchestra.run.smithi121.stdout:662 2024-04-20T10:26:42.647 INFO:teuthology.orchestra.run.smithi121.stdout:662.out 2024-04-20T10:26:42.647 INFO:teuthology.orchestra.run.smithi121.stdout:663 2024-04-20T10:26:42.648 INFO:teuthology.orchestra.run.smithi121.stdout:663.out 2024-04-20T10:26:42.648 INFO:teuthology.orchestra.run.smithi121.stdout:664 2024-04-20T10:26:42.648 INFO:teuthology.orchestra.run.smithi121.stdout:664.out 2024-04-20T10:26:42.648 INFO:teuthology.orchestra.run.smithi121.stdout:665 2024-04-20T10:26:42.648 INFO:teuthology.orchestra.run.smithi121.stdout:665.out 2024-04-20T10:26:42.648 INFO:teuthology.orchestra.run.smithi121.stdout:666 2024-04-20T10:26:42.648 INFO:teuthology.orchestra.run.smithi121.stdout:666.out 2024-04-20T10:26:42.648 INFO:teuthology.orchestra.run.smithi121.stdout:667 2024-04-20T10:26:42.648 INFO:teuthology.orchestra.run.smithi121.stdout:667.out 2024-04-20T10:26:42.648 INFO:teuthology.orchestra.run.smithi121.stdout:668 2024-04-20T10:26:42.648 INFO:teuthology.orchestra.run.smithi121.stdout:668.out 2024-04-20T10:26:42.648 INFO:teuthology.orchestra.run.smithi121.stdout:669 2024-04-20T10:26:42.648 INFO:teuthology.orchestra.run.smithi121.stdout:669.out 2024-04-20T10:26:42.648 INFO:teuthology.orchestra.run.smithi121.stdout:670 2024-04-20T10:26:42.648 INFO:teuthology.orchestra.run.smithi121.stdout:670.out 2024-04-20T10:26:42.648 INFO:teuthology.orchestra.run.smithi121.stdout:671 2024-04-20T10:26:42.648 INFO:teuthology.orchestra.run.smithi121.stdout:671.out 2024-04-20T10:26:42.649 INFO:teuthology.orchestra.run.smithi121.stdout:672 2024-04-20T10:26:42.649 INFO:teuthology.orchestra.run.smithi121.stdout:672.out 2024-04-20T10:26:42.649 INFO:teuthology.orchestra.run.smithi121.stdout:673 2024-04-20T10:26:42.649 INFO:teuthology.orchestra.run.smithi121.stdout:673.out 2024-04-20T10:26:42.649 INFO:teuthology.orchestra.run.smithi121.stdout:674 2024-04-20T10:26:42.649 INFO:teuthology.orchestra.run.smithi121.stdout:674.out 2024-04-20T10:26:42.649 INFO:teuthology.orchestra.run.smithi121.stdout:675 2024-04-20T10:26:42.649 INFO:teuthology.orchestra.run.smithi121.stdout:675.out 2024-04-20T10:26:42.649 INFO:teuthology.orchestra.run.smithi121.stdout:676 2024-04-20T10:26:42.649 INFO:teuthology.orchestra.run.smithi121.stdout:676.out 2024-04-20T10:26:42.649 INFO:teuthology.orchestra.run.smithi121.stdout:677 2024-04-20T10:26:42.649 INFO:teuthology.orchestra.run.smithi121.stdout:677.out 2024-04-20T10:26:42.649 INFO:teuthology.orchestra.run.smithi121.stdout:678 2024-04-20T10:26:42.649 INFO:teuthology.orchestra.run.smithi121.stdout:678.out 2024-04-20T10:26:42.649 INFO:teuthology.orchestra.run.smithi121.stdout:679 2024-04-20T10:26:42.650 INFO:teuthology.orchestra.run.smithi121.stdout:679.out 2024-04-20T10:26:42.650 INFO:teuthology.orchestra.run.smithi121.stdout:680 2024-04-20T10:26:42.650 INFO:teuthology.orchestra.run.smithi121.stdout:680.out 2024-04-20T10:26:42.650 INFO:teuthology.orchestra.run.smithi121.stdout:681 2024-04-20T10:26:42.650 INFO:teuthology.orchestra.run.smithi121.stdout:681.out 2024-04-20T10:26:42.650 INFO:teuthology.orchestra.run.smithi121.stdout:682 2024-04-20T10:26:42.650 INFO:teuthology.orchestra.run.smithi121.stdout:682.out 2024-04-20T10:26:42.650 INFO:teuthology.orchestra.run.smithi121.stdout:683 2024-04-20T10:26:42.650 INFO:teuthology.orchestra.run.smithi121.stdout:683.out 2024-04-20T10:26:42.650 INFO:teuthology.orchestra.run.smithi121.stdout:684 2024-04-20T10:26:42.650 INFO:teuthology.orchestra.run.smithi121.stdout:684.out 2024-04-20T10:26:42.650 INFO:teuthology.orchestra.run.smithi121.stdout:685 2024-04-20T10:26:42.650 INFO:teuthology.orchestra.run.smithi121.stdout:685.out 2024-04-20T10:26:42.650 INFO:teuthology.orchestra.run.smithi121.stdout:686 2024-04-20T10:26:42.651 INFO:teuthology.orchestra.run.smithi121.stdout:686.out 2024-04-20T10:26:42.651 INFO:teuthology.orchestra.run.smithi121.stdout:687 2024-04-20T10:26:42.651 INFO:teuthology.orchestra.run.smithi121.stdout:687.out 2024-04-20T10:26:42.651 INFO:teuthology.orchestra.run.smithi121.stdout:688 2024-04-20T10:26:42.651 INFO:teuthology.orchestra.run.smithi121.stdout:688.out 2024-04-20T10:26:42.651 INFO:teuthology.orchestra.run.smithi121.stdout:689 2024-04-20T10:26:42.651 INFO:teuthology.orchestra.run.smithi121.stdout:689.out 2024-04-20T10:26:42.651 INFO:teuthology.orchestra.run.smithi121.stdout:690 2024-04-20T10:26:42.651 INFO:teuthology.orchestra.run.smithi121.stdout:690.out 2024-04-20T10:26:42.651 INFO:teuthology.orchestra.run.smithi121.stdout:691 2024-04-20T10:26:42.651 INFO:teuthology.orchestra.run.smithi121.stdout:691.out 2024-04-20T10:26:42.651 INFO:teuthology.orchestra.run.smithi121.stdout:692 2024-04-20T10:26:42.651 INFO:teuthology.orchestra.run.smithi121.stdout:692.out 2024-04-20T10:26:42.651 INFO:teuthology.orchestra.run.smithi121.stdout:693 2024-04-20T10:26:42.651 INFO:teuthology.orchestra.run.smithi121.stdout:693.out 2024-04-20T10:26:42.651 INFO:teuthology.orchestra.run.smithi121.stdout:694 2024-04-20T10:26:42.652 INFO:teuthology.orchestra.run.smithi121.stdout:694.out 2024-04-20T10:26:42.652 INFO:teuthology.orchestra.run.smithi121.stdout:695 2024-04-20T10:26:42.652 INFO:teuthology.orchestra.run.smithi121.stdout:695.out 2024-04-20T10:26:42.652 INFO:teuthology.orchestra.run.smithi121.stdout:696 2024-04-20T10:26:42.652 INFO:teuthology.orchestra.run.smithi121.stdout:696.out 2024-04-20T10:26:42.652 INFO:teuthology.orchestra.run.smithi121.stdout:697 2024-04-20T10:26:42.652 INFO:teuthology.orchestra.run.smithi121.stdout:697.out 2024-04-20T10:26:42.652 INFO:teuthology.orchestra.run.smithi121.stdout:698 2024-04-20T10:26:42.652 INFO:teuthology.orchestra.run.smithi121.stdout:698.out 2024-04-20T10:26:42.652 INFO:teuthology.orchestra.run.smithi121.stdout:699 2024-04-20T10:26:42.652 INFO:teuthology.orchestra.run.smithi121.stdout:699.out 2024-04-20T10:26:42.652 INFO:teuthology.orchestra.run.smithi121.stdout:700 2024-04-20T10:26:42.652 INFO:teuthology.orchestra.run.smithi121.stdout:700.out 2024-04-20T10:26:42.652 INFO:teuthology.orchestra.run.smithi121.stdout:701 2024-04-20T10:26:42.652 INFO:teuthology.orchestra.run.smithi121.stdout:701.out 2024-04-20T10:26:42.652 INFO:teuthology.orchestra.run.smithi121.stdout:702 2024-04-20T10:26:42.653 INFO:teuthology.orchestra.run.smithi121.stdout:702.out 2024-04-20T10:26:42.653 INFO:teuthology.orchestra.run.smithi121.stdout:703 2024-04-20T10:26:42.653 INFO:teuthology.orchestra.run.smithi121.stdout:703.out 2024-04-20T10:26:42.653 INFO:teuthology.orchestra.run.smithi121.stdout:704 2024-04-20T10:26:42.653 INFO:teuthology.orchestra.run.smithi121.stdout:704.out 2024-04-20T10:26:42.653 INFO:teuthology.orchestra.run.smithi121.stdout:705 2024-04-20T10:26:42.653 INFO:teuthology.orchestra.run.smithi121.stdout:705.out 2024-04-20T10:26:42.653 INFO:teuthology.orchestra.run.smithi121.stdout:706 2024-04-20T10:26:42.653 INFO:teuthology.orchestra.run.smithi121.stdout:706.out 2024-04-20T10:26:42.653 INFO:teuthology.orchestra.run.smithi121.stdout:707 2024-04-20T10:26:42.653 INFO:teuthology.orchestra.run.smithi121.stdout:707.out 2024-04-20T10:26:42.653 INFO:teuthology.orchestra.run.smithi121.stdout:708 2024-04-20T10:26:42.653 INFO:teuthology.orchestra.run.smithi121.stdout:708.out 2024-04-20T10:26:42.653 INFO:teuthology.orchestra.run.smithi121.stdout:709 2024-04-20T10:26:42.653 INFO:teuthology.orchestra.run.smithi121.stdout:709.out 2024-04-20T10:26:42.653 INFO:teuthology.orchestra.run.smithi121.stdout:710 2024-04-20T10:26:42.654 INFO:teuthology.orchestra.run.smithi121.stdout:710.out 2024-04-20T10:26:42.654 INFO:teuthology.orchestra.run.smithi121.stdout:711 2024-04-20T10:26:42.654 INFO:teuthology.orchestra.run.smithi121.stdout:711.out 2024-04-20T10:26:42.654 INFO:teuthology.orchestra.run.smithi121.stdout:712 2024-04-20T10:26:42.654 INFO:teuthology.orchestra.run.smithi121.stdout:712.out 2024-04-20T10:26:42.654 INFO:teuthology.orchestra.run.smithi121.stdout:713 2024-04-20T10:26:42.654 INFO:teuthology.orchestra.run.smithi121.stdout:713.out 2024-04-20T10:26:42.654 INFO:teuthology.orchestra.run.smithi121.stdout:714 2024-04-20T10:26:42.654 INFO:teuthology.orchestra.run.smithi121.stdout:714.out 2024-04-20T10:26:42.654 INFO:teuthology.orchestra.run.smithi121.stdout:715 2024-04-20T10:26:42.654 INFO:teuthology.orchestra.run.smithi121.stdout:715.out 2024-04-20T10:26:42.654 INFO:teuthology.orchestra.run.smithi121.stdout:716 2024-04-20T10:26:42.654 INFO:teuthology.orchestra.run.smithi121.stdout:716.out 2024-04-20T10:26:42.655 INFO:teuthology.orchestra.run.smithi121.stdout:717 2024-04-20T10:26:42.655 INFO:teuthology.orchestra.run.smithi121.stdout:717.out 2024-04-20T10:26:42.655 INFO:teuthology.orchestra.run.smithi121.stdout:718 2024-04-20T10:26:42.655 INFO:teuthology.orchestra.run.smithi121.stdout:718.out 2024-04-20T10:26:42.655 INFO:teuthology.orchestra.run.smithi121.stdout:719 2024-04-20T10:26:42.655 INFO:teuthology.orchestra.run.smithi121.stdout:719.out 2024-04-20T10:26:42.655 INFO:teuthology.orchestra.run.smithi121.stdout:720 2024-04-20T10:26:42.655 INFO:teuthology.orchestra.run.smithi121.stdout:720.out 2024-04-20T10:26:42.655 INFO:teuthology.orchestra.run.smithi121.stdout:721 2024-04-20T10:26:42.655 INFO:teuthology.orchestra.run.smithi121.stdout:721.out 2024-04-20T10:26:42.655 INFO:teuthology.orchestra.run.smithi121.stdout:722 2024-04-20T10:26:42.655 INFO:teuthology.orchestra.run.smithi121.stdout:722.out 2024-04-20T10:26:42.655 INFO:teuthology.orchestra.run.smithi121.stdout:723 2024-04-20T10:26:42.655 INFO:teuthology.orchestra.run.smithi121.stdout:723.out 2024-04-20T10:26:42.655 INFO:teuthology.orchestra.run.smithi121.stdout:724 2024-04-20T10:26:42.656 INFO:teuthology.orchestra.run.smithi121.stdout:724.out 2024-04-20T10:26:42.656 INFO:teuthology.orchestra.run.smithi121.stdout:725 2024-04-20T10:26:42.656 INFO:teuthology.orchestra.run.smithi121.stdout:725.out 2024-04-20T10:26:42.656 INFO:teuthology.orchestra.run.smithi121.stdout:726 2024-04-20T10:26:42.656 INFO:teuthology.orchestra.run.smithi121.stdout:726.out 2024-04-20T10:26:42.656 INFO:teuthology.orchestra.run.smithi121.stdout:727 2024-04-20T10:26:42.656 INFO:teuthology.orchestra.run.smithi121.stdout:727.out 2024-04-20T10:26:42.656 INFO:teuthology.orchestra.run.smithi121.stdout:728 2024-04-20T10:26:42.656 INFO:teuthology.orchestra.run.smithi121.stdout:728.out 2024-04-20T10:26:42.656 INFO:teuthology.orchestra.run.smithi121.stdout:729 2024-04-20T10:26:42.656 INFO:teuthology.orchestra.run.smithi121.stdout:729.out 2024-04-20T10:26:42.656 INFO:teuthology.orchestra.run.smithi121.stdout:730 2024-04-20T10:26:42.656 INFO:teuthology.orchestra.run.smithi121.stdout:730.out 2024-04-20T10:26:42.656 INFO:teuthology.orchestra.run.smithi121.stdout:731 2024-04-20T10:26:42.657 INFO:teuthology.orchestra.run.smithi121.stdout:731.out 2024-04-20T10:26:42.657 INFO:teuthology.orchestra.run.smithi121.stdout:732 2024-04-20T10:26:42.657 INFO:teuthology.orchestra.run.smithi121.stdout:732.out 2024-04-20T10:26:42.657 INFO:teuthology.orchestra.run.smithi121.stdout:733 2024-04-20T10:26:42.657 INFO:teuthology.orchestra.run.smithi121.stdout:733.out 2024-04-20T10:26:42.657 INFO:teuthology.orchestra.run.smithi121.stdout:734 2024-04-20T10:26:42.657 INFO:teuthology.orchestra.run.smithi121.stdout:734.out 2024-04-20T10:26:42.657 INFO:teuthology.orchestra.run.smithi121.stdout:735 2024-04-20T10:26:42.657 INFO:teuthology.orchestra.run.smithi121.stdout:735.out 2024-04-20T10:26:42.657 INFO:teuthology.orchestra.run.smithi121.stdout:736 2024-04-20T10:26:42.657 INFO:teuthology.orchestra.run.smithi121.stdout:736.out 2024-04-20T10:26:42.657 INFO:teuthology.orchestra.run.smithi121.stdout:737 2024-04-20T10:26:42.657 INFO:teuthology.orchestra.run.smithi121.stdout:737.out 2024-04-20T10:26:42.657 INFO:teuthology.orchestra.run.smithi121.stdout:738 2024-04-20T10:26:42.657 INFO:teuthology.orchestra.run.smithi121.stdout:738.out 2024-04-20T10:26:42.658 INFO:teuthology.orchestra.run.smithi121.stdout:739 2024-04-20T10:26:42.658 INFO:teuthology.orchestra.run.smithi121.stdout:739.out 2024-04-20T10:26:42.658 INFO:teuthology.orchestra.run.smithi121.stdout:741 2024-04-20T10:26:42.658 INFO:teuthology.orchestra.run.smithi121.stdout:741.out 2024-04-20T10:26:42.658 INFO:teuthology.orchestra.run.smithi121.stdout:742 2024-04-20T10:26:42.658 INFO:teuthology.orchestra.run.smithi121.stdout:742.out 2024-04-20T10:26:42.658 INFO:teuthology.orchestra.run.smithi121.stdout:743 2024-04-20T10:26:42.658 INFO:teuthology.orchestra.run.smithi121.stdout:743.out 2024-04-20T10:26:42.658 INFO:teuthology.orchestra.run.smithi121.stdout:group.list 2024-04-20T10:26:42.658 INFO:teuthology.orchestra.run.smithi121.stdout:Makefile 2024-04-20T10:26:42.660 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/001) 2024-04-20T10:26:43.222 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:26:43.234 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:26:43.234 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:26:51.473 INFO:teuthology.orchestra.run.smithi121.stdout:generic/001 8s 2024-04-20T10:26:51.491 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/001 2024-04-20T10:26:51.491 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:26:51.501 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:26:51.506 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi121.stderr:ceph-fuse[39624]: fuse finished with error 0 and tester_r 0 2024-04-20T10:26:51.532 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:26:51.532 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/002) 2024-04-20T10:26:52.063 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:26:52.076 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:26:52.077 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:26:53.121 INFO:teuthology.orchestra.run.smithi121.stdout:generic/002 1s 2024-04-20T10:26:53.137 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/002 2024-04-20T10:26:53.137 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:26:53.146 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:26:53.201 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:26:53.202 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/003) 2024-04-20T10:26:53.681 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:26:53.695 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:26:53.695 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:26:54.101 INFO:teuthology.orchestra.run.smithi121.stdout:generic/003 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:26:54.112 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/003 2024-04-20T10:26:54.112 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/003 2024-04-20T10:26:54.112 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:26:54.122 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:26:54.150 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:26:54.150 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/004) 2024-04-20T10:26:54.659 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:26:54.673 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:26:54.673 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:26:55.122 INFO:teuthology.orchestra.run.smithi121.stdout:generic/004 [not run] O_TMPFILE is not supported 2024-04-20T10:26:55.133 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/004 2024-04-20T10:26:55.133 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/004 2024-04-20T10:26:55.133 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:26:55.141 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:26:55.172 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:26:55.172 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/005) 2024-04-20T10:26:55.669 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:26:55.682 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:26:55.682 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:26:56.797 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:26:56.806 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:26:57.883 INFO:teuthology.orchestra.run.smithi121.stdout:generic/005 2s 2024-04-20T10:26:57.906 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/005 2024-04-20T10:26:57.907 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:26:57.915 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:26:57.956 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:26:57.956 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/006) 2024-04-20T10:26:58.429 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:26:58.442 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:26:58.442 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:27:26.944 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:27:26.953 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:27:57.082 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:27:57.090 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:28:23.089 INFO:teuthology.orchestra.run.smithi121.stdout:generic/006 85s 2024-04-20T10:28:23.102 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/006 2024-04-20T10:28:23.102 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:28:23.109 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:28:23.333 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:28:23.334 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/007) 2024-04-20T10:28:23.867 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:28:23.878 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:28:23.878 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:28:27.230 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:28:27.241 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:28:57.350 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:28:57.359 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:29:28.206 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:29:28.214 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:29:59.285 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:29:59.296 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:30:29.424 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:30:29.435 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:30:59.553 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:30:59.562 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:31:29.675 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:31:29.684 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:31:48.261 INFO:teuthology.orchestra.run.smithi121.stdout:generic/007 205s 2024-04-20T10:31:48.276 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/007 2024-04-20T10:31:48.276 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:31:48.283 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:31:48.308 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:31:48.309 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/008) 2024-04-20T10:31:48.781 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:31:48.794 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:31:48.794 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:31:52.771 INFO:teuthology.orchestra.run.smithi121.stdout:generic/008 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-04-20T10:31:52.782 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/008 2024-04-20T10:31:52.783 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/008 2024-04-20T10:31:52.783 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:31:52.791 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:31:52.819 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:31:52.819 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/009) 2024-04-20T10:31:53.303 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:31:53.315 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:31:53.315 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:31:56.785 INFO:teuthology.orchestra.run.smithi121.stdout:generic/009 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-04-20T10:31:56.800 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/009 2024-04-20T10:31:56.800 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/009 2024-04-20T10:31:56.801 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:31:56.814 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:31:56.851 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:31:56.851 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/010) 2024-04-20T10:31:57.357 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:31:57.369 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:31:57.369 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:31:58.948 INFO:teuthology.orchestra.run.smithi121.stdout:generic/010 1s 2024-04-20T10:31:59.107 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/010 2024-04-20T10:31:59.107 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:31:59.119 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:31:59.161 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:31:59.161 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/011) 2024-04-20T10:31:59.623 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:31:59.635 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:31:59.635 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:31:59.806 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:31:59.811 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:32:29.952 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:32:29.961 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:33:01.105 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:33:01.114 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:33:09.200 INFO:teuthology.orchestra.run.smithi121.stdout:generic/011 70s 2024-04-20T10:33:09.215 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/011 2024-04-20T10:33:09.215 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:33:09.224 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:33:09.251 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:33:09.252 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/012) 2024-04-20T10:33:09.858 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:33:09.869 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:33:09.869 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:33:13.620 INFO:teuthology.orchestra.run.smithi121.stdout:generic/012 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-20T10:33:13.631 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/012 2024-04-20T10:33:13.631 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/012 2024-04-20T10:33:13.632 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:33:13.640 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:33:13.687 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:33:13.688 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/013) 2024-04-20T10:33:14.162 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:33:14.175 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:33:14.176 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:33:31.288 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:33:31.298 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:34:01.432 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:34:01.438 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:34:31.884 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:34:31.892 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:35:02.488 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:35:02.494 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:35:02.609 INFO:tasks.ceph.mds.a.smithi121.stderr:2024-04-20T10:35:02.608+0000 7f95c3c00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 105817) UID: 0 2024-04-20T10:35:02.609 INFO:tasks.ceph.mds.c.smithi121.stderr:2024-04-20T10:35:02.608+0000 7f4d9da00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 105817) UID: 0 2024-04-20T10:35:32.478 INFO:teuthology.orchestra.run.smithi121.stdout:generic/013 138s 2024-04-20T10:35:32.498 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/013 2024-04-20T10:35:32.498 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:35:32.508 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:35:32.539 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:35:32.540 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/014) 2024-04-20T10:35:33.072 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:35:33.090 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:35:33.090 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:35:33.654 INFO:teuthology.orchestra.run.smithi121.stdout:generic/014 [not run] Sparse files are not supported or do not work as expected 2024-04-20T10:35:33.667 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/014 2024-04-20T10:35:33.668 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/014 2024-04-20T10:35:33.668 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:35:33.678 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:35:33.713 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:35:33.713 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/015) 2024-04-20T10:35:34.377 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:35:34.393 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:35:34.393 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:35:34.849 INFO:teuthology.orchestra.run.smithi121.stdout:generic/015 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:35:34.866 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/015 2024-04-20T10:35:34.866 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/015 2024-04-20T10:35:34.866 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:35:34.878 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:35:34.918 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:35:34.918 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/016) 2024-04-20T10:35:35.562 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:35:35.579 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:35:35.580 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:35:38.727 INFO:teuthology.orchestra.run.smithi121.stdout:generic/016 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-20T10:35:38.742 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/016 2024-04-20T10:35:38.742 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/016 2024-04-20T10:35:38.742 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:35:38.752 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:35:38.806 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:35:38.807 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/017) 2024-04-20T10:35:39.529 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:35:39.546 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:35:39.546 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:35:40.010 INFO:teuthology.orchestra.run.smithi121.stdout:generic/017 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:35:40.024 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/017 2024-04-20T10:35:40.024 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/017 2024-04-20T10:35:40.024 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:35:40.034 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:35:40.069 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:35:40.069 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/018) 2024-04-20T10:35:40.731 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:35:40.749 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:35:40.749 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:35:41.217 INFO:teuthology.orchestra.run.smithi121.stdout:generic/018 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:35:41.230 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/018 2024-04-20T10:35:41.230 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/018 2024-04-20T10:35:41.230 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:35:41.238 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:35:41.269 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:35:41.269 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/019) 2024-04-20T10:35:41.933 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:35:41.952 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:35:41.952 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:35:42.423 INFO:teuthology.orchestra.run.smithi121.stdout:generic/019 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:35:42.438 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/019 2024-04-20T10:35:42.438 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/019 2024-04-20T10:35:42.438 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:35:42.446 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:35:42.483 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:35:42.483 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/020) 2024-04-20T10:35:43.143 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:35:43.161 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:35:43.161 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:02.959 INFO:teuthology.orchestra.run.smithi121.stdout:generic/020 - output mismatch (see /tmp/tmp.yONpfXTHudxfstests-dev/results//generic/020.out.bad) 2024-04-20T10:36:02.964 INFO:teuthology.orchestra.run.smithi121.stdout: --- tests/generic/020.out 2024-04-20 10:23:05.889113168 +0000 2024-04-20T10:36:02.964 INFO:teuthology.orchestra.run.smithi121.stdout: +++ /tmp/tmp.yONpfXTHudxfstests-dev/results//generic/020.out.bad 2024-04-20 10:36:02.956114221 +0000 2024-04-20T10:36:02.965 INFO:teuthology.orchestra.run.smithi121.stdout: @@ -47,9 +47,13 @@ 2024-04-20T10:36:02.965 INFO:teuthology.orchestra.run.smithi121.stdout: user.snrub="fish2\012" 2024-04-20T10:36:02.965 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:02.965 INFO:teuthology.orchestra.run.smithi121.stdout: *** really long value 2024-04-20T10:36:02.965 INFO:teuthology.orchestra.run.smithi121.stdout: -0000000 00 2024-04-20T10:36:02.965 INFO:teuthology.orchestra.run.smithi121.stdout: -* 2024-04-20T10:36:02.965 INFO:teuthology.orchestra.run.smithi121.stdout: -ATTRSIZE 2024-04-20T10:36:02.965 INFO:teuthology.orchestra.run.smithi121.stdout: +attr_set: No space left on device 2024-04-20T10:36:02.965 INFO:teuthology.orchestra.run.smithi121.stdout: +Could not set "long_attr" for 2024-04-20T10:36:02.965 INFO:teuthology.orchestra.run.smithi121.stdout: +attr_get: No data available 2024-04-20T10:36:02.965 INFO:teuthology.orchestra.run.smithi121.stdout: +Could not get "long_attr" for 2024-04-20T10:36:02.965 INFO:teuthology.orchestra.run.smithi121.stdout: +0000000 2024-04-20T10:36:02.965 INFO:teuthology.orchestra.run.smithi121.stdout: +attr_remove: No data available 2024-04-20T10:36:02.965 INFO:teuthology.orchestra.run.smithi121.stdout: +Could not remove "long_attr" for 2024-04-20T10:36:02.965 INFO:teuthology.orchestra.run.smithi121.stdout: *** set/get/remove really long names (expect failure) 2024-04-20T10:36:02.966 INFO:teuthology.orchestra.run.smithi121.stdout: attr_set: Invalid argument 2024-04-20T10:36:02.966 INFO:teuthology.orchestra.run.smithi121.stdout: Could not set "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" for 2024-04-20T10:36:02.971 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/020 2024-04-20T10:36:02.971 INFO:teuthology.orchestra.run.smithi121.stdout:Failures: generic/020 2024-04-20T10:36:02.971 INFO:teuthology.orchestra.run.smithi121.stdout:Failed 1 of 1 tests 2024-04-20T10:36:02.978 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:03.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:36:03.023 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-20T10:36:03.023 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-20T10:36:03.024 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/021) 2024-04-20T10:36:03.673 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:36:03.690 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:36:03.690 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:06.737 INFO:teuthology.orchestra.run.smithi121.stdout:generic/021 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-20T10:36:06.751 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/021 2024-04-20T10:36:06.751 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/021 2024-04-20T10:36:06.751 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:36:06.763 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:06.811 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:36:06.811 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/022) 2024-04-20T10:36:07.530 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:36:07.546 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:36:07.547 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:10.686 INFO:teuthology.orchestra.run.smithi121.stdout:generic/022 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-20T10:36:10.704 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/022 2024-04-20T10:36:10.704 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/022 2024-04-20T10:36:10.704 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:36:10.714 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:10.772 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:36:10.772 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/023) 2024-04-20T10:36:11.503 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:36:11.515 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:36:11.515 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:15.081 INFO:teuthology.orchestra.run.smithi121.stdout:generic/023 4s 2024-04-20T10:36:15.104 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/023 2024-04-20T10:36:15.105 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:36:15.114 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:15.150 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:36:15.151 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/024) 2024-04-20T10:36:15.908 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:36:15.924 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:36:15.924 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:16.449 INFO:teuthology.orchestra.run.smithi121.stdout:generic/024 [not run] kernel doesn't support renameat2 syscall 2024-04-20T10:36:16.463 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/024 2024-04-20T10:36:16.463 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/024 2024-04-20T10:36:16.463 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:36:16.474 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:16.536 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:36:16.536 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/025) 2024-04-20T10:36:17.270 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:36:17.287 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:36:17.287 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:17.815 INFO:teuthology.orchestra.run.smithi121.stdout:generic/025 [not run] kernel doesn't support renameat2 syscall 2024-04-20T10:36:17.828 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/025 2024-04-20T10:36:17.828 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/025 2024-04-20T10:36:17.828 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:36:17.841 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:17.877 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:36:17.877 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/026) 2024-04-20T10:36:18.630 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:36:18.646 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:36:18.648 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:19.201 INFO:teuthology.orchestra.run.smithi121.stdout:generic/026 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-04-20T10:36:19.215 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/026 2024-04-20T10:36:19.215 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/026 2024-04-20T10:36:19.215 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:36:19.225 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:19.261 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:36:19.261 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/027) 2024-04-20T10:36:19.919 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:36:19.935 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:36:19.935 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:20.369 INFO:teuthology.orchestra.run.smithi121.stdout:generic/027 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:36:20.383 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/027 2024-04-20T10:36:20.383 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/027 2024-04-20T10:36:20.383 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:36:20.392 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:20.429 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:36:20.429 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/028) 2024-04-20T10:36:21.084 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:36:21.101 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:36:21.101 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:26.611 INFO:teuthology.orchestra.run.smithi121.stdout:generic/028 5s 2024-04-20T10:36:26.629 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/028 2024-04-20T10:36:26.629 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:36:26.639 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:26.675 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:36:26.675 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/029) 2024-04-20T10:36:27.325 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:36:27.339 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:36:27.339 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:27.798 INFO:teuthology.orchestra.run.smithi121.stdout:generic/029 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:36:27.810 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/029 2024-04-20T10:36:27.810 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/029 2024-04-20T10:36:27.810 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:36:27.821 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:27.856 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:36:27.856 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/030) 2024-04-20T10:36:28.496 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:36:28.512 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:36:28.513 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:28.951 INFO:teuthology.orchestra.run.smithi121.stdout:generic/030 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:36:28.968 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/030 2024-04-20T10:36:28.968 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/030 2024-04-20T10:36:28.968 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:36:28.976 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:29.005 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:36:29.006 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/031) 2024-04-20T10:36:29.638 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:36:29.653 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:36:29.653 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:30.100 INFO:teuthology.orchestra.run.smithi121.stdout:generic/031 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:36:30.112 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/031 2024-04-20T10:36:30.112 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/031 2024-04-20T10:36:30.112 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:36:30.122 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:30.154 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:36:30.154 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/032) 2024-04-20T10:36:30.808 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:36:30.827 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:36:30.827 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:31.298 INFO:teuthology.orchestra.run.smithi121.stdout:generic/032 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:36:31.310 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/032 2024-04-20T10:36:31.310 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/032 2024-04-20T10:36:31.311 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:36:31.322 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:31.362 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:36:31.363 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/033) 2024-04-20T10:36:31.903 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:36:31.922 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:36:31.922 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:32.371 INFO:teuthology.orchestra.run.smithi121.stdout:generic/033 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:36:32.387 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/033 2024-04-20T10:36:32.387 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/033 2024-04-20T10:36:32.387 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:36:32.396 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:32.442 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:36:32.443 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/034) 2024-04-20T10:36:33.116 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:36:33.127 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:36:33.127 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:33.496 INFO:teuthology.orchestra.run.smithi121.stdout:generic/034 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:36:33.509 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/034 2024-04-20T10:36:33.510 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/034 2024-04-20T10:36:33.510 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:36:33.523 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:33.566 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:36:33.566 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/035) 2024-04-20T10:36:34.230 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:36:34.249 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:36:34.249 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:34.920 INFO:teuthology.orchestra.run.smithi121.stdout:generic/035 0s 2024-04-20T10:36:35.083 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/035 2024-04-20T10:36:35.084 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:36:35.095 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:35.163 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:36:35.163 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/036) 2024-04-20T10:36:35.887 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:36:35.908 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:36:35.908 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:46.562 INFO:teuthology.orchestra.run.smithi121.stdout:generic/036 11s 2024-04-20T10:36:46.581 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/036 2024-04-20T10:36:46.581 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:36:46.592 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:46.634 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:36:46.634 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/037) 2024-04-20T10:36:47.399 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:36:47.414 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:36:47.414 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:47.849 INFO:teuthology.orchestra.run.smithi121.stdout:generic/037 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:36:47.863 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/037 2024-04-20T10:36:47.863 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/037 2024-04-20T10:36:47.863 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:36:47.872 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:47.909 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:36:47.909 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/038) 2024-04-20T10:36:48.550 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:36:48.569 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:36:48.570 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:49.012 INFO:teuthology.orchestra.run.smithi121.stdout:generic/038 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:36:49.028 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/038 2024-04-20T10:36:49.028 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/038 2024-04-20T10:36:49.028 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:36:49.037 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:49.076 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:36:49.077 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/039) 2024-04-20T10:36:49.707 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:36:49.723 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:36:49.724 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:50.201 INFO:teuthology.orchestra.run.smithi121.stdout:generic/039 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:36:50.215 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/039 2024-04-20T10:36:50.215 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/039 2024-04-20T10:36:50.216 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:36:50.225 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:50.266 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:36:50.266 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/040) 2024-04-20T10:36:50.943 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:36:50.962 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:36:50.962 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:51.429 INFO:teuthology.orchestra.run.smithi121.stdout:generic/040 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:36:51.440 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/040 2024-04-20T10:36:51.440 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/040 2024-04-20T10:36:51.441 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:36:51.449 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:51.487 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:36:51.488 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/041) 2024-04-20T10:36:52.103 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:36:52.122 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:36:52.122 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:52.597 INFO:teuthology.orchestra.run.smithi121.stdout:generic/041 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:36:52.610 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/041 2024-04-20T10:36:52.610 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/041 2024-04-20T10:36:52.610 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:36:52.621 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:52.656 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:36:52.656 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/042) 2024-04-20T10:36:53.277 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:36:53.292 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:36:53.292 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:53.732 INFO:teuthology.orchestra.run.smithi121.stdout:generic/042 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:36:53.746 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/042 2024-04-20T10:36:53.746 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/042 2024-04-20T10:36:53.746 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:36:53.756 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:53.785 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:36:53.786 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/043) 2024-04-20T10:36:54.460 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:36:54.479 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:36:54.479 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:54.908 INFO:teuthology.orchestra.run.smithi121.stdout:generic/043 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:36:54.919 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/043 2024-04-20T10:36:54.919 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/043 2024-04-20T10:36:54.919 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:36:54.927 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:54.960 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:36:54.960 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/044) 2024-04-20T10:36:55.606 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:36:55.624 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:36:55.624 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:56.110 INFO:teuthology.orchestra.run.smithi121.stdout:generic/044 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:36:56.125 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/044 2024-04-20T10:36:56.125 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/044 2024-04-20T10:36:56.125 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:36:56.134 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:56.174 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:36:56.174 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/045) 2024-04-20T10:36:56.838 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:36:56.852 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:36:56.852 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:57.295 INFO:teuthology.orchestra.run.smithi121.stdout:generic/045 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:36:57.307 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/045 2024-04-20T10:36:57.307 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/045 2024-04-20T10:36:57.307 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:36:57.321 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:57.365 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:36:57.365 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/046) 2024-04-20T10:36:58.008 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:36:58.024 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:36:58.024 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:58.447 INFO:teuthology.orchestra.run.smithi121.stdout:generic/046 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:36:58.460 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/046 2024-04-20T10:36:58.460 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/046 2024-04-20T10:36:58.460 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:36:58.469 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:58.507 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:36:58.508 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/047) 2024-04-20T10:36:59.156 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:36:59.179 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:36:59.179 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:59.608 INFO:teuthology.orchestra.run.smithi121.stdout:generic/047 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:36:59.621 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/047 2024-04-20T10:36:59.621 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/047 2024-04-20T10:36:59.621 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:36:59.633 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:36:59.670 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:36:59.670 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/048) 2024-04-20T10:37:00.313 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:37:00.329 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:37:00.329 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:00.768 INFO:teuthology.orchestra.run.smithi121.stdout:generic/048 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:37:00.782 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/048 2024-04-20T10:37:00.782 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/048 2024-04-20T10:37:00.782 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:37:00.792 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:00.824 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:37:00.824 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/049) 2024-04-20T10:37:01.447 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:37:01.462 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:37:01.463 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:01.881 INFO:teuthology.orchestra.run.smithi121.stdout:generic/049 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:37:01.897 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/049 2024-04-20T10:37:01.897 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/049 2024-04-20T10:37:01.897 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:37:01.907 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:01.945 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:37:01.945 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/050) 2024-04-20T10:37:02.596 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:37:02.613 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:37:02.614 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:04.327 INFO:teuthology.orchestra.run.smithi121.stdout:generic/050 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:37:04.341 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/050 2024-04-20T10:37:04.341 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/050 2024-04-20T10:37:04.341 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:37:04.354 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:04.399 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:37:04.400 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/051) 2024-04-20T10:37:05.046 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:37:05.063 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:37:05.063 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:05.506 INFO:teuthology.orchestra.run.smithi121.stdout:generic/051 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:37:05.518 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/051 2024-04-20T10:37:05.519 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/051 2024-04-20T10:37:05.519 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:37:05.530 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:05.573 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:37:05.573 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/052) 2024-04-20T10:37:06.202 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:37:06.220 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:37:06.220 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:06.672 INFO:teuthology.orchestra.run.smithi121.stdout:generic/052 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:37:06.684 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/052 2024-04-20T10:37:06.685 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/052 2024-04-20T10:37:06.685 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:37:06.696 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:06.726 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:37:06.726 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/053) 2024-04-20T10:37:07.357 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:37:07.372 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:37:07.372 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:07.818 INFO:teuthology.orchestra.run.smithi121.stdout:generic/053 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:37:07.829 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/053 2024-04-20T10:37:07.829 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/053 2024-04-20T10:37:07.829 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:37:07.840 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:07.880 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:37:07.880 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/054) 2024-04-20T10:37:08.510 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:37:08.530 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:37:08.530 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:08.993 INFO:teuthology.orchestra.run.smithi121.stdout:generic/054 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:37:09.006 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/054 2024-04-20T10:37:09.006 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/054 2024-04-20T10:37:09.007 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:37:09.015 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:09.046 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:37:09.046 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/055) 2024-04-20T10:37:09.879 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:37:09.898 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:37:09.898 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:10.358 INFO:teuthology.orchestra.run.smithi121.stdout:generic/055 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:37:10.370 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/055 2024-04-20T10:37:10.370 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/055 2024-04-20T10:37:10.370 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:37:10.380 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:10.417 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:37:10.417 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/056) 2024-04-20T10:37:11.106 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:37:11.124 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:37:11.124 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:11.632 INFO:teuthology.orchestra.run.smithi121.stdout:generic/056 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:37:11.645 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/056 2024-04-20T10:37:11.645 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/056 2024-04-20T10:37:11.645 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:37:11.658 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:11.695 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:37:11.696 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/057) 2024-04-20T10:37:12.335 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:37:12.357 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:37:12.357 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:12.814 INFO:teuthology.orchestra.run.smithi121.stdout:generic/057 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:37:12.827 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/057 2024-04-20T10:37:12.827 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/057 2024-04-20T10:37:12.827 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:37:12.837 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:12.870 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:37:12.870 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/058) 2024-04-20T10:37:13.512 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:37:13.529 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:37:13.530 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:16.592 INFO:teuthology.orchestra.run.smithi121.stdout:generic/058 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-20T10:37:16.607 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/058 2024-04-20T10:37:16.608 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/058 2024-04-20T10:37:16.608 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:37:16.617 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:16.658 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:37:16.658 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/059) 2024-04-20T10:37:17.395 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:37:17.416 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:37:17.416 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:17.897 INFO:teuthology.orchestra.run.smithi121.stdout:generic/059 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:37:17.910 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/059 2024-04-20T10:37:17.910 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/059 2024-04-20T10:37:17.910 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:37:17.919 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:17.955 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:37:17.956 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/060) 2024-04-20T10:37:18.581 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:37:18.598 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:37:18.598 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:21.703 INFO:teuthology.orchestra.run.smithi121.stdout:generic/060 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-20T10:37:21.715 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/060 2024-04-20T10:37:21.716 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/060 2024-04-20T10:37:21.716 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:37:21.726 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:21.777 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:37:21.777 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/061) 2024-04-20T10:37:22.526 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:37:22.542 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:37:22.542 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:25.605 INFO:teuthology.orchestra.run.smithi121.stdout:generic/061 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-20T10:37:25.618 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/061 2024-04-20T10:37:25.618 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/061 2024-04-20T10:37:25.618 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:37:25.629 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:25.692 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:37:25.692 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/062) 2024-04-20T10:37:26.412 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:37:26.432 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:37:26.432 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:26.870 INFO:teuthology.orchestra.run.smithi121.stdout:generic/062 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:37:26.882 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/062 2024-04-20T10:37:26.883 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/062 2024-04-20T10:37:26.883 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:37:26.892 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:26.931 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:37:26.931 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/063) 2024-04-20T10:37:27.628 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:37:27.646 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:37:27.647 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:30.692 INFO:teuthology.orchestra.run.smithi121.stdout:generic/063 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-20T10:37:30.702 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/063 2024-04-20T10:37:30.702 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/063 2024-04-20T10:37:30.703 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:37:30.714 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:30.761 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:37:30.761 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/064) 2024-04-20T10:37:31.506 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:37:31.526 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:37:31.526 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:31.952 INFO:teuthology.orchestra.run.smithi121.stdout:generic/064 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:37:31.963 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/064 2024-04-20T10:37:31.963 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/064 2024-04-20T10:37:31.963 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:37:31.972 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:32.004 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:37:32.004 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/065) 2024-04-20T10:37:32.659 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:37:32.673 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:37:32.673 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:33.153 INFO:teuthology.orchestra.run.smithi121.stdout:generic/065 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:37:33.171 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/065 2024-04-20T10:37:33.171 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/065 2024-04-20T10:37:33.172 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:37:33.181 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:33.214 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:37:33.214 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/066) 2024-04-20T10:37:33.884 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:37:33.900 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:37:33.900 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:34.374 INFO:teuthology.orchestra.run.smithi121.stdout:generic/066 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:37:34.388 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/066 2024-04-20T10:37:34.388 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/066 2024-04-20T10:37:34.388 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:37:34.396 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:34.437 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:37:34.437 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/067) 2024-04-20T10:37:35.102 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:37:35.119 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:37:35.119 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:35.643 INFO:teuthology.orchestra.run.smithi121.stdout:generic/067 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:37:35.656 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/067 2024-04-20T10:37:35.656 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/067 2024-04-20T10:37:35.656 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:37:35.665 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:35.699 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:37:35.699 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/068) 2024-04-20T10:37:36.325 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:37:36.346 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:37:36.348 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:36.809 INFO:teuthology.orchestra.run.smithi121.stdout:generic/068 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:37:36.822 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/068 2024-04-20T10:37:36.823 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/068 2024-04-20T10:37:36.823 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:37:36.833 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:36.869 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:37:36.869 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/069) 2024-04-20T10:37:37.518 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:37:37.534 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:37:37.534 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:37.976 INFO:teuthology.orchestra.run.smithi121.stdout:generic/069 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:37:37.989 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/069 2024-04-20T10:37:37.989 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/069 2024-04-20T10:37:37.989 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:37:37.997 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:37:38.033 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:37:38.033 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/070) 2024-04-20T10:37:38.648 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:37:38.658 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:37:38.658 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:38:17.709 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:38:17.715 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:38:47.885 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:38:47.890 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:39:18.061 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:39:18.068 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:39:43.244 INFO:teuthology.orchestra.run.smithi121.stdout:generic/070 125s 2024-04-20T10:39:43.260 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/070 2024-04-20T10:39:43.260 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:39:43.271 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:39:43.308 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:39:43.308 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/071) 2024-04-20T10:39:44.065 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:39:44.081 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:39:44.082 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:39:44.488 INFO:teuthology.orchestra.run.smithi121.stdout:generic/071 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:39:44.506 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/071 2024-04-20T10:39:44.506 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/071 2024-04-20T10:39:44.506 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:39:44.514 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:39:44.551 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:39:44.551 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/072) 2024-04-20T10:39:45.117 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:39:45.130 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:39:45.130 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:39:47.687 INFO:teuthology.orchestra.run.smithi121.stdout:generic/072 [not run] xfs_io fcollapse failed (old kernel/wrong fs?) 2024-04-20T10:39:47.698 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/072 2024-04-20T10:39:47.698 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/072 2024-04-20T10:39:47.698 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:39:47.707 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:39:47.740 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:39:47.741 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/073) 2024-04-20T10:39:48.221 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:39:48.226 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:39:48.322 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:39:48.340 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:39:48.340 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:39:48.777 INFO:teuthology.orchestra.run.smithi121.stdout:generic/073 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T10:39:48.788 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/073 2024-04-20T10:39:48.788 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/073 2024-04-20T10:39:48.788 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T10:39:48.796 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:39:48.830 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T10:39:48.830 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/074) 2024-04-20T10:39:49.431 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T10:39:49.446 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T10:39:49.446 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T10:40:18.354 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:40:18.361 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:40:48.519 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:40:48.529 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:41:18.675 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:41:18.686 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:41:48.861 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:41:48.868 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:42:19.106 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:42:19.115 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:42:49.917 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:42:49.928 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:43:20.196 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:43:20.205 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:43:50.423 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:43:50.430 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:44:21.816 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:44:21.827 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:44:51.998 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:44:52.005 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:45:23.227 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:45:23.234 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:45:55.019 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:45:55.029 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:46:29.014 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:46:29.024 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:47:01.804 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:47:01.811 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:47:32.498 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:47:32.504 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:48:02.667 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:48:02.677 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:48:32.794 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:48:32.804 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:48:32.912 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T10:48:32.911+0000 7fad6ce00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 30105) UID: 0 2024-04-20T10:48:32.912 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-04-20T10:48:32.911+0000 7f1143800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 30105) UID: 0 2024-04-20T10:48:32.913 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-04-20T10:48:32.912+0000 7f4dce800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 30105) UID: 0 2024-04-20T10:48:32.913 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-04-20T10:48:32.912+0000 7f5d77a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 30105) UID: 0 2024-04-20T10:53:08.965 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:53:08.972 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:53:09.103 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-04-20T10:53:09.103+0000 7f1143800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 30175) UID: 0 2024-04-20T10:53:09.103 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-04-20T10:53:09.103+0000 7f5d77a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 30175) UID: 0 2024-04-20T10:53:09.103 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-04-20T10:53:09.103+0000 7f4dce800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 30175) UID: 0 2024-04-20T10:53:09.106 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T10:53:09.106+0000 7fad6ce00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 30175) UID: 0 2024-04-20T10:53:09.127 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T10:53:09.127+0000 7f9c11a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 136407) UID: 0 2024-04-20T10:53:09.127 INFO:tasks.ceph.osd.2.smithi121.stderr:2024-04-20T10:53:09.127+0000 7f11dfa00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 136407) UID: 0 2024-04-20T10:53:09.128 INFO:tasks.ceph.osd.3.smithi121.stderr:2024-04-20T10:53:09.127+0000 7fca40a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 136407) UID: 0 2024-04-20T10:53:09.128 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-04-20T10:53:09.127+0000 7fb3ce400640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 136407) UID: 0 2024-04-20T11:03:24.733 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:03:24.739 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:03:24.904 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-04-20T11:03:24.904+0000 7f4dce800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 30386) UID: 0 2024-04-20T11:03:24.905 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-04-20T11:03:24.904+0000 7f1143800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 30386) UID: 0 2024-04-20T11:03:24.905 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T11:03:24.904+0000 7fad6ce00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 30386) UID: 0 2024-04-20T11:03:24.905 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-04-20T11:03:24.904+0000 7f5d77a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 30386) UID: 0 2024-04-20T11:12:31.436 INFO:teuthology.orchestra.run.smithi121.stdout:generic/074 1962s 2024-04-20T11:12:31.466 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/074 2024-04-20T11:12:31.466 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:12:31.476 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:12:31.523 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:12:31.524 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/075) 2024-04-20T11:12:32.423 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:12:32.433 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:12:32.433 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:14:20.076 INFO:teuthology.orchestra.run.smithi121.stdout:generic/075 108s 2024-04-20T11:14:20.096 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/075 2024-04-20T11:14:20.096 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:14:20.108 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:14:20.169 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:14:20.170 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/076) 2024-04-20T11:14:20.839 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:14:20.853 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:14:20.853 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:14:21.240 INFO:teuthology.orchestra.run.smithi121.stdout:generic/076 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T11:14:21.251 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/076 2024-04-20T11:14:21.251 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/076 2024-04-20T11:14:21.251 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:14:21.260 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:14:21.288 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:14:21.288 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/077) 2024-04-20T11:14:21.806 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:14:21.819 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:14:21.819 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:14:22.932 INFO:teuthology.orchestra.run.smithi121.stdout:generic/077 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T11:14:22.944 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/077 2024-04-20T11:14:22.944 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/077 2024-04-20T11:14:22.944 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:14:22.953 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:14:22.981 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:14:22.982 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/078) 2024-04-20T11:14:23.477 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:14:23.492 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:14:23.492 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:14:24.029 INFO:teuthology.orchestra.run.smithi121.stdout:generic/078 [not run] kernel doesn't support renameat2 syscall 2024-04-20T11:14:24.044 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/078 2024-04-20T11:14:24.044 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/078 2024-04-20T11:14:24.045 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:14:24.056 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:14:24.090 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:14:24.091 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/079) 2024-04-20T11:14:24.728 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:14:24.740 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:14:24.740 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:14:25.221 INFO:teuthology.orchestra.run.smithi121.stdout:generic/079 [not run] file system doesn't support chattr +ia 2024-04-20T11:14:25.233 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/079 2024-04-20T11:14:25.233 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/079 2024-04-20T11:14:25.233 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:14:25.243 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:14:25.271 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:14:25.271 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/080) 2024-04-20T11:14:25.824 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:14:25.839 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:14:25.839 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:14:28.406 INFO:teuthology.orchestra.run.smithi121.stdout:generic/080 3s 2024-04-20T11:14:28.423 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/080 2024-04-20T11:14:28.424 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:14:28.432 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:14:28.462 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:14:28.463 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/081) 2024-04-20T11:14:28.997 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:14:29.011 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:14:29.011 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:14:29.410 INFO:teuthology.orchestra.run.smithi121.stdout:generic/081 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T11:14:29.422 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/081 2024-04-20T11:14:29.422 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/081 2024-04-20T11:14:29.423 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:14:29.431 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:14:29.461 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:14:29.461 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/082) 2024-04-20T11:14:30.023 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:14:30.038 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:14:30.038 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:14:30.507 INFO:teuthology.orchestra.run.smithi121.stdout:generic/082 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T11:14:30.519 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/082 2024-04-20T11:14:30.519 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/082 2024-04-20T11:14:30.519 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:14:30.530 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:14:30.559 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:14:30.559 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/083) 2024-04-20T11:14:31.098 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:14:31.111 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:14:31.111 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:14:31.521 INFO:teuthology.orchestra.run.smithi121.stdout:generic/083 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T11:14:31.533 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/083 2024-04-20T11:14:31.533 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/083 2024-04-20T11:14:31.533 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:14:31.543 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:14:31.572 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:14:31.572 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/084) 2024-04-20T11:14:32.112 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:14:32.126 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:14:32.126 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:14:32.543 INFO:teuthology.orchestra.run.smithi121.stdout:generic/084 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T11:14:32.556 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/084 2024-04-20T11:14:32.556 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/084 2024-04-20T11:14:32.556 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:14:32.568 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:14:32.599 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:14:32.599 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/085) 2024-04-20T11:14:33.146 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:14:33.159 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:14:33.159 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:14:33.640 INFO:teuthology.orchestra.run.smithi121.stdout:generic/085 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T11:14:33.651 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/085 2024-04-20T11:14:33.651 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/085 2024-04-20T11:14:33.651 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:14:33.660 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:14:33.688 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:14:33.688 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/086) 2024-04-20T11:14:34.212 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:14:34.225 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:14:34.225 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:14:55.962 INFO:teuthology.orchestra.run.smithi121.stdout:generic/086 21s 2024-04-20T11:14:56.054 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/086 2024-04-20T11:14:56.054 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:14:56.066 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:14:56.319 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:14:56.319 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/087) 2024-04-20T11:14:57.986 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:14:57.996 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:14:57.996 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:14:58.813 INFO:teuthology.orchestra.run.smithi121.stdout:generic/087 0s 2024-04-20T11:14:58.830 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/087 2024-04-20T11:14:58.830 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:14:58.839 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:14:58.885 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:14:58.885 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/088) 2024-04-20T11:14:59.548 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:14:59.561 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:14:59.561 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:15:00.055 INFO:teuthology.orchestra.run.smithi121.stdout:generic/088 1s 2024-04-20T11:15:00.074 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/088 2024-04-20T11:15:00.074 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:15:00.085 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:15:00.131 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:15:00.131 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/089) 2024-04-20T11:15:00.776 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:15:00.788 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:15:00.788 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:16:53.241 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:16:53.248 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:16:53.419 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T11:16:53.418+0000 7f9c11a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 144792) UID: 0 2024-04-20T11:16:53.419 INFO:tasks.ceph.osd.3.smithi121.stderr:2024-04-20T11:16:53.418+0000 7fca40a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 144792) UID: 0 2024-04-20T11:16:53.419 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-04-20T11:16:53.418+0000 7fb3ce400640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 144792) UID: 0 2024-04-20T11:16:53.420 INFO:tasks.ceph.osd.2.smithi121.stderr:2024-04-20T11:16:53.418+0000 7f11dfa00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 144792) UID: 0 2024-04-20T11:16:53.512 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-04-20T11:16:53.512+0000 7f4dce800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 30616) UID: 0 2024-04-20T11:16:53.513 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-04-20T11:16:53.512+0000 7f1143800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 30616) UID: 0 2024-04-20T11:16:53.513 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T11:16:53.512+0000 7fad6ce00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 30616) UID: 0 2024-04-20T11:16:53.513 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-04-20T11:16:53.512+0000 7f5d77a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 30616) UID: 0 2024-04-20T11:29:12.681 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:29:12.721 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:29:13.083 INFO:tasks.ceph.mds.a.smithi121.stderr:2024-04-20T11:29:13.083+0000 7f95c3c00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 145045) UID: 0 2024-04-20T11:29:13.083 INFO:tasks.ceph.mds.c.smithi121.stderr:2024-04-20T11:29:13.083+0000 7f4d9da00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 145045) UID: 0 2024-04-20T11:36:48.774 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T11:36:48.773+0000 7f9c11a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 147080) UID: 0 2024-04-20T11:36:48.775 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-04-20T11:36:48.773+0000 7fb3ce400640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 147080) UID: 0 2024-04-20T11:36:48.775 INFO:tasks.ceph.osd.2.smithi121.stderr:2024-04-20T11:36:48.773+0000 7f11dfa00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 147080) UID: 0 2024-04-20T11:36:48.776 INFO:tasks.ceph.osd.3.smithi121.stderr:2024-04-20T11:36:48.773+0000 7fca40a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 147080) UID: 0 2024-04-20T11:41:00.993 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:41:01.003 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:41:01.392 INFO:tasks.ceph.mds.a.smithi121.stderr:2024-04-20T11:41:01.392+0000 7f95c3c00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 147110) UID: 0 2024-04-20T11:41:01.392 INFO:tasks.ceph.mds.c.smithi121.stderr:2024-04-20T11:41:01.392+0000 7f4d9da00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 147110) UID: 0 2024-04-20T11:45:12.645 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:45:12.682 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:45:42.843 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:45:42.852 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:45:46.843 INFO:teuthology.orchestra.run.smithi121.stdout:generic/089 1846s 2024-04-20T11:45:46.898 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/089 2024-04-20T11:45:46.898 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:45:46.910 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:45:46.951 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:45:46.952 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/090) 2024-04-20T11:45:47.924 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:45:47.938 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:45:47.938 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:45:48.486 INFO:teuthology.orchestra.run.smithi121.stdout:generic/090 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T11:45:48.498 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/090 2024-04-20T11:45:48.498 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/090 2024-04-20T11:45:48.498 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:45:48.508 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:45:48.538 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:45:48.539 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/091) 2024-04-20T11:45:49.098 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:45:49.111 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:45:49.111 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:46:12.971 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:46:12.979 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:46:43.109 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:46:43.119 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:47:13.247 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:47:13.256 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:47:43.369 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:47:43.379 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:48:13.514 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:48:13.523 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:48:43.648 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:48:43.661 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:49:13.778 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:49:13.786 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:49:43.909 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:49:43.919 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:50:14.084 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:50:14.093 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:50:44.228 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:50:44.243 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:51:14.384 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:51:14.393 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:51:36.067 INFO:teuthology.orchestra.run.smithi121.stdout:generic/091 346s 2024-04-20T11:51:36.088 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/091 2024-04-20T11:51:36.089 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:51:36.101 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:51:36.142 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:51:36.143 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/092) 2024-04-20T11:51:36.831 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:51:36.843 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:51:36.843 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:51:37.397 INFO:teuthology.orchestra.run.smithi121.stdout:generic/092 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-20T11:51:37.409 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/092 2024-04-20T11:51:37.409 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/092 2024-04-20T11:51:37.409 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:51:37.417 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:51:37.459 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:51:37.460 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/093) 2024-04-20T11:51:38.150 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:51:38.162 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:51:38.162 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:51:40.007 INFO:teuthology.orchestra.run.smithi121.stdout:generic/093 1s 2024-04-20T11:51:40.024 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/093 2024-04-20T11:51:40.024 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:51:40.032 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:51:40.076 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:51:40.076 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/094) 2024-04-20T11:51:40.754 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:51:40.766 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:51:40.766 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:51:41.160 INFO:teuthology.orchestra.run.smithi121.stdout:generic/094 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T11:51:41.173 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/094 2024-04-20T11:51:41.173 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/094 2024-04-20T11:51:41.173 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:51:41.183 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:51:41.211 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:51:41.211 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/095) 2024-04-20T11:51:41.749 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:51:41.762 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:51:41.762 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:51:42.164 INFO:teuthology.orchestra.run.smithi121.stdout:generic/095 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T11:51:42.175 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/095 2024-04-20T11:51:42.176 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/095 2024-04-20T11:51:42.176 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:51:42.186 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:51:42.216 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:51:42.216 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/096) 2024-04-20T11:51:42.738 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:51:42.751 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:51:42.751 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:51:43.192 INFO:teuthology.orchestra.run.smithi121.stdout:generic/096 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T11:51:43.206 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/096 2024-04-20T11:51:43.206 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/096 2024-04-20T11:51:43.207 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:51:43.218 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:51:43.248 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:51:43.249 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/097) 2024-04-20T11:51:43.783 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:51:43.797 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:51:43.797 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:51:44.521 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:51:44.529 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:51:44.915 INFO:teuthology.orchestra.run.smithi121.stdout:generic/097 1s 2024-04-20T11:51:44.927 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/097 2024-04-20T11:51:44.927 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:51:44.934 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:51:44.954 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:51:44.954 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/098) 2024-04-20T11:51:45.440 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:51:45.452 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:51:45.453 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:51:45.913 INFO:teuthology.orchestra.run.smithi121.stdout:generic/098 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T11:51:45.926 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/098 2024-04-20T11:51:45.926 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/098 2024-04-20T11:51:45.926 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:51:45.938 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:51:45.968 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:51:45.968 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/099) 2024-04-20T11:51:46.475 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:51:46.490 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:51:46.490 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:51:47.039 INFO:teuthology.orchestra.run.smithi121.stdout:generic/099 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-04-20T11:51:47.055 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/099 2024-04-20T11:51:47.055 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/099 2024-04-20T11:51:47.055 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:51:47.065 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:51:47.095 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:51:47.095 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/100) 2024-04-20T11:51:47.663 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:51:47.675 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:51:47.675 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:52:14.736 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:52:14.747 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:52:44.873 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:52:44.882 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:53:15.004 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:53:15.014 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:53:37.932 INFO:teuthology.orchestra.run.smithi121.stdout:generic/100 110s 2024-04-20T11:53:37.954 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/100 2024-04-20T11:53:37.955 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:53:37.967 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:53:38.030 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:53:38.031 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/101) 2024-04-20T11:53:38.766 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:53:38.776 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:53:38.777 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:53:39.143 INFO:teuthology.orchestra.run.smithi121.stdout:generic/101 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T11:53:39.154 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/101 2024-04-20T11:53:39.154 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/101 2024-04-20T11:53:39.155 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:53:39.165 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:53:39.198 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:53:39.198 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/102) 2024-04-20T11:53:39.751 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:53:39.763 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:53:39.764 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:53:40.184 INFO:teuthology.orchestra.run.smithi121.stdout:generic/102 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T11:53:40.197 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/102 2024-04-20T11:53:40.197 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/102 2024-04-20T11:53:40.197 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:53:40.207 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:53:40.240 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:53:40.241 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/103) 2024-04-20T11:53:40.801 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:53:40.814 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:53:40.814 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:53:41.247 INFO:teuthology.orchestra.run.smithi121.stdout:generic/103 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T11:53:41.258 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/103 2024-04-20T11:53:41.259 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/103 2024-04-20T11:53:41.259 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:53:41.268 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:53:41.297 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:53:41.298 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/104) 2024-04-20T11:53:41.848 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:53:41.861 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:53:41.861 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:53:42.327 INFO:teuthology.orchestra.run.smithi121.stdout:generic/104 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T11:53:42.338 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/104 2024-04-20T11:53:42.338 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/104 2024-04-20T11:53:42.338 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:53:42.348 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:53:42.377 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:53:42.378 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/105) 2024-04-20T11:53:42.932 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:53:42.946 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:53:42.946 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:53:43.381 INFO:teuthology.orchestra.run.smithi121.stdout:generic/105 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T11:53:43.394 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/105 2024-04-20T11:53:43.394 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/105 2024-04-20T11:53:43.394 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:53:43.405 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:53:43.439 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:53:43.439 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/106) 2024-04-20T11:53:43.959 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:53:43.974 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:53:43.974 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:53:44.387 INFO:teuthology.orchestra.run.smithi121.stdout:generic/106 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T11:53:44.399 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/106 2024-04-20T11:53:44.399 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/106 2024-04-20T11:53:44.399 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:53:44.408 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:53:44.438 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:53:44.438 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/107) 2024-04-20T11:53:44.952 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:53:44.965 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:53:44.965 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:53:45.126 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:53:45.132 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:53:45.349 INFO:teuthology.orchestra.run.smithi121.stdout:generic/107 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T11:53:45.362 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/107 2024-04-20T11:53:45.362 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/107 2024-04-20T11:53:45.362 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:53:45.371 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:53:45.405 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:53:45.405 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/108) 2024-04-20T11:53:45.903 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:53:45.916 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:53:45.916 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:53:46.614 INFO:teuthology.orchestra.run.smithi121.stdout:generic/108 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T11:53:46.626 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/108 2024-04-20T11:53:46.627 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/108 2024-04-20T11:53:46.627 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:53:46.635 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:53:46.663 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:53:46.664 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/109) 2024-04-20T11:53:47.184 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:53:47.196 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:53:47.196 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:53:47.589 INFO:teuthology.orchestra.run.smithi121.stdout:generic/109 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T11:53:47.600 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/109 2024-04-20T11:53:47.600 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/109 2024-04-20T11:53:47.600 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:53:47.609 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:53:47.636 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:53:47.637 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/110) 2024-04-20T11:53:48.199 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:53:48.212 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:53:48.212 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:53:48.788 INFO:teuthology.orchestra.run.smithi121.stdout:generic/110 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-20T11:53:48.800 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/110 2024-04-20T11:53:48.800 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/110 2024-04-20T11:53:48.800 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:53:48.809 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:53:48.858 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:53:48.859 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/111) 2024-04-20T11:53:49.521 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:53:49.533 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:53:49.533 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:53:50.072 INFO:teuthology.orchestra.run.smithi121.stdout:generic/111 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-20T11:53:50.084 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/111 2024-04-20T11:53:50.085 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/111 2024-04-20T11:53:50.085 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:53:50.094 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:53:50.140 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:53:50.140 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/112) 2024-04-20T11:53:50.788 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:53:50.800 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:53:50.800 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:54:15.266 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:54:15.275 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:54:45.461 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:54:45.470 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:55:15.618 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:55:15.627 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:55:45.740 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:55:45.748 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:55:56.743 INFO:teuthology.orchestra.run.smithi121.stdout:generic/112 126s 2024-04-20T11:55:56.760 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/112 2024-04-20T11:55:56.760 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:55:56.769 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:55:56.805 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:55:56.806 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/113) 2024-04-20T11:55:57.472 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:55:57.485 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:55:57.485 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:56:15.863 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:56:15.870 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:56:45.857 INFO:teuthology.orchestra.run.smithi121.stdout:generic/113 48s 2024-04-20T11:56:46.133 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:56:46.142 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:56:46.191 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/113 2024-04-20T11:56:46.191 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:56:46.201 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:56:46.251 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:56:46.251 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/114) 2024-04-20T11:56:47.104 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:56:47.115 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:56:47.115 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:56:48.116 INFO:teuthology.orchestra.run.smithi121.stdout:generic/114 [not run] Sparse files are not supported or do not work as expected 2024-04-20T11:56:48.128 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/114 2024-04-20T11:56:48.129 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/114 2024-04-20T11:56:48.129 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:56:48.138 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:56:48.168 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:56:48.168 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/115) 2024-04-20T11:56:48.763 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:56:48.777 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:56:48.777 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:56:49.316 INFO:teuthology.orchestra.run.smithi121.stdout:generic/115 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-20T11:56:49.327 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/115 2024-04-20T11:56:49.327 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/115 2024-04-20T11:56:49.327 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:56:49.336 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:56:49.366 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:56:49.366 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/116) 2024-04-20T11:56:50.048 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:56:50.060 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:56:50.061 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:56:50.590 INFO:teuthology.orchestra.run.smithi121.stdout:generic/116 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-20T11:56:50.603 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/116 2024-04-20T11:56:50.604 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/116 2024-04-20T11:56:50.604 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:56:50.615 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:56:50.643 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:56:50.643 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/117) 2024-04-20T11:56:51.320 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:56:51.333 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:56:51.333 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:56:51.717 INFO:teuthology.orchestra.run.smithi121.stdout:generic/117 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T11:56:51.730 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/117 2024-04-20T11:56:51.730 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/117 2024-04-20T11:56:51.731 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:56:51.741 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:56:51.770 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:56:51.770 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/118) 2024-04-20T11:56:52.318 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:56:52.331 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:56:52.331 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:56:52.886 INFO:teuthology.orchestra.run.smithi121.stdout:generic/118 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-20T11:56:52.897 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/118 2024-04-20T11:56:52.897 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/118 2024-04-20T11:56:52.897 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:56:52.905 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:56:53.307 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:56:53.307 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/119) 2024-04-20T11:56:53.949 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:56:53.961 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:56:53.961 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:56:54.461 INFO:teuthology.orchestra.run.smithi121.stdout:generic/119 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-20T11:56:54.472 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/119 2024-04-20T11:56:54.472 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/119 2024-04-20T11:56:54.472 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:56:54.483 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:56:54.531 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:56:54.531 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/120) 2024-04-20T11:56:55.190 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:56:55.202 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:56:55.202 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:56:55.670 INFO:teuthology.orchestra.run.smithi121.stdout:generic/120 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T11:56:55.681 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/120 2024-04-20T11:56:55.682 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/120 2024-04-20T11:56:55.682 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:56:55.690 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:56:55.719 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:56:55.719 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/121) 2024-04-20T11:56:56.299 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:56:56.312 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:56:56.313 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:56:56.883 INFO:teuthology.orchestra.run.smithi121.stdout:generic/121 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2024-04-20T11:56:56.896 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/121 2024-04-20T11:56:56.897 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/121 2024-04-20T11:56:56.897 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:56:56.907 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:56:56.942 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:56:56.942 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/122) 2024-04-20T11:56:57.642 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:56:57.655 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:56:57.655 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:56:58.228 INFO:teuthology.orchestra.run.smithi121.stdout:generic/122 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2024-04-20T11:56:58.243 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/122 2024-04-20T11:56:58.243 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/122 2024-04-20T11:56:58.243 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:56:58.253 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:56:58.282 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:56:58.283 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/123) 2024-04-20T11:56:58.973 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:56:58.984 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:56:58.985 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:57:00.179 INFO:teuthology.orchestra.run.smithi121.stdout:generic/123 1s 2024-04-20T11:57:00.195 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/123 2024-04-20T11:57:00.195 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:57:00.204 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:57:00.232 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:57:00.233 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/124) 2024-04-20T11:57:00.944 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:57:00.956 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:57:00.956 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:57:01.383 INFO:teuthology.orchestra.run.smithi121.stdout:generic/124 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T11:57:01.393 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/124 2024-04-20T11:57:01.394 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/124 2024-04-20T11:57:01.394 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:57:01.403 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:57:01.432 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:57:01.432 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/125) 2024-04-20T11:57:01.965 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:57:01.978 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:57:01.978 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:57:17.106 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:57:17.114 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:57:47.282 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:57:47.292 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:58:04.032 INFO:teuthology.orchestra.run.smithi121.stdout:generic/125 62s 2024-04-20T11:58:04.052 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/125 2024-04-20T11:58:04.052 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T11:58:04.063 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:58:04.101 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T11:58:04.102 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/126) 2024-04-20T11:58:04.748 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:58:04.759 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:58:04.759 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:58:06.358 INFO:teuthology.orchestra.run.smithi121.stdout:generic/126 - output mismatch (see /tmp/tmp.yONpfXTHudxfstests-dev/results//generic/126.out.bad) 2024-04-20T11:58:06.366 INFO:teuthology.orchestra.run.smithi121.stdout: --- tests/generic/126.out 2024-04-20 10:23:05.905112959 +0000 2024-04-20T11:58:06.366 INFO:teuthology.orchestra.run.smithi121.stdout: +++ /tmp/tmp.yONpfXTHudxfstests-dev/results//generic/126.out.bad 2024-04-20 11:58:06.354474875 +0000 2024-04-20T11:58:06.367 INFO:teuthology.orchestra.run.smithi121.stdout: @@ -1,7 +1,7 @@ 2024-04-20T11:58:06.367 INFO:teuthology.orchestra.run.smithi121.stdout: QA output created by 126 2024-04-20T11:58:06.367 INFO:teuthology.orchestra.run.smithi121.stdout: -x a 001 file owned by (99/99) as user/group(12/100) PASS 2024-04-20T11:58:06.367 INFO:teuthology.orchestra.run.smithi121.stdout: -x a 010 file owned by (99/99) as user/group(200/99) PASS 2024-04-20T11:58:06.367 INFO:teuthology.orchestra.run.smithi121.stdout: -x a 100 file owned by (99/99) as user/group(99/500) PASS 2024-04-20T11:58:06.367 INFO:teuthology.orchestra.run.smithi121.stdout: +x a 001 file owned by (99/99) as user/group(12/100) FAIL 2024-04-20T11:58:06.367 INFO:teuthology.orchestra.run.smithi121.stdout: +x a 010 file owned by (99/99) as user/group(200/99) FAIL 2024-04-20T11:58:06.367 INFO:teuthology.orchestra.run.smithi121.stdout: +x a 100 file owned by (99/99) as user/group(99/500) FAIL 2024-04-20T11:58:06.367 INFO:teuthology.orchestra.run.smithi121.stdout: w a 002 file owned by (99/99) as user/group(12/100) PASS 2024-04-20T11:58:06.367 INFO:teuthology.orchestra.run.smithi121.stdout: w a 020 file owned by (99/99) as user/group(200/99) PASS 2024-04-20T11:58:06.367 INFO:teuthology.orchestra.run.smithi121.stdout: w a 200 file owned by (99/99) as user/group(99/500) PASS 2024-04-20T11:58:06.374 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/126 2024-04-20T11:58:06.374 INFO:teuthology.orchestra.run.smithi121.stdout:Failures: generic/126 2024-04-20T11:58:06.374 INFO:teuthology.orchestra.run.smithi121.stdout:Failed 1 of 1 tests 2024-04-20T11:58:06.384 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:58:06.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T11:58:06.410 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-20T11:58:06.411 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-20T11:58:06.411 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/127) 2024-04-20T11:58:07.122 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T11:58:07.134 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T11:58:07.134 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T11:58:17.486 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:58:17.495 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:58:47.616 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:58:47.626 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:59:17.753 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:59:17.762 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:59:47.885 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T11:59:47.893 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:00:18.042 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:00:18.105 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:00:48.238 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:00:48.248 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:01:18.369 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:01:18.379 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:01:48.491 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:01:48.499 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:02:18.621 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:02:18.633 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:02:48.778 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:02:48.786 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:03:18.899 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:03:18.908 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:03:49.037 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:03:49.046 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:04:19.160 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:04:19.170 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:04:49.298 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:04:49.308 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:05:19.468 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:05:19.476 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:05:49.610 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:05:49.619 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:06:19.725 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:06:19.734 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:06:49.861 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:06:49.871 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:07:19.991 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:07:20.001 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:07:50.114 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:07:50.123 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:08:20.267 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:08:20.277 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:08:50.404 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:08:50.412 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:09:20.521 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:09:20.532 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:09:50.653 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:09:50.663 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:10:20.774 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:10:20.785 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:10:50.904 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:10:50.914 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:11:21.115 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:11:21.123 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:11:51.234 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:11:51.244 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:12:21.473 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:12:21.481 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:12:51.627 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:12:51.636 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:13:22.266 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:13:22.275 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:13:52.411 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:13:52.421 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:14:22.564 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:14:22.573 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:14:52.697 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:14:52.707 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:15:22.819 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:15:22.828 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:15:53.098 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:15:53.107 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:16:23.238 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:16:23.248 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:16:53.364 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:16:53.374 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:17:23.481 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:17:23.491 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:17:53.621 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:17:53.630 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:18:23.749 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:18:23.758 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:18:53.880 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:18:53.889 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:19:24.011 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:19:24.021 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:19:54.175 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:19:54.183 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:20:24.433 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:20:24.442 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:20:54.629 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:20:54.638 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:21:24.784 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:21:24.793 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:21:54.928 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:21:54.938 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:22:25.067 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:22:25.076 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:22:55.188 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:22:55.198 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:23:25.501 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:23:25.510 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:23:55.699 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:23:55.709 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:24:25.888 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:24:25.898 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:24:56.044 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:24:56.057 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:25:26.185 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:25:26.193 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:25:56.355 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:25:56.364 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:26:26.485 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:26:26.494 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:26:56.642 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:26:56.651 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:27:26.784 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:27:26.793 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:27:56.909 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:27:56.918 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:28:27.059 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:28:27.068 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:28:57.196 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:28:57.206 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:29:27.333 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:29:27.342 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:29:57.470 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:29:57.480 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:30:27.607 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:30:27.617 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:30:57.817 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:30:57.827 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:31:27.957 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:31:27.966 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:31:58.103 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:31:58.113 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:32:28.248 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:32:28.257 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:32:58.402 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:32:58.410 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:33:28.524 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:33:28.535 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:33:58.681 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:33:58.690 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:34:28.804 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:34:28.813 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:34:58.940 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:34:58.949 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:35:29.064 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:35:29.075 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:35:59.185 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:35:59.195 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:36:29.321 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:36:29.330 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:36:59.481 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:36:59.490 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:37:29.611 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:37:29.623 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:37:59.757 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:37:59.766 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:38:29.897 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:38:29.907 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:38:34.456 INFO:teuthology.orchestra.run.smithi121.stdout:generic/127 2427s 2024-04-20T12:38:34.473 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/127 2024-04-20T12:38:34.474 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:38:34.482 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:38:34.525 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:38:34.525 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/128) 2024-04-20T12:38:35.219 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:38:35.229 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:38:35.230 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:38:35.749 INFO:teuthology.orchestra.run.smithi121.stdout:generic/128 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:38:35.763 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/128 2024-04-20T12:38:35.763 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/128 2024-04-20T12:38:35.764 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:38:35.776 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:38:35.809 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:38:35.810 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/129) 2024-04-20T12:38:36.350 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:38:36.363 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:38:36.364 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:38:36.778 INFO:teuthology.orchestra.run.smithi121.stdout:generic/129 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:38:36.788 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/129 2024-04-20T12:38:36.788 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/129 2024-04-20T12:38:36.788 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:38:36.796 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:38:36.822 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:38:36.822 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/130) 2024-04-20T12:38:37.356 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:38:37.370 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:38:37.371 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:38:37.818 INFO:teuthology.orchestra.run.smithi121.stdout:generic/130 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:38:37.828 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/130 2024-04-20T12:38:37.829 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/130 2024-04-20T12:38:37.829 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:38:37.838 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:38:37.866 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:38:37.866 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/131) 2024-04-20T12:38:38.411 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:38:38.423 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:38:38.423 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:38:40.395 INFO:teuthology.orchestra.run.smithi121.stdout:generic/131 2s 2024-04-20T12:38:40.412 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/131 2024-04-20T12:38:40.412 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:38:40.420 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:38:40.449 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:38:40.449 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/132) 2024-04-20T12:38:41.138 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:38:41.151 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:38:41.151 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:38:41.628 INFO:teuthology.orchestra.run.smithi121.stdout:generic/132 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:38:41.639 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/132 2024-04-20T12:38:41.639 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/132 2024-04-20T12:38:41.640 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:38:41.649 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:38:41.677 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:38:41.677 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/133) 2024-04-20T12:38:42.217 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:38:42.232 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:38:42.232 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:39:00.027 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:39:00.036 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:39:30.208 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:39:30.217 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:40:00.346 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:40:00.356 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:40:30.484 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:40:30.494 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:41:00.854 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:41:00.864 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:41:30.996 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:41:31.006 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:42:01.123 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:42:01.133 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:42:31.242 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:42:31.252 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:43:01.371 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:43:01.380 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:43:31.500 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:43:31.509 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:44:01.621 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:44:01.631 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:44:31.769 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:44:31.778 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:45:01.895 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:45:01.903 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:45:32.058 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:45:32.069 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:46:02.201 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:46:02.210 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:46:32.670 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:46:32.679 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:47:02.804 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:47:02.813 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:47:32.959 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:47:32.968 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:48:03.118 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:48:03.127 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:48:33.286 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:48:33.295 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:49:03.400 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:49:03.409 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:49:33.529 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:49:33.537 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:49:38.597 INFO:teuthology.orchestra.run.smithi121.stdout:generic/133 656s 2024-04-20T12:49:38.617 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/133 2024-04-20T12:49:38.617 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:49:38.628 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:49:38.680 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:49:38.681 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/134) 2024-04-20T12:49:39.438 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:49:39.449 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:49:39.449 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:49:40.006 INFO:teuthology.orchestra.run.smithi121.stdout:generic/134 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-20T12:49:40.020 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/134 2024-04-20T12:49:40.020 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/134 2024-04-20T12:49:40.020 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:49:40.030 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:49:40.072 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:49:40.073 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/135) 2024-04-20T12:49:40.756 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:49:40.767 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:49:40.767 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:49:41.295 INFO:teuthology.orchestra.run.smithi121.stdout:generic/135 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:49:41.309 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/135 2024-04-20T12:49:41.310 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/135 2024-04-20T12:49:41.310 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:49:41.321 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:49:41.351 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:49:41.351 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/136) 2024-04-20T12:49:41.892 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:49:41.904 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:49:41.904 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:49:42.493 INFO:teuthology.orchestra.run.smithi121.stdout:generic/136 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2024-04-20T12:49:42.505 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/136 2024-04-20T12:49:42.505 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/136 2024-04-20T12:49:42.505 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:49:42.515 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:49:42.554 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:49:42.554 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/137) 2024-04-20T12:49:43.211 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:49:43.222 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:49:43.222 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:49:43.728 INFO:teuthology.orchestra.run.smithi121.stdout:generic/137 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-20T12:49:43.742 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/137 2024-04-20T12:49:43.742 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/137 2024-04-20T12:49:43.743 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:49:43.755 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:49:43.803 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:49:43.804 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/138) 2024-04-20T12:49:44.449 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:49:44.462 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:49:44.462 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:49:44.986 INFO:teuthology.orchestra.run.smithi121.stdout:generic/138 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-20T12:49:44.997 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/138 2024-04-20T12:49:44.997 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/138 2024-04-20T12:49:44.997 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:49:45.006 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:49:45.046 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:49:45.046 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/139) 2024-04-20T12:49:45.706 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:49:45.719 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:49:45.719 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:49:46.234 INFO:teuthology.orchestra.run.smithi121.stdout:generic/139 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-20T12:49:46.244 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/139 2024-04-20T12:49:46.244 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/139 2024-04-20T12:49:46.244 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:49:46.253 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:49:46.296 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:49:46.296 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/140) 2024-04-20T12:49:46.962 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:49:46.975 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:49:46.975 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:49:47.501 INFO:teuthology.orchestra.run.smithi121.stdout:generic/140 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-20T12:49:47.514 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/140 2024-04-20T12:49:47.514 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/140 2024-04-20T12:49:47.514 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:49:47.525 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:49:47.566 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:49:47.567 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/141) 2024-04-20T12:49:48.230 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:49:48.242 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:49:48.243 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:49:48.627 INFO:teuthology.orchestra.run.smithi121.stdout:generic/141 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:49:48.638 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/141 2024-04-20T12:49:48.638 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/141 2024-04-20T12:49:48.638 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:49:48.646 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:49:48.677 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:49:48.678 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/142) 2024-04-20T12:49:49.213 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:49:49.226 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:49:49.227 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:49:49.784 INFO:teuthology.orchestra.run.smithi121.stdout:generic/142 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-20T12:49:49.795 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/142 2024-04-20T12:49:49.796 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/142 2024-04-20T12:49:49.796 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:49:49.805 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:49:49.861 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:49:49.861 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/143) 2024-04-20T12:49:50.521 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:49:50.532 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:49:50.532 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:49:51.055 INFO:teuthology.orchestra.run.smithi121.stdout:generic/143 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-20T12:49:51.066 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/143 2024-04-20T12:49:51.066 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/143 2024-04-20T12:49:51.067 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:49:51.076 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:49:51.111 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:49:51.111 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/144) 2024-04-20T12:49:51.786 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:49:51.798 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:49:51.798 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:49:52.290 INFO:teuthology.orchestra.run.smithi121.stdout:generic/144 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-20T12:49:52.302 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/144 2024-04-20T12:49:52.302 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/144 2024-04-20T12:49:52.302 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:49:52.310 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:49:52.355 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:49:52.356 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/145) 2024-04-20T12:49:52.999 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:49:53.010 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:49:53.010 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:49:53.517 INFO:teuthology.orchestra.run.smithi121.stdout:generic/145 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-20T12:49:53.530 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/145 2024-04-20T12:49:53.530 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/145 2024-04-20T12:49:53.531 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:49:53.542 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:49:53.606 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:49:53.606 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/146) 2024-04-20T12:49:54.248 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:49:54.258 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:49:54.258 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:49:54.771 INFO:teuthology.orchestra.run.smithi121.stdout:generic/146 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-20T12:49:54.785 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/146 2024-04-20T12:49:54.785 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/146 2024-04-20T12:49:54.785 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:49:54.797 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:49:54.838 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:49:54.838 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/147) 2024-04-20T12:49:55.481 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:49:55.493 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:49:55.493 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:49:56.043 INFO:teuthology.orchestra.run.smithi121.stdout:generic/147 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-20T12:49:56.053 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/147 2024-04-20T12:49:56.054 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/147 2024-04-20T12:49:56.054 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:49:56.062 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:49:56.094 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:49:56.094 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/148) 2024-04-20T12:49:56.774 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:49:56.787 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:49:56.787 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:49:57.284 INFO:teuthology.orchestra.run.smithi121.stdout:generic/148 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-20T12:49:57.295 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/148 2024-04-20T12:49:57.295 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/148 2024-04-20T12:49:57.295 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:49:57.305 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:49:57.349 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:49:57.349 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/149) 2024-04-20T12:49:57.991 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:49:58.004 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:49:58.004 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:49:58.531 INFO:teuthology.orchestra.run.smithi121.stdout:generic/149 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-20T12:49:58.542 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/149 2024-04-20T12:49:58.542 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/149 2024-04-20T12:49:58.542 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:49:58.550 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:49:58.583 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:49:58.583 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/150) 2024-04-20T12:49:59.273 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:49:59.286 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:49:59.286 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:49:59.789 INFO:teuthology.orchestra.run.smithi121.stdout:generic/150 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-20T12:49:59.802 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/150 2024-04-20T12:49:59.802 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/150 2024-04-20T12:49:59.802 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:49:59.814 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:49:59.854 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:49:59.854 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/151) 2024-04-20T12:50:00.529 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:00.540 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:00.540 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:01.106 INFO:teuthology.orchestra.run.smithi121.stdout:generic/151 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-20T12:50:01.117 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/151 2024-04-20T12:50:01.117 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/151 2024-04-20T12:50:01.117 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:01.126 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:01.163 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:01.163 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/152) 2024-04-20T12:50:01.847 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:01.860 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:01.860 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:02.374 INFO:teuthology.orchestra.run.smithi121.stdout:generic/152 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-20T12:50:02.385 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/152 2024-04-20T12:50:02.385 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/152 2024-04-20T12:50:02.386 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:02.395 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:02.446 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:02.446 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/153) 2024-04-20T12:50:03.071 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:03.083 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:03.084 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:03.562 INFO:teuthology.orchestra.run.smithi121.stdout:generic/153 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-20T12:50:03.573 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/153 2024-04-20T12:50:03.573 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/153 2024-04-20T12:50:03.573 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:03.581 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:03.619 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:03.619 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/154) 2024-04-20T12:50:03.650 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:50:03.681 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:50:04.295 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:04.307 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:04.307 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:04.807 INFO:teuthology.orchestra.run.smithi121.stdout:generic/154 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-20T12:50:04.819 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/154 2024-04-20T12:50:04.819 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/154 2024-04-20T12:50:04.819 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:04.830 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:04.878 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:04.879 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/155) 2024-04-20T12:50:05.539 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:05.552 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:05.552 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:06.087 INFO:teuthology.orchestra.run.smithi121.stdout:generic/155 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-20T12:50:06.100 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/155 2024-04-20T12:50:06.100 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/155 2024-04-20T12:50:06.100 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:06.111 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:06.161 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:06.161 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/156) 2024-04-20T12:50:06.836 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:06.849 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:06.849 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:07.414 INFO:teuthology.orchestra.run.smithi121.stdout:generic/156 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-20T12:50:07.426 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/156 2024-04-20T12:50:07.426 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/156 2024-04-20T12:50:07.426 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:07.435 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:07.480 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:07.480 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/157) 2024-04-20T12:50:08.142 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:08.154 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:08.154 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:08.681 INFO:teuthology.orchestra.run.smithi121.stdout:generic/157 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-20T12:50:08.693 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/157 2024-04-20T12:50:08.693 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/157 2024-04-20T12:50:08.693 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:08.702 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:08.748 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:08.748 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/158) 2024-04-20T12:50:09.415 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:09.426 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:09.426 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:09.927 INFO:teuthology.orchestra.run.smithi121.stdout:generic/158 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2024-04-20T12:50:09.938 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/158 2024-04-20T12:50:09.938 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/158 2024-04-20T12:50:09.938 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:09.945 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:09.985 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:09.985 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/159) 2024-04-20T12:50:10.620 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:10.632 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:10.633 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:11.142 INFO:teuthology.orchestra.run.smithi121.stdout:generic/159 [not run] file system doesn't support chattr +i 2024-04-20T12:50:11.155 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/159 2024-04-20T12:50:11.155 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/159 2024-04-20T12:50:11.155 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:11.166 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:11.200 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:11.200 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/160) 2024-04-20T12:50:11.723 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:11.736 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:11.736 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:12.220 INFO:teuthology.orchestra.run.smithi121.stdout:generic/160 [not run] file system doesn't support chattr +i 2024-04-20T12:50:12.232 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/160 2024-04-20T12:50:12.232 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/160 2024-04-20T12:50:12.232 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:12.241 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:12.270 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:12.270 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/161) 2024-04-20T12:50:12.786 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:12.798 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:12.798 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:13.240 INFO:teuthology.orchestra.run.smithi121.stdout:generic/161 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:50:13.255 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/161 2024-04-20T12:50:13.255 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/161 2024-04-20T12:50:13.255 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:13.267 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:13.298 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:13.298 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/162) 2024-04-20T12:50:13.842 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:13.855 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:13.855 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:14.335 INFO:teuthology.orchestra.run.smithi121.stdout:generic/162 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:50:14.346 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/162 2024-04-20T12:50:14.347 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/162 2024-04-20T12:50:14.347 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:14.356 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:14.383 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:14.383 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/163) 2024-04-20T12:50:14.932 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:14.945 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:14.946 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:15.395 INFO:teuthology.orchestra.run.smithi121.stdout:generic/163 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:50:15.409 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/163 2024-04-20T12:50:15.409 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/163 2024-04-20T12:50:15.409 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:15.421 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:15.454 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:15.454 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/164) 2024-04-20T12:50:16.014 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:16.028 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:16.028 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:16.429 INFO:teuthology.orchestra.run.smithi121.stdout:generic/164 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:50:16.443 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/164 2024-04-20T12:50:16.443 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/164 2024-04-20T12:50:16.443 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:16.455 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:16.487 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:16.487 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/165) 2024-04-20T12:50:17.059 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:17.073 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:17.074 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:17.490 INFO:teuthology.orchestra.run.smithi121.stdout:generic/165 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:50:17.504 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/165 2024-04-20T12:50:17.504 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/165 2024-04-20T12:50:17.504 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:17.516 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:17.546 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:17.546 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/166) 2024-04-20T12:50:18.117 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:18.130 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:18.130 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:18.556 INFO:teuthology.orchestra.run.smithi121.stdout:generic/166 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:50:18.567 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/166 2024-04-20T12:50:18.568 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/166 2024-04-20T12:50:18.568 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:18.577 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:18.604 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:18.605 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/167) 2024-04-20T12:50:19.154 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:19.167 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:19.168 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:19.594 INFO:teuthology.orchestra.run.smithi121.stdout:generic/167 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:50:19.605 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/167 2024-04-20T12:50:19.605 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/167 2024-04-20T12:50:19.605 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:19.614 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:19.640 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:19.641 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/168) 2024-04-20T12:50:20.180 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:20.193 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:20.193 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:20.619 INFO:teuthology.orchestra.run.smithi121.stdout:generic/168 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:50:20.633 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/168 2024-04-20T12:50:20.633 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/168 2024-04-20T12:50:20.634 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:20.645 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:20.677 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:20.677 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/169) 2024-04-20T12:50:21.195 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:21.209 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:21.209 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:21.646 INFO:teuthology.orchestra.run.smithi121.stdout:generic/169 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:50:21.660 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/169 2024-04-20T12:50:21.660 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/169 2024-04-20T12:50:21.660 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:21.671 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:21.708 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:21.708 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/170) 2024-04-20T12:50:22.256 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:22.271 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:22.271 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:22.683 INFO:teuthology.orchestra.run.smithi121.stdout:generic/170 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:50:22.694 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/170 2024-04-20T12:50:22.695 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/170 2024-04-20T12:50:22.695 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:22.704 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:22.737 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:22.737 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/171) 2024-04-20T12:50:23.289 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:23.303 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:23.303 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:23.734 INFO:teuthology.orchestra.run.smithi121.stdout:generic/171 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:50:23.745 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/171 2024-04-20T12:50:23.745 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/171 2024-04-20T12:50:23.746 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:23.754 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:23.784 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:23.785 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/172) 2024-04-20T12:50:24.333 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:24.345 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:24.345 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:24.773 INFO:teuthology.orchestra.run.smithi121.stdout:generic/172 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:50:24.786 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/172 2024-04-20T12:50:24.786 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/172 2024-04-20T12:50:24.787 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:24.799 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:24.827 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:24.828 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/173) 2024-04-20T12:50:25.381 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:25.394 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:25.394 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:25.832 INFO:teuthology.orchestra.run.smithi121.stdout:generic/173 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:50:25.845 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/173 2024-04-20T12:50:25.845 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/173 2024-04-20T12:50:25.845 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:25.858 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:25.887 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:25.888 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/174) 2024-04-20T12:50:26.452 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:26.466 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:26.466 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:26.919 INFO:teuthology.orchestra.run.smithi121.stdout:generic/174 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:50:26.931 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/174 2024-04-20T12:50:26.931 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/174 2024-04-20T12:50:26.931 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:26.941 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:26.972 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:26.973 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/175) 2024-04-20T12:50:27.512 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:27.524 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:27.524 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:27.954 INFO:teuthology.orchestra.run.smithi121.stdout:generic/175 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:50:27.967 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/175 2024-04-20T12:50:27.967 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/175 2024-04-20T12:50:27.967 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:27.978 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:28.015 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:28.015 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/176) 2024-04-20T12:50:28.570 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:28.586 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:28.586 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:29.023 INFO:teuthology.orchestra.run.smithi121.stdout:generic/176 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:50:29.035 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/176 2024-04-20T12:50:29.035 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/176 2024-04-20T12:50:29.036 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:29.045 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:29.075 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:29.075 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/177) 2024-04-20T12:50:29.602 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:29.615 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:29.615 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:30.099 INFO:teuthology.orchestra.run.smithi121.stdout:generic/177 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:50:30.113 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/177 2024-04-20T12:50:30.113 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/177 2024-04-20T12:50:30.114 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:30.125 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:30.156 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:30.156 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/178) 2024-04-20T12:50:30.688 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:30.702 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:30.702 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:31.261 INFO:teuthology.orchestra.run.smithi121.stdout:generic/178 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-20T12:50:31.273 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/178 2024-04-20T12:50:31.273 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/178 2024-04-20T12:50:31.273 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:31.281 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:31.318 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:31.318 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/179) 2024-04-20T12:50:31.992 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:32.003 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:32.003 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:32.586 INFO:teuthology.orchestra.run.smithi121.stdout:generic/179 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-20T12:50:32.600 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/179 2024-04-20T12:50:32.600 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/179 2024-04-20T12:50:32.601 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:32.612 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:32.654 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:32.654 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/180) 2024-04-20T12:50:33.379 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:33.392 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:33.392 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:33.918 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:50:33.927 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:50:33.931 INFO:teuthology.orchestra.run.smithi121.stdout:generic/180 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-20T12:50:33.942 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/180 2024-04-20T12:50:33.942 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/180 2024-04-20T12:50:33.942 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:33.950 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:33.992 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:33.992 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/181) 2024-04-20T12:50:34.661 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:34.674 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:34.674 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:35.245 INFO:teuthology.orchestra.run.smithi121.stdout:generic/181 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-20T12:50:35.259 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/181 2024-04-20T12:50:35.260 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/181 2024-04-20T12:50:35.260 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:35.271 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:35.320 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:35.320 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/182) 2024-04-20T12:50:35.972 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:35.983 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:35.984 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:36.516 INFO:teuthology.orchestra.run.smithi121.stdout:generic/182 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2024-04-20T12:50:36.530 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/182 2024-04-20T12:50:36.530 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/182 2024-04-20T12:50:36.530 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:36.541 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:36.594 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:36.594 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/183) 2024-04-20T12:50:37.253 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:37.266 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:37.266 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:37.689 INFO:teuthology.orchestra.run.smithi121.stdout:generic/183 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:50:37.702 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/183 2024-04-20T12:50:37.702 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/183 2024-04-20T12:50:37.702 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:37.711 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:37.741 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:37.741 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/184) 2024-04-20T12:50:38.279 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:38.291 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:38.291 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:38.831 INFO:teuthology.orchestra.run.smithi121.stdout:generic/184 [failed, exit status 1]- output mismatch (see /tmp/tmp.yONpfXTHudxfstests-dev/results//generic/184.out.bad) 2024-04-20T12:50:38.839 INFO:teuthology.orchestra.run.smithi121.stdout: --- tests/generic/184.out 2024-04-20 10:23:05.914112841 +0000 2024-04-20T12:50:38.839 INFO:teuthology.orchestra.run.smithi121.stdout: +++ /tmp/tmp.yONpfXTHudxfstests-dev/results//generic/184.out.bad 2024-04-20 12:50:38.827714214 +0000 2024-04-20T12:50:38.839 INFO:teuthology.orchestra.run.smithi121.stdout: @@ -1 +1,2 @@ 2024-04-20T12:50:38.839 INFO:teuthology.orchestra.run.smithi121.stdout: QA output created by 184 2024-04-20T12:50:38.839 INFO:teuthology.orchestra.run.smithi121.stdout: +/tmp/tmp.yONpfXTHudxfstests-dev/tests/generic/184: line 23: /tmp/tmp.ru8ajEsoUutest/null: Permission denied 2024-04-20T12:50:38.847 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/184 2024-04-20T12:50:38.847 INFO:teuthology.orchestra.run.smithi121.stdout:Failures: generic/184 2024-04-20T12:50:38.847 INFO:teuthology.orchestra.run.smithi121.stdout:Failed 1 of 1 tests 2024-04-20T12:50:38.857 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:38.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T12:50:38.887 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-20T12:50:38.887 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-20T12:50:38.887 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/185) 2024-04-20T12:50:39.429 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:39.443 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:39.443 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:39.874 INFO:teuthology.orchestra.run.smithi121.stdout:generic/185 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:50:39.885 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/185 2024-04-20T12:50:39.885 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/185 2024-04-20T12:50:39.885 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:39.893 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:39.922 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:39.922 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/186) 2024-04-20T12:50:40.450 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:40.464 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:40.464 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:40.906 INFO:teuthology.orchestra.run.smithi121.stdout:generic/186 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:50:40.918 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/186 2024-04-20T12:50:40.918 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/186 2024-04-20T12:50:40.918 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:40.926 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:40.954 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:40.954 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/187) 2024-04-20T12:50:41.492 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:41.505 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:41.505 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:41.892 INFO:teuthology.orchestra.run.smithi121.stdout:generic/187 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:50:41.903 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/187 2024-04-20T12:50:41.903 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/187 2024-04-20T12:50:41.903 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:41.914 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:41.944 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:41.944 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/188) 2024-04-20T12:50:42.477 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:42.491 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:42.492 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:42.941 INFO:teuthology.orchestra.run.smithi121.stdout:generic/188 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:50:42.955 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/188 2024-04-20T12:50:42.955 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/188 2024-04-20T12:50:42.955 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:42.966 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:42.995 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:42.995 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/189) 2024-04-20T12:50:43.514 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:43.527 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:43.528 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:43.947 INFO:teuthology.orchestra.run.smithi121.stdout:generic/189 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:50:43.958 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/189 2024-04-20T12:50:43.958 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/189 2024-04-20T12:50:43.958 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:43.967 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:43.991 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:43.991 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/190) 2024-04-20T12:50:44.528 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:44.541 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:44.542 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:44.944 INFO:teuthology.orchestra.run.smithi121.stdout:generic/190 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:50:44.955 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/190 2024-04-20T12:50:44.955 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/190 2024-04-20T12:50:44.955 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:44.964 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:44.990 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:44.991 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/191) 2024-04-20T12:50:45.484 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:45.497 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:45.497 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:45.946 INFO:teuthology.orchestra.run.smithi121.stdout:generic/191 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:50:45.960 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/191 2024-04-20T12:50:45.960 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/191 2024-04-20T12:50:45.960 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:45.972 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:46.004 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:46.005 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/192) 2024-04-20T12:50:46.555 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:46.569 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:46.569 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:47.013 INFO:teuthology.orchestra.run.smithi121.stdout:generic/192 [not run] atime not maintained by ceph-fuse 2024-04-20T12:50:47.024 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/192 2024-04-20T12:50:47.025 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/192 2024-04-20T12:50:47.025 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:47.034 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:47.062 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:47.062 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/193) 2024-04-20T12:50:47.627 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:47.640 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:47.640 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:50.300 INFO:teuthology.orchestra.run.smithi121.stdout:generic/193 - output mismatch (see /tmp/tmp.yONpfXTHudxfstests-dev/results//generic/193.out.bad) 2024-04-20T12:50:50.307 INFO:teuthology.orchestra.run.smithi121.stdout: --- tests/generic/193.out 2024-04-20 10:23:05.916112815 +0000 2024-04-20T12:50:50.307 INFO:teuthology.orchestra.run.smithi121.stdout: +++ /tmp/tmp.yONpfXTHudxfstests-dev/results//generic/193.out.bad 2024-04-20 12:50:50.296562091 +0000 2024-04-20T12:50:50.307 INFO:teuthology.orchestra.run.smithi121.stdout: @@ -24,7 +24,6 @@ 2024-04-20T12:50:50.308 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:50.308 INFO:teuthology.orchestra.run.smithi121.stdout: user: chmod a+r on qa_user owned file (should succeed) 2024-04-20T12:50:50.308 INFO:teuthology.orchestra.run.smithi121.stdout: user: chmod a+r on root owned file (should fail) 2024-04-20T12:50:50.308 INFO:teuthology.orchestra.run.smithi121.stdout: -chmod: changing permissions of 'test.root': Operation not permitted 2024-04-20T12:50:50.308 INFO:teuthology.orchestra.run.smithi121.stdout: check that the sgid bit is cleared 2024-04-20T12:50:50.308 INFO:teuthology.orchestra.run.smithi121.stdout: -rw-rw-rw- 2024-04-20T12:50:50.308 INFO:teuthology.orchestra.run.smithi121.stdout: check that suid bit is not cleared 2024-04-20T12:50:50.314 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/193 2024-04-20T12:50:50.314 INFO:teuthology.orchestra.run.smithi121.stdout:Failures: generic/193 2024-04-20T12:50:50.315 INFO:teuthology.orchestra.run.smithi121.stdout:Failed 1 of 1 tests 2024-04-20T12:50:50.322 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:50.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T12:50:50.361 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-20T12:50:50.361 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-20T12:50:50.361 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/194) 2024-04-20T12:50:51.042 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:51.056 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:51.056 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:51.453 INFO:teuthology.orchestra.run.smithi121.stdout:generic/194 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:50:51.466 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/194 2024-04-20T12:50:51.466 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/194 2024-04-20T12:50:51.467 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:51.478 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:51.508 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:51.508 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/195) 2024-04-20T12:50:52.035 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:52.049 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:52.049 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:52.458 INFO:teuthology.orchestra.run.smithi121.stdout:generic/195 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:50:52.468 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/195 2024-04-20T12:50:52.468 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/195 2024-04-20T12:50:52.468 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:52.475 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:52.507 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:52.507 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/196) 2024-04-20T12:50:53.027 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:53.039 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:53.039 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:53.457 INFO:teuthology.orchestra.run.smithi121.stdout:generic/196 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:50:53.468 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/196 2024-04-20T12:50:53.468 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/196 2024-04-20T12:50:53.468 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:53.477 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:53.506 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:53.506 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/197) 2024-04-20T12:50:54.082 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:54.095 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:54.095 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:54.508 INFO:teuthology.orchestra.run.smithi121.stdout:generic/197 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:50:54.520 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/197 2024-04-20T12:50:54.520 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/197 2024-04-20T12:50:54.520 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:54.529 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:54.559 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:54.559 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/198) 2024-04-20T12:50:55.064 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:55.077 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:55.077 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:57.071 INFO:teuthology.orchestra.run.smithi121.stdout:generic/198 2s 2024-04-20T12:50:57.088 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/198 2024-04-20T12:50:57.088 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:57.097 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:57.128 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:57.128 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/199) 2024-04-20T12:50:57.847 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:57.859 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:57.859 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:58.279 INFO:teuthology.orchestra.run.smithi121.stdout:generic/199 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:50:58.289 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/199 2024-04-20T12:50:58.289 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/199 2024-04-20T12:50:58.289 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:58.298 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:58.323 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:58.323 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/200) 2024-04-20T12:50:58.862 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:58.875 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:58.875 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:59.298 INFO:teuthology.orchestra.run.smithi121.stdout:generic/200 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:50:59.310 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/200 2024-04-20T12:50:59.310 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/200 2024-04-20T12:50:59.310 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:50:59.319 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:50:59.349 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:50:59.349 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/201) 2024-04-20T12:50:59.878 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:50:59.890 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:50:59.890 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:51:00.319 INFO:teuthology.orchestra.run.smithi121.stdout:generic/201 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:51:00.332 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/201 2024-04-20T12:51:00.332 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/201 2024-04-20T12:51:00.332 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:51:00.340 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:51:00.367 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:51:00.367 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/202) 2024-04-20T12:51:00.893 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:51:00.906 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:51:00.906 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:51:01.352 INFO:teuthology.orchestra.run.smithi121.stdout:generic/202 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:51:01.366 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/202 2024-04-20T12:51:01.366 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/202 2024-04-20T12:51:01.367 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:51:01.379 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:51:01.418 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:51:01.418 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/203) 2024-04-20T12:51:01.977 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:51:01.989 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:51:01.989 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:51:02.390 INFO:teuthology.orchestra.run.smithi121.stdout:generic/203 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:51:02.403 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/203 2024-04-20T12:51:02.403 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/203 2024-04-20T12:51:02.404 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:51:02.415 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:51:02.452 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:51:02.452 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/204) 2024-04-20T12:51:03.000 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:51:03.014 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:51:03.014 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:51:03.899 INFO:teuthology.orchestra.run.smithi121.stdout:generic/204 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:51:03.914 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/204 2024-04-20T12:51:03.914 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/204 2024-04-20T12:51:03.914 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:51:03.926 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:51:03.959 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:51:03.960 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/205) 2024-04-20T12:51:04.095 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:51:04.102 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:51:04.426 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:51:04.438 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:51:04.438 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:51:04.837 INFO:teuthology.orchestra.run.smithi121.stdout:generic/205 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:51:04.848 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/205 2024-04-20T12:51:04.848 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/205 2024-04-20T12:51:04.848 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:51:04.857 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:51:04.881 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:51:04.881 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/206) 2024-04-20T12:51:05.369 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:51:05.380 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:51:05.380 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:51:05.729 INFO:teuthology.orchestra.run.smithi121.stdout:generic/206 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:51:05.738 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/206 2024-04-20T12:51:05.738 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/206 2024-04-20T12:51:05.738 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:51:05.745 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:51:05.771 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:51:05.771 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/207) 2024-04-20T12:51:06.274 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:51:06.287 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:51:06.287 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:51:34.259 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:51:34.268 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:51:53.957 INFO:teuthology.orchestra.run.smithi121.stdout:generic/207 47s 2024-04-20T12:51:53.974 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/207 2024-04-20T12:51:53.974 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:51:53.984 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:51:54.029 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:51:54.030 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/208) 2024-04-20T12:51:54.721 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:51:54.733 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:51:54.733 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:52:04.389 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:52:04.397 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:52:34.593 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:52:34.601 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:53:04.725 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:53:04.736 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:53:34.863 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:53:34.872 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:54:05.399 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:54:05.408 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:54:05.498 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-04-20T12:54:05.498+0000 7f5d77a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 33352) UID: 0 2024-04-20T12:54:05.498 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-04-20T12:54:05.498+0000 7f1143800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 33352) UID: 0 2024-04-20T12:54:05.499 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T12:54:05.498+0000 7fad6ce00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 33352) UID: 0 2024-04-20T12:54:05.499 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-04-20T12:54:05.498+0000 7f4dce800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 33352) UID: 0 2024-04-20T12:55:15.585 INFO:teuthology.orchestra.run.smithi121.stdout:generic/208 201s 2024-04-20T12:55:15.606 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/208 2024-04-20T12:55:15.606 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:55:15.618 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:55:15.673 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:55:15.674 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/209) 2024-04-20T12:55:16.336 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:55:16.347 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:55:16.347 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:55:39.151 INFO:teuthology.orchestra.run.smithi121.stdout:generic/209 [failed, exit status 1]- output mismatch (see /tmp/tmp.yONpfXTHudxfstests-dev/results//generic/209.out.bad) 2024-04-20T12:55:39.166 INFO:teuthology.orchestra.run.smithi121.stdout: --- tests/generic/209.out 2024-04-20 10:23:05.918112788 +0000 2024-04-20T12:55:39.166 INFO:teuthology.orchestra.run.smithi121.stdout: +++ /tmp/tmp.yONpfXTHudxfstests-dev/results//generic/209.out.bad 2024-04-20 12:55:39.101741953 +0000 2024-04-20T12:55:39.166 INFO:teuthology.orchestra.run.smithi121.stdout: @@ -1,2 +1,2 @@ 2024-04-20T12:55:39.166 INFO:teuthology.orchestra.run.smithi121.stdout: QA output created by 209 2024-04-20T12:55:39.166 INFO:teuthology.orchestra.run.smithi121.stdout: -test ran for 30 seconds without error 2024-04-20T12:55:39.167 INFO:teuthology.orchestra.run.smithi121.stdout: +reader found old byte at pos 4096 2024-04-20T12:55:39.167 INFO:teuthology.orchestra.run.smithi121.stdout: \ No newline at end of file 2024-04-20T12:55:39.178 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/209 2024-04-20T12:55:39.181 INFO:teuthology.orchestra.run.smithi121.stdout:Failures: generic/209 2024-04-20T12:55:39.181 INFO:teuthology.orchestra.run.smithi121.stdout:Failed 1 of 1 tests 2024-04-20T12:55:39.203 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:55:39.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T12:55:39.257 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-20T12:55:39.257 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-20T12:55:39.257 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/210) 2024-04-20T12:55:39.793 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:55:39.806 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:55:39.806 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:55:40.478 INFO:teuthology.orchestra.run.smithi121.stdout:generic/210 1s 2024-04-20T12:55:40.494 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/210 2024-04-20T12:55:40.494 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:55:40.504 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:55:40.555 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:55:40.556 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/211) 2024-04-20T12:55:41.232 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:55:41.244 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:55:41.244 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:55:42.239 INFO:teuthology.orchestra.run.smithi121.stdout:generic/211 1s 2024-04-20T12:55:42.255 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/211 2024-04-20T12:55:42.256 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:55:42.265 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:55:42.292 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:55:42.292 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/212) 2024-04-20T12:55:42.818 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:55:42.833 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:55:42.833 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:55:43.436 INFO:teuthology.orchestra.run.smithi121.stdout:generic/212 1s 2024-04-20T12:55:43.453 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/212 2024-04-20T12:55:43.453 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:55:43.461 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:55:43.488 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:55:43.489 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/213) 2024-04-20T12:55:44.033 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:55:44.046 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:55:44.046 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:55:44.859 INFO:teuthology.orchestra.run.smithi121.stdout:generic/213 - output mismatch (see /tmp/tmp.yONpfXTHudxfstests-dev/results//generic/213.out.bad) 2024-04-20T12:55:44.865 INFO:teuthology.orchestra.run.smithi121.stdout: --- tests/generic/213.out 2024-04-20 10:23:05.919112775 +0000 2024-04-20T12:55:44.865 INFO:teuthology.orchestra.run.smithi121.stdout: +++ /tmp/tmp.yONpfXTHudxfstests-dev/results//generic/213.out.bad 2024-04-20 12:55:44.856666827 +0000 2024-04-20T12:55:44.865 INFO:teuthology.orchestra.run.smithi121.stdout: @@ -1,4 +1,3 @@ 2024-04-20T12:55:44.865 INFO:teuthology.orchestra.run.smithi121.stdout: QA output created by 213 2024-04-20T12:55:44.865 INFO:teuthology.orchestra.run.smithi121.stdout: We should get: fallocate: No space left on device 2024-04-20T12:55:44.865 INFO:teuthology.orchestra.run.smithi121.stdout: Strangely, xfs_io sometimes says "Success" when something went wrong, FYI 2024-04-20T12:55:44.865 INFO:teuthology.orchestra.run.smithi121.stdout: -fallocate: No space left on device 2024-04-20T12:55:44.871 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/213 2024-04-20T12:55:44.871 INFO:teuthology.orchestra.run.smithi121.stdout:Failures: generic/213 2024-04-20T12:55:44.871 INFO:teuthology.orchestra.run.smithi121.stdout:Failed 1 of 1 tests 2024-04-20T12:55:44.878 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:55:44.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T12:55:44.932 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-20T12:55:44.932 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-20T12:55:44.933 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/214) 2024-04-20T12:55:45.566 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:55:45.578 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:55:45.579 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:55:46.597 INFO:teuthology.orchestra.run.smithi121.stdout:generic/214 1s 2024-04-20T12:55:46.614 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/214 2024-04-20T12:55:46.614 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:55:46.623 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:55:46.664 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:55:46.665 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/215) 2024-04-20T12:55:47.378 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:55:47.393 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:55:47.393 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:55:51.371 INFO:teuthology.orchestra.run.smithi121.stdout:generic/215 4s 2024-04-20T12:55:51.390 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/215 2024-04-20T12:55:51.390 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:55:51.400 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:55:51.428 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:55:51.429 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/216) 2024-04-20T12:55:51.984 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:55:51.998 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:55:51.999 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:55:52.378 INFO:teuthology.orchestra.run.smithi121.stdout:generic/216 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:55:52.389 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/216 2024-04-20T12:55:52.389 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/216 2024-04-20T12:55:52.389 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:55:52.396 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:55:52.427 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:55:52.428 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/217) 2024-04-20T12:55:52.985 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:55:53.000 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:55:53.000 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:55:53.373 INFO:teuthology.orchestra.run.smithi121.stdout:generic/217 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:55:53.383 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/217 2024-04-20T12:55:53.383 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/217 2024-04-20T12:55:53.384 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:55:53.392 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:55:53.425 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:55:53.425 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/218) 2024-04-20T12:55:53.980 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:55:53.995 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:55:53.995 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:55:54.384 INFO:teuthology.orchestra.run.smithi121.stdout:generic/218 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:55:54.398 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/218 2024-04-20T12:55:54.398 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/218 2024-04-20T12:55:54.398 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:55:54.406 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:55:54.438 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:55:54.439 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/219) 2024-04-20T12:55:55.003 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:55:55.020 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:55:55.021 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:55:55.402 INFO:teuthology.orchestra.run.smithi121.stdout:generic/219 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:55:55.414 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/219 2024-04-20T12:55:55.414 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/219 2024-04-20T12:55:55.414 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:55:55.422 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:55:55.449 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:55:55.449 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/220) 2024-04-20T12:55:56.010 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:55:56.027 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:55:56.028 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:55:56.464 INFO:teuthology.orchestra.run.smithi121.stdout:generic/220 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:55:56.475 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/220 2024-04-20T12:55:56.476 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/220 2024-04-20T12:55:56.476 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:55:56.488 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:55:56.517 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:55:56.517 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/221) 2024-04-20T12:55:57.079 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:55:57.096 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:55:57.096 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:55:58.610 INFO:teuthology.orchestra.run.smithi121.stdout:generic/221 1s 2024-04-20T12:55:58.624 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/221 2024-04-20T12:55:58.624 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:55:58.632 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:55:58.659 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:55:58.660 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/222) 2024-04-20T12:55:59.209 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:55:59.232 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:55:59.232 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:55:59.624 INFO:teuthology.orchestra.run.smithi121.stdout:generic/222 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:55:59.636 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/222 2024-04-20T12:55:59.636 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/222 2024-04-20T12:55:59.637 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:55:59.645 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:55:59.678 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:55:59.678 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/223) 2024-04-20T12:56:00.231 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:56:00.246 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:56:00.247 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:56:00.619 INFO:teuthology.orchestra.run.smithi121.stdout:generic/223 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:56:00.630 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/223 2024-04-20T12:56:00.630 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/223 2024-04-20T12:56:00.630 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:56:00.640 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:56:00.671 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:56:00.671 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/224) 2024-04-20T12:56:01.362 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:56:01.381 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:56:01.381 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:56:02.823 INFO:teuthology.orchestra.run.smithi121.stdout:generic/224 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:56:02.836 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/224 2024-04-20T12:56:02.836 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/224 2024-04-20T12:56:02.836 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:56:02.845 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:56:02.876 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:56:02.876 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/225) 2024-04-20T12:56:03.444 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:56:03.457 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:56:03.457 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:56:03.843 INFO:teuthology.orchestra.run.smithi121.stdout:generic/225 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:56:03.855 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/225 2024-04-20T12:56:03.855 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/225 2024-04-20T12:56:03.855 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:56:03.863 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:56:03.890 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:56:03.890 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/226) 2024-04-20T12:56:04.430 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:56:04.442 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:56:04.442 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:56:04.807 INFO:teuthology.orchestra.run.smithi121.stdout:generic/226 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:56:04.821 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/226 2024-04-20T12:56:04.821 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/226 2024-04-20T12:56:04.821 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:56:04.830 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:56:04.861 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:56:04.861 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/227) 2024-04-20T12:56:05.429 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:56:05.442 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:56:05.442 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:56:05.812 INFO:teuthology.orchestra.run.smithi121.stdout:generic/227 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:56:05.822 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/227 2024-04-20T12:56:05.822 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/227 2024-04-20T12:56:05.822 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:56:05.831 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:56:05.865 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:56:05.865 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/228) 2024-04-20T12:56:06.403 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:56:06.418 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:56:06.418 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:56:07.029 INFO:teuthology.orchestra.run.smithi121.stdout:generic/228 1s 2024-04-20T12:56:07.045 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/228 2024-04-20T12:56:07.045 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:56:07.054 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:56:07.118 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:56:07.118 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/229) 2024-04-20T12:56:07.831 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:56:07.847 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:56:07.847 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:56:08.215 INFO:teuthology.orchestra.run.smithi121.stdout:generic/229 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:56:08.228 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/229 2024-04-20T12:56:08.228 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/229 2024-04-20T12:56:08.228 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:56:08.235 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:56:08.259 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:56:08.259 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/230) 2024-04-20T12:56:08.801 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:56:08.817 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:56:08.817 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:56:09.184 INFO:teuthology.orchestra.run.smithi121.stdout:generic/230 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:56:09.194 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/230 2024-04-20T12:56:09.194 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/230 2024-04-20T12:56:09.194 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:56:09.203 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:56:09.237 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:56:09.238 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/231) 2024-04-20T12:56:09.817 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:56:09.832 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:56:09.832 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:56:10.322 INFO:teuthology.orchestra.run.smithi121.stdout:generic/231 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:56:10.331 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/231 2024-04-20T12:56:10.332 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/231 2024-04-20T12:56:10.332 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:56:10.343 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:56:10.373 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:56:10.373 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/232) 2024-04-20T12:56:10.929 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:56:10.941 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:56:10.941 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:56:11.332 INFO:teuthology.orchestra.run.smithi121.stdout:generic/232 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:56:11.343 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/232 2024-04-20T12:56:11.343 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/232 2024-04-20T12:56:11.344 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:56:11.352 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:56:11.383 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:56:11.383 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/233) 2024-04-20T12:56:11.938 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:56:11.949 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:56:11.950 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:56:12.348 INFO:teuthology.orchestra.run.smithi121.stdout:generic/233 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:56:12.358 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/233 2024-04-20T12:56:12.359 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/233 2024-04-20T12:56:12.359 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:56:12.367 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:56:12.395 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:56:12.396 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/234) 2024-04-20T12:56:12.931 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:56:12.948 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:56:12.949 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:56:13.318 INFO:teuthology.orchestra.run.smithi121.stdout:generic/234 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:56:13.328 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/234 2024-04-20T12:56:13.329 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/234 2024-04-20T12:56:13.329 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:56:13.337 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:56:13.367 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:56:13.367 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/235) 2024-04-20T12:56:13.937 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:56:13.951 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:56:13.951 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:56:14.330 INFO:teuthology.orchestra.run.smithi121.stdout:generic/235 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:56:14.340 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/235 2024-04-20T12:56:14.340 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/235 2024-04-20T12:56:14.341 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:56:14.349 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:56:14.379 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:56:14.379 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/236) 2024-04-20T12:56:14.929 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:56:14.940 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:56:14.941 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:56:16.501 INFO:teuthology.orchestra.run.smithi121.stdout:generic/236 2s 2024-04-20T12:56:16.518 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/236 2024-04-20T12:56:16.519 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:56:16.526 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:56:16.576 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:56:16.576 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/237) 2024-04-20T12:56:17.270 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:56:17.285 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:56:17.285 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:56:17.753 INFO:teuthology.orchestra.run.smithi121.stdout:generic/237 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-04-20T12:56:17.763 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/237 2024-04-20T12:56:17.764 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/237 2024-04-20T12:56:17.764 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:56:17.770 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:56:17.794 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:56:17.794 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/238) 2024-04-20T12:56:18.345 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:56:18.361 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:56:18.361 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:56:18.730 INFO:teuthology.orchestra.run.smithi121.stdout:generic/238 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:56:18.740 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/238 2024-04-20T12:56:18.740 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/238 2024-04-20T12:56:18.740 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:56:18.748 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:56:18.777 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:56:18.777 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/239) 2024-04-20T12:56:19.340 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:56:19.359 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:56:19.359 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:56:19.793 INFO:teuthology.orchestra.run.smithi121.stdout:generic/239 [not run] Sparse files are not supported or do not work as expected 2024-04-20T12:56:19.804 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/239 2024-04-20T12:56:19.804 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/239 2024-04-20T12:56:19.804 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:56:19.813 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:56:19.837 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:56:19.838 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/240) 2024-04-20T12:56:20.403 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:56:20.417 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:56:20.417 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:56:20.875 INFO:teuthology.orchestra.run.smithi121.stdout:generic/240 [not run] Sparse files are not supported or do not work as expected 2024-04-20T12:56:20.887 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/240 2024-04-20T12:56:20.888 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/240 2024-04-20T12:56:20.888 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:56:20.896 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:56:20.929 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:56:20.930 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/241) 2024-04-20T12:56:21.472 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:56:21.489 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:56:21.489 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:57:22.703 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:57:22.713 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T12:57:22.844 INFO:tasks.ceph.mds.a.smithi121.stderr:2024-04-20T12:57:22.844+0000 7f95c3c00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 231045) UID: 0 2024-04-20T12:57:22.844 INFO:tasks.ceph.mds.c.smithi121.stderr:2024-04-20T12:57:22.844+0000 7f4d9da00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 231045) UID: 0 2024-04-20T12:57:39.454 INFO:teuthology.orchestra.run.smithi121.stdout:generic/241 78s 2024-04-20T12:57:39.511 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/241 2024-04-20T12:57:39.511 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:57:39.522 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:57:39.564 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:57:39.564 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/242) 2024-04-20T12:57:41.232 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:57:41.250 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:57:41.250 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:57:41.764 INFO:teuthology.orchestra.run.smithi121.stdout:generic/242 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:57:41.778 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/242 2024-04-20T12:57:41.778 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/242 2024-04-20T12:57:41.778 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:57:41.790 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:57:41.824 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:57:41.824 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/243) 2024-04-20T12:57:42.396 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:57:42.408 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:57:42.408 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:57:42.937 INFO:teuthology.orchestra.run.smithi121.stdout:generic/243 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:57:42.946 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/243 2024-04-20T12:57:42.946 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/243 2024-04-20T12:57:42.946 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:57:42.952 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:57:42.974 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:57:42.974 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/244) 2024-04-20T12:57:43.546 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:57:43.558 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:57:43.558 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:57:44.974 INFO:teuthology.orchestra.run.smithi121.stdout:generic/244 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2024-04-20T12:57:44.988 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/244 2024-04-20T12:57:44.989 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/244 2024-04-20T12:57:44.989 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:57:45.000 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:57:45.028 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:57:45.029 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/245) 2024-04-20T12:57:45.586 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:57:45.604 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:57:45.604 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:57:46.359 INFO:teuthology.orchestra.run.smithi121.stdout:generic/245 1s 2024-04-20T12:57:46.381 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/245 2024-04-20T12:57:46.382 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:57:46.391 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:57:46.428 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:57:46.428 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/246) 2024-04-20T12:57:47.322 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:57:47.339 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:57:47.340 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:57:48.010 INFO:teuthology.orchestra.run.smithi121.stdout:generic/246 0s 2024-04-20T12:57:48.030 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/246 2024-04-20T12:57:48.030 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:57:48.038 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:57:48.076 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:57:48.076 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/247) 2024-04-20T12:57:48.998 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:57:49.010 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:57:49.010 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:58:41.732 INFO:teuthology.orchestra.run.smithi121.stdout:generic/247 52s 2024-04-20T12:58:41.754 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/247 2024-04-20T12:58:41.754 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:58:41.765 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:58:41.799 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:58:41.800 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/248) 2024-04-20T12:58:42.431 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:58:42.446 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:58:42.447 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:58:43.071 INFO:teuthology.orchestra.run.smithi121.stdout:generic/248 1s 2024-04-20T12:58:43.094 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/248 2024-04-20T12:58:43.095 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:58:43.106 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:58:43.142 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:58:43.142 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/249) 2024-04-20T12:58:43.784 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:58:43.801 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:58:43.801 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:58:48.227 INFO:teuthology.orchestra.run.smithi121.stdout:generic/249 5s 2024-04-20T12:58:48.248 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/249 2024-04-20T12:58:48.248 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:58:48.255 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:58:48.301 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:58:48.301 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/250) 2024-04-20T12:58:49.145 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:58:49.161 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:58:49.161 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:58:49.855 INFO:teuthology.orchestra.run.smithi121.stdout:generic/250 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:58:49.869 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/250 2024-04-20T12:58:49.869 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/250 2024-04-20T12:58:49.869 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:58:49.879 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:58:49.913 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:58:49.913 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/251) 2024-04-20T12:58:50.542 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:58:50.562 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:58:50.562 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:58:51.031 INFO:teuthology.orchestra.run.smithi121.stdout:generic/251 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:58:51.043 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/251 2024-04-20T12:58:51.043 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/251 2024-04-20T12:58:51.043 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:58:51.053 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:58:51.082 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:58:51.083 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/252) 2024-04-20T12:58:51.728 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:58:51.745 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:58:51.745 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:58:52.785 INFO:teuthology.orchestra.run.smithi121.stdout:generic/252 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:58:52.800 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/252 2024-04-20T12:58:52.801 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/252 2024-04-20T12:58:52.801 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:58:52.812 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:58:52.846 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:58:52.847 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/253) 2024-04-20T12:58:53.353 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:58:53.371 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:58:53.371 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:58:53.971 INFO:teuthology.orchestra.run.smithi121.stdout:generic/253 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:58:53.982 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/253 2024-04-20T12:58:53.982 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/253 2024-04-20T12:58:53.982 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:58:53.991 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:58:54.023 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:58:54.024 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/254) 2024-04-20T12:58:54.650 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:58:54.669 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:58:54.670 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:58:55.346 INFO:teuthology.orchestra.run.smithi121.stdout:generic/254 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:58:55.359 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/254 2024-04-20T12:58:55.359 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/254 2024-04-20T12:58:55.360 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:58:55.372 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:58:55.404 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:58:55.404 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/255) 2024-04-20T12:58:55.996 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:58:56.008 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:58:56.008 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:58:59.589 INFO:teuthology.orchestra.run.smithi121.stdout:generic/255 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-20T12:58:59.600 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/255 2024-04-20T12:58:59.600 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/255 2024-04-20T12:58:59.600 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:58:59.612 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:58:59.654 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:58:59.655 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/256) 2024-04-20T12:59:00.477 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:59:00.494 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:59:00.494 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:59:03.547 INFO:teuthology.orchestra.run.smithi121.stdout:generic/256 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:59:03.561 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/256 2024-04-20T12:59:03.561 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/256 2024-04-20T12:59:03.561 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:59:03.570 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:59:03.605 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:59:03.605 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/257) 2024-04-20T12:59:04.213 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:59:04.230 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:59:04.230 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:59:07.319 INFO:teuthology.orchestra.run.smithi121.stdout:generic/257 3s 2024-04-20T12:59:07.340 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/257 2024-04-20T12:59:07.340 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:59:07.348 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:59:07.392 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:59:07.392 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/258) 2024-04-20T12:59:08.193 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:59:08.206 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:59:08.207 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:59:08.771 INFO:teuthology.orchestra.run.smithi121.stdout:generic/258 [failed, exit status 1]- output mismatch (see /tmp/tmp.yONpfXTHudxfstests-dev/results//generic/258.out.bad) 2024-04-20T12:59:08.781 INFO:teuthology.orchestra.run.smithi121.stdout: --- tests/generic/258.out 2024-04-20 10:23:05.926112684 +0000 2024-04-20T12:59:08.781 INFO:teuthology.orchestra.run.smithi121.stdout: +++ /tmp/tmp.yONpfXTHudxfstests-dev/results//generic/258.out.bad 2024-04-20 12:59:08.767004933 +0000 2024-04-20T12:59:08.781 INFO:teuthology.orchestra.run.smithi121.stdout: @@ -1,5 +1,6 @@ 2024-04-20T12:59:08.781 INFO:teuthology.orchestra.run.smithi121.stdout: QA output created by 258 2024-04-20T12:59:08.781 INFO:teuthology.orchestra.run.smithi121.stdout: Creating file with timestamp of Jan 1, 1960 2024-04-20T12:59:08.781 INFO:teuthology.orchestra.run.smithi121.stdout: Testing for negative seconds since epoch 2024-04-20T12:59:08.781 INFO:teuthology.orchestra.run.smithi121.stdout: -Remounting to flush cache 2024-04-20T12:59:08.781 INFO:teuthology.orchestra.run.smithi121.stdout: -Testing for negative seconds since epoch 2024-04-20T12:59:08.781 INFO:teuthology.orchestra.run.smithi121.stdout: +Timestamp wrapped: 3979351756 2024-04-20T12:59:08.781 INFO:teuthology.orchestra.run.smithi121.stdout: +Timestamp wrapped 2024-04-20T12:59:08.781 INFO:teuthology.orchestra.run.smithi121.stdout: +(see /tmp/tmp.yONpfXTHudxfstests-dev/results//generic/258.full for details) 2024-04-20T12:59:08.790 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/258 2024-04-20T12:59:08.791 INFO:teuthology.orchestra.run.smithi121.stdout:Failures: generic/258 2024-04-20T12:59:08.791 INFO:teuthology.orchestra.run.smithi121.stdout:Failed 1 of 1 tests 2024-04-20T12:59:08.802 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:59:08.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T12:59:08.837 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-20T12:59:08.837 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-20T12:59:08.837 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/259) 2024-04-20T12:59:09.442 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:59:09.454 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:59:09.454 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:59:09.869 INFO:teuthology.orchestra.run.smithi121.stdout:generic/259 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:59:09.882 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/259 2024-04-20T12:59:09.882 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/259 2024-04-20T12:59:09.883 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:59:09.893 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:59:09.923 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:59:09.923 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/260) 2024-04-20T12:59:10.489 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:59:10.504 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:59:10.504 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:59:10.949 INFO:teuthology.orchestra.run.smithi121.stdout:generic/260 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:59:10.962 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/260 2024-04-20T12:59:10.962 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/260 2024-04-20T12:59:10.963 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:59:10.972 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:59:11.005 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:59:11.005 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/261) 2024-04-20T12:59:11.622 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:59:11.639 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:59:11.639 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:59:12.066 INFO:teuthology.orchestra.run.smithi121.stdout:generic/261 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:59:12.079 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/261 2024-04-20T12:59:12.079 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/261 2024-04-20T12:59:12.079 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:59:12.094 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:59:12.127 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:59:12.127 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/262) 2024-04-20T12:59:12.702 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:59:12.717 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:59:12.717 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:59:13.143 INFO:teuthology.orchestra.run.smithi121.stdout:generic/262 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T12:59:13.155 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/262 2024-04-20T12:59:13.156 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/262 2024-04-20T12:59:13.156 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T12:59:13.165 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T12:59:13.202 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T12:59:13.202 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/263) 2024-04-20T12:59:13.798 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T12:59:13.815 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T12:59:13.815 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:10.467 INFO:teuthology.orchestra.run.smithi121.stdout:generic/263 117s 2024-04-20T13:01:10.487 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/263 2024-04-20T13:01:10.488 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:01:10.496 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:10.536 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:01:10.536 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/264) 2024-04-20T13:01:11.219 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:01:11.231 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:01:11.231 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:11.653 INFO:teuthology.orchestra.run.smithi121.stdout:generic/264 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:01:11.667 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/264 2024-04-20T13:01:11.667 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/264 2024-04-20T13:01:11.667 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:01:11.679 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:11.713 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:01:11.713 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/265) 2024-04-20T13:01:12.251 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:01:12.265 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:01:12.266 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:12.795 INFO:teuthology.orchestra.run.smithi121.stdout:generic/265 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:01:12.814 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/265 2024-04-20T13:01:12.814 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/265 2024-04-20T13:01:12.814 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:01:12.817 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:12.851 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:01:12.852 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/266) 2024-04-20T13:01:13.384 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:01:13.397 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:01:13.397 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:13.889 INFO:teuthology.orchestra.run.smithi121.stdout:generic/266 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:01:13.901 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/266 2024-04-20T13:01:13.902 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/266 2024-04-20T13:01:13.902 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:01:13.911 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:13.942 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:01:13.943 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/267) 2024-04-20T13:01:14.479 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:01:14.492 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:01:14.492 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:14.975 INFO:teuthology.orchestra.run.smithi121.stdout:generic/267 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:01:14.988 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/267 2024-04-20T13:01:14.989 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/267 2024-04-20T13:01:14.989 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:01:14.999 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:15.029 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:01:15.029 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/268) 2024-04-20T13:01:15.563 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:01:15.576 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:01:15.577 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:16.035 INFO:teuthology.orchestra.run.smithi121.stdout:generic/268 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:01:16.045 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/268 2024-04-20T13:01:16.045 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/268 2024-04-20T13:01:16.045 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:01:16.054 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:16.085 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:01:16.085 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/269) 2024-04-20T13:01:16.633 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:01:16.647 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:01:16.647 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:17.068 INFO:teuthology.orchestra.run.smithi121.stdout:generic/269 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:01:17.080 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/269 2024-04-20T13:01:17.080 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/269 2024-04-20T13:01:17.080 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:01:17.089 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:17.119 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:01:17.119 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/270) 2024-04-20T13:01:17.653 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:01:17.665 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:01:17.666 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:18.087 INFO:teuthology.orchestra.run.smithi121.stdout:generic/270 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2024-04-20T13:01:18.101 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/270 2024-04-20T13:01:18.101 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/270 2024-04-20T13:01:18.101 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:01:18.112 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:18.145 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:01:18.145 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/271) 2024-04-20T13:01:18.677 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:01:18.691 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:01:18.691 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:19.160 INFO:teuthology.orchestra.run.smithi121.stdout:generic/271 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:01:19.174 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/271 2024-04-20T13:01:19.174 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/271 2024-04-20T13:01:19.174 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:01:19.185 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:19.216 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:01:19.216 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/272) 2024-04-20T13:01:19.778 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:01:19.792 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:01:19.792 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:20.234 INFO:teuthology.orchestra.run.smithi121.stdout:generic/272 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:01:20.248 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/272 2024-04-20T13:01:20.248 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/272 2024-04-20T13:01:20.249 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:01:20.260 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:20.292 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:01:20.292 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/273) 2024-04-20T13:01:20.824 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:01:20.841 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:01:20.841 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:21.284 INFO:teuthology.orchestra.run.smithi121.stdout:generic/273 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:01:21.298 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/273 2024-04-20T13:01:21.298 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/273 2024-04-20T13:01:21.298 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:01:21.311 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:21.343 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:01:21.344 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/274) 2024-04-20T13:01:21.873 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:01:21.885 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:01:21.885 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:22.303 INFO:teuthology.orchestra.run.smithi121.stdout:generic/274 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:01:22.313 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/274 2024-04-20T13:01:22.313 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/274 2024-04-20T13:01:22.313 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:01:22.321 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:22.350 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:01:22.350 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/275) 2024-04-20T13:01:22.873 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:01:22.886 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:01:22.886 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:23.306 INFO:teuthology.orchestra.run.smithi121.stdout:generic/275 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:01:23.318 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/275 2024-04-20T13:01:23.318 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/275 2024-04-20T13:01:23.318 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:01:23.326 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:23.357 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:01:23.357 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/276) 2024-04-20T13:01:23.870 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:01:23.882 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:01:23.882 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:24.323 INFO:teuthology.orchestra.run.smithi121.stdout:generic/276 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:01:24.334 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/276 2024-04-20T13:01:24.334 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/276 2024-04-20T13:01:24.335 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:01:24.343 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:24.373 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:01:24.373 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/277) 2024-04-20T13:01:24.902 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:01:24.916 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:01:24.916 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:25.343 INFO:teuthology.orchestra.run.smithi121.stdout:generic/277 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:01:25.357 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/277 2024-04-20T13:01:25.357 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/277 2024-04-20T13:01:25.357 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:01:25.369 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:25.399 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:01:25.399 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/278) 2024-04-20T13:01:25.592 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:01:25.597 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:01:25.798 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-04-20T13:01:25.798+0000 7f4dce800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 33507) UID: 0 2024-04-20T13:01:25.799 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-04-20T13:01:25.798+0000 7f5d77a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 33507) UID: 0 2024-04-20T13:01:25.799 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-04-20T13:01:25.798+0000 7f1143800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 33507) UID: 0 2024-04-20T13:01:25.799 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T13:01:25.798+0000 7fad6ce00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 33507) UID: 0 2024-04-20T13:01:25.874 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:01:25.886 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:01:25.886 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:26.360 INFO:teuthology.orchestra.run.smithi121.stdout:generic/278 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:01:26.371 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/278 2024-04-20T13:01:26.371 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/278 2024-04-20T13:01:26.371 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:01:26.380 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:26.410 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:01:26.410 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/279) 2024-04-20T13:01:26.899 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:01:26.911 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:01:26.911 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:27.358 INFO:teuthology.orchestra.run.smithi121.stdout:generic/279 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:01:27.372 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/279 2024-04-20T13:01:27.372 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/279 2024-04-20T13:01:27.372 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:01:27.383 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:27.412 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:01:27.413 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/280) 2024-04-20T13:01:27.887 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:01:27.899 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:01:27.899 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:28.353 INFO:teuthology.orchestra.run.smithi121.stdout:generic/280 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:01:28.365 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/280 2024-04-20T13:01:28.365 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/280 2024-04-20T13:01:28.365 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:01:28.373 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:28.400 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:01:28.400 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/281) 2024-04-20T13:01:28.900 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:01:28.914 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:01:28.914 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:29.382 INFO:teuthology.orchestra.run.smithi121.stdout:generic/281 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:01:29.396 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/281 2024-04-20T13:01:29.397 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/281 2024-04-20T13:01:29.397 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:01:29.408 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:29.440 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:01:29.440 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/282) 2024-04-20T13:01:29.947 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:01:29.960 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:01:29.960 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:30.400 INFO:teuthology.orchestra.run.smithi121.stdout:generic/282 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:01:30.412 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/282 2024-04-20T13:01:30.413 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/282 2024-04-20T13:01:30.413 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:01:30.424 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:30.458 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:01:30.459 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/283) 2024-04-20T13:01:31.002 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:01:31.016 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:01:31.016 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:31.449 INFO:teuthology.orchestra.run.smithi121.stdout:generic/283 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:01:31.462 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/283 2024-04-20T13:01:31.462 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/283 2024-04-20T13:01:31.462 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:01:31.472 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:31.505 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:01:31.505 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/284) 2024-04-20T13:01:32.059 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:01:32.073 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:01:32.073 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:32.486 INFO:teuthology.orchestra.run.smithi121.stdout:generic/284 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:01:32.496 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/284 2024-04-20T13:01:32.496 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/284 2024-04-20T13:01:32.497 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:01:32.505 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:32.536 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:01:32.536 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/285) 2024-04-20T13:01:33.063 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:01:33.076 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:01:33.077 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:38.627 INFO:teuthology.orchestra.run.smithi121.stdout:generic/285 5s 2024-04-20T13:01:38.643 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/285 2024-04-20T13:01:38.643 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:01:38.652 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:01:38.682 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:01:38.682 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/286) 2024-04-20T13:01:39.407 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:01:39.420 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:01:39.420 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:32.725 INFO:teuthology.orchestra.run.smithi121.stdout:generic/286 173s 2024-04-20T13:04:32.744 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/286 2024-04-20T13:04:32.744 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:04:32.756 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:32.793 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:04:32.793 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/287) 2024-04-20T13:04:33.542 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:04:33.558 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:04:33.558 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:33.870 INFO:teuthology.orchestra.run.smithi121.stdout:generic/287 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:04:33.880 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/287 2024-04-20T13:04:33.880 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/287 2024-04-20T13:04:33.880 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:04:33.887 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:33.915 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:04:33.915 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/288) 2024-04-20T13:04:34.380 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:04:34.392 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:04:34.392 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:34.811 INFO:teuthology.orchestra.run.smithi121.stdout:generic/288 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:04:34.825 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/288 2024-04-20T13:04:34.826 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/288 2024-04-20T13:04:34.826 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:04:34.838 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:34.870 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:04:34.870 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/289) 2024-04-20T13:04:35.404 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:04:35.417 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:04:35.417 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:35.862 INFO:teuthology.orchestra.run.smithi121.stdout:generic/289 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:04:35.873 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/289 2024-04-20T13:04:35.873 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/289 2024-04-20T13:04:35.873 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:04:35.881 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:35.910 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:04:35.910 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/290) 2024-04-20T13:04:36.468 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:04:36.480 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:04:36.481 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:36.921 INFO:teuthology.orchestra.run.smithi121.stdout:generic/290 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:04:36.935 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/290 2024-04-20T13:04:36.936 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/290 2024-04-20T13:04:36.936 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:04:36.947 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:36.979 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:04:36.979 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/291) 2024-04-20T13:04:37.539 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:04:37.553 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:04:37.553 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:37.971 INFO:teuthology.orchestra.run.smithi121.stdout:generic/291 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:04:37.985 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/291 2024-04-20T13:04:37.985 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/291 2024-04-20T13:04:37.985 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:04:37.996 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:38.030 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:04:38.030 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/292) 2024-04-20T13:04:38.562 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:04:38.576 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:04:38.576 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:38.988 INFO:teuthology.orchestra.run.smithi121.stdout:generic/292 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:04:38.998 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/292 2024-04-20T13:04:38.999 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/292 2024-04-20T13:04:38.999 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:04:39.009 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:39.041 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:04:39.041 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/293) 2024-04-20T13:04:39.583 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:04:39.596 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:04:39.597 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:39.998 INFO:teuthology.orchestra.run.smithi121.stdout:generic/293 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:04:40.011 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/293 2024-04-20T13:04:40.011 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/293 2024-04-20T13:04:40.012 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:04:40.021 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:40.051 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:04:40.052 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/294) 2024-04-20T13:04:40.575 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:04:40.589 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:04:40.589 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:40.994 INFO:teuthology.orchestra.run.smithi121.stdout:generic/294 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:04:41.006 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/294 2024-04-20T13:04:41.006 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/294 2024-04-20T13:04:41.006 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:04:41.016 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:41.044 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:04:41.044 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/295) 2024-04-20T13:04:41.590 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:04:41.603 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:04:41.603 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:42.015 INFO:teuthology.orchestra.run.smithi121.stdout:generic/295 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:04:42.030 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/295 2024-04-20T13:04:42.030 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/295 2024-04-20T13:04:42.031 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:04:42.042 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:42.074 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:04:42.075 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/296) 2024-04-20T13:04:42.587 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:04:42.600 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:04:42.600 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:43.021 INFO:teuthology.orchestra.run.smithi121.stdout:generic/296 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:04:43.031 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/296 2024-04-20T13:04:43.031 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/296 2024-04-20T13:04:43.031 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:04:43.040 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:43.068 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:04:43.068 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/297) 2024-04-20T13:04:43.592 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:04:43.605 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:04:43.606 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:44.055 INFO:teuthology.orchestra.run.smithi121.stdout:generic/297 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:04:44.065 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/297 2024-04-20T13:04:44.065 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/297 2024-04-20T13:04:44.065 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:04:44.073 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:44.101 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:04:44.102 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/298) 2024-04-20T13:04:44.654 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:04:44.667 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:04:44.667 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:45.133 INFO:teuthology.orchestra.run.smithi121.stdout:generic/298 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:04:45.146 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/298 2024-04-20T13:04:45.147 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/298 2024-04-20T13:04:45.147 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:04:45.158 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:45.189 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:04:45.189 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/299) 2024-04-20T13:04:45.747 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:04:45.760 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:04:45.760 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:46.166 INFO:teuthology.orchestra.run.smithi121.stdout:generic/299 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:04:46.178 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/299 2024-04-20T13:04:46.178 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/299 2024-04-20T13:04:46.178 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:04:46.187 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:46.215 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:04:46.215 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/300) 2024-04-20T13:04:46.722 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:04:46.736 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:04:46.736 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:47.181 INFO:teuthology.orchestra.run.smithi121.stdout:generic/300 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:04:47.192 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/300 2024-04-20T13:04:47.192 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/300 2024-04-20T13:04:47.192 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:04:47.201 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:47.234 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:04:47.234 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/301) 2024-04-20T13:04:47.770 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:04:47.783 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:04:47.783 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:48.163 INFO:teuthology.orchestra.run.smithi121.stdout:generic/301 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:04:48.177 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/301 2024-04-20T13:04:48.177 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/301 2024-04-20T13:04:48.177 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:04:48.188 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:48.220 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:04:48.220 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/302) 2024-04-20T13:04:48.791 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:04:48.804 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:04:48.804 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:49.209 INFO:teuthology.orchestra.run.smithi121.stdout:generic/302 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:04:49.223 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/302 2024-04-20T13:04:49.223 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/302 2024-04-20T13:04:49.224 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:04:49.236 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:49.267 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:04:49.267 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/303) 2024-04-20T13:04:49.783 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:04:49.796 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:04:49.796 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:50.375 INFO:teuthology.orchestra.run.smithi121.stdout:generic/303 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-20T13:04:50.387 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/303 2024-04-20T13:04:50.387 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/303 2024-04-20T13:04:50.388 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:04:50.397 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:50.462 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:04:50.462 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/304) 2024-04-20T13:04:51.134 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:04:51.147 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:04:51.147 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:51.668 INFO:teuthology.orchestra.run.smithi121.stdout:generic/304 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2024-04-20T13:04:51.678 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/304 2024-04-20T13:04:51.678 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/304 2024-04-20T13:04:51.678 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:04:51.686 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:51.743 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:04:51.744 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/305) 2024-04-20T13:04:52.382 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:04:52.393 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:04:52.393 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:52.783 INFO:teuthology.orchestra.run.smithi121.stdout:generic/305 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:04:52.797 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/305 2024-04-20T13:04:52.798 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/305 2024-04-20T13:04:52.798 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:04:52.809 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:52.839 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:04:52.840 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/306) 2024-04-20T13:04:53.367 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:04:53.379 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:04:53.379 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:53.791 INFO:teuthology.orchestra.run.smithi121.stdout:generic/306 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:04:53.802 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/306 2024-04-20T13:04:53.802 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/306 2024-04-20T13:04:53.803 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:04:53.811 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:53.839 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:04:53.840 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/307) 2024-04-20T13:04:54.367 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:04:54.382 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:04:54.382 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:54.843 INFO:teuthology.orchestra.run.smithi121.stdout:generic/307 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:04:54.855 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/307 2024-04-20T13:04:54.855 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/307 2024-04-20T13:04:54.856 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:04:54.865 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:54.895 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:04:54.895 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/308) 2024-04-20T13:04:55.436 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:04:55.450 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:04:55.450 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:55.972 INFO:teuthology.orchestra.run.smithi121.stdout:generic/308 0s 2024-04-20T13:04:55.989 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/308 2024-04-20T13:04:55.989 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:04:55.997 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:56.026 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:04:56.026 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/309) 2024-04-20T13:04:56.538 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:04:56.552 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:04:56.552 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:58.108 INFO:teuthology.orchestra.run.smithi121.stdout:generic/309 2s 2024-04-20T13:04:58.124 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/309 2024-04-20T13:04:58.124 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:04:58.131 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:04:58.176 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:04:58.176 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/310) 2024-04-20T13:04:58.828 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:04:58.839 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:04:58.839 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:06:32.551 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:06:32.561 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:06:56.612 INFO:teuthology.orchestra.run.smithi121.stdout:generic/310 118s 2024-04-20T13:06:56.633 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/310 2024-04-20T13:06:56.633 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:06:56.644 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:06:56.699 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:06:56.699 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/311) 2024-04-20T13:06:57.541 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:06:57.555 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:06:57.555 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:06:57.868 INFO:teuthology.orchestra.run.smithi121.stdout:generic/311 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:06:57.878 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/311 2024-04-20T13:06:57.879 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/311 2024-04-20T13:06:57.879 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:06:57.886 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:06:57.916 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:06:57.916 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/312) 2024-04-20T13:06:58.430 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:06:58.442 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:06:58.442 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:06:59.039 INFO:teuthology.orchestra.run.smithi121.stdout:generic/312 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:06:59.053 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/312 2024-04-20T13:06:59.053 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/312 2024-04-20T13:06:59.053 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:06:59.061 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:06:59.089 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:06:59.089 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/313) 2024-04-20T13:06:59.584 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:06:59.596 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:06:59.596 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:07:02.688 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:07:02.697 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:07:05.287 INFO:teuthology.orchestra.run.smithi121.stdout:generic/313 6s 2024-04-20T13:07:05.304 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/313 2024-04-20T13:07:05.304 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:07:05.313 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:07:05.343 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:07:05.343 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/314) 2024-04-20T13:07:05.864 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:07:05.879 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:07:05.879 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:07:07.193 INFO:teuthology.orchestra.run.smithi121.stdout:generic/314 2s 2024-04-20T13:07:07.213 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/314 2024-04-20T13:07:07.213 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:07:07.224 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:07:07.256 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:07:07.256 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/315) 2024-04-20T13:07:07.789 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:07:07.802 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:07:07.802 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:07:08.890 INFO:teuthology.orchestra.run.smithi121.stdout:generic/315 1s 2024-04-20T13:07:08.906 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/315 2024-04-20T13:07:08.907 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:07:08.917 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:07:08.965 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:07:08.965 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/316) 2024-04-20T13:07:09.629 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:07:09.641 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:07:09.641 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:07:13.074 INFO:teuthology.orchestra.run.smithi121.stdout:generic/316 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-20T13:07:13.086 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/316 2024-04-20T13:07:13.086 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/316 2024-04-20T13:07:13.086 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:07:13.095 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:07:13.144 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:07:13.145 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/317) 2024-04-20T13:07:13.821 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:07:13.832 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:07:13.832 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:07:14.229 INFO:teuthology.orchestra.run.smithi121.stdout:generic/317 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:07:14.241 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/317 2024-04-20T13:07:14.241 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/317 2024-04-20T13:07:14.241 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:07:14.250 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:07:14.278 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:07:14.279 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/318) 2024-04-20T13:07:14.803 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:07:14.816 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:07:14.816 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:07:15.237 INFO:teuthology.orchestra.run.smithi121.stdout:generic/318 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:07:15.251 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/318 2024-04-20T13:07:15.251 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/318 2024-04-20T13:07:15.251 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:07:15.262 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:07:15.295 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:07:15.295 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/319) 2024-04-20T13:07:15.825 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:07:15.837 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:07:15.837 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:07:16.307 INFO:teuthology.orchestra.run.smithi121.stdout:generic/319 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-04-20T13:07:16.322 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/319 2024-04-20T13:07:16.322 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/319 2024-04-20T13:07:16.322 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:07:16.334 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:07:16.370 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:07:16.370 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/320) 2024-04-20T13:07:16.928 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:07:16.942 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:07:16.942 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:07:17.352 INFO:teuthology.orchestra.run.smithi121.stdout:generic/320 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:07:17.363 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/320 2024-04-20T13:07:17.363 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/320 2024-04-20T13:07:17.363 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:07:17.372 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:07:17.399 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:07:17.400 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/321) 2024-04-20T13:07:17.937 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:07:17.949 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:07:17.950 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:07:18.392 INFO:teuthology.orchestra.run.smithi121.stdout:generic/321 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:07:18.403 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/321 2024-04-20T13:07:18.404 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/321 2024-04-20T13:07:18.404 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:07:18.412 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:07:18.439 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:07:18.439 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/322) 2024-04-20T13:07:18.969 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:07:18.981 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:07:18.981 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:07:19.442 INFO:teuthology.orchestra.run.smithi121.stdout:generic/322 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:07:19.454 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/322 2024-04-20T13:07:19.454 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/322 2024-04-20T13:07:19.454 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:07:19.464 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:07:19.496 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:07:19.497 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/323) 2024-04-20T13:07:20.049 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:07:20.063 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:07:20.063 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:07:32.866 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:07:32.871 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:08:03.061 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:08:03.071 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:08:33.235 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:08:33.245 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:09:04.197 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:09:04.209 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:09:04.270 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-04-20T13:09:04.268+0000 7fb3ce400640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 300332) UID: 0 2024-04-20T13:09:04.270 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T13:09:04.268+0000 7f9c11a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 300332) UID: 0 2024-04-20T13:09:04.271 INFO:tasks.ceph.osd.2.smithi121.stderr:2024-04-20T13:09:04.268+0000 7f11dfa00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 300332) UID: 0 2024-04-20T13:09:04.271 INFO:tasks.ceph.osd.3.smithi121.stderr:2024-04-20T13:09:04.268+0000 7fca40a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 300332) UID: 0 2024-04-20T13:09:22.394 INFO:teuthology.orchestra.run.smithi121.stdout:generic/323 122s 2024-04-20T13:09:22.412 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/323 2024-04-20T13:09:22.412 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:09:22.420 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:22.484 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:09:22.484 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/324) 2024-04-20T13:09:23.397 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:09:23.409 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:09:23.409 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:23.802 INFO:teuthology.orchestra.run.smithi121.stdout:generic/324 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:09:23.812 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/324 2024-04-20T13:09:23.812 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/324 2024-04-20T13:09:23.812 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:09:23.820 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:23.846 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:09:23.846 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/325) 2024-04-20T13:09:24.320 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:09:24.332 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:09:24.332 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:24.789 INFO:teuthology.orchestra.run.smithi121.stdout:generic/325 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:09:24.800 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/325 2024-04-20T13:09:24.800 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/325 2024-04-20T13:09:24.800 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:09:24.809 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:24.841 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:09:24.841 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/326) 2024-04-20T13:09:25.323 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:09:25.335 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:09:25.335 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:25.704 INFO:teuthology.orchestra.run.smithi121.stdout:generic/326 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:09:25.713 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/326 2024-04-20T13:09:25.713 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/326 2024-04-20T13:09:25.713 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:09:25.721 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:25.746 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:09:25.746 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/327) 2024-04-20T13:09:26.226 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:09:26.237 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:09:26.237 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:26.587 INFO:teuthology.orchestra.run.smithi121.stdout:generic/327 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:09:26.599 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/327 2024-04-20T13:09:26.600 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/327 2024-04-20T13:09:26.600 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:09:26.609 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:26.636 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:09:26.636 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/328) 2024-04-20T13:09:27.097 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:09:27.110 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:09:27.111 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:27.507 INFO:teuthology.orchestra.run.smithi121.stdout:generic/328 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:09:27.520 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/328 2024-04-20T13:09:27.521 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/328 2024-04-20T13:09:27.521 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:09:27.532 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:27.559 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:09:27.559 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/329) 2024-04-20T13:09:28.031 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:09:28.043 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:09:28.044 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:28.458 INFO:teuthology.orchestra.run.smithi121.stdout:generic/329 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:09:28.468 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/329 2024-04-20T13:09:28.468 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/329 2024-04-20T13:09:28.468 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:09:28.476 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:28.503 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:09:28.503 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/330) 2024-04-20T13:09:28.980 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:09:29.274 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:09:29.275 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:29.372 INFO:teuthology.orchestra.run.smithi121.stdout:generic/330 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:09:29.381 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/330 2024-04-20T13:09:29.381 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/330 2024-04-20T13:09:29.381 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:09:29.388 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:29.412 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:09:29.412 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/331) 2024-04-20T13:09:29.903 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:09:29.915 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:09:29.916 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:30.321 INFO:teuthology.orchestra.run.smithi121.stdout:generic/331 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:09:30.333 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/331 2024-04-20T13:09:30.333 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/331 2024-04-20T13:09:30.333 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:09:30.343 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:30.369 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:09:30.369 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/332) 2024-04-20T13:09:30.840 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:09:30.852 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:09:30.852 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:31.260 INFO:teuthology.orchestra.run.smithi121.stdout:generic/332 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:09:31.274 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/332 2024-04-20T13:09:31.274 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/332 2024-04-20T13:09:31.274 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:09:31.285 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:31.317 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:09:31.317 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/333) 2024-04-20T13:09:31.786 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:09:31.799 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:09:31.800 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:32.165 INFO:teuthology.orchestra.run.smithi121.stdout:generic/333 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:09:32.175 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/333 2024-04-20T13:09:32.175 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/333 2024-04-20T13:09:32.175 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:09:32.185 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:32.215 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:09:32.216 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/334) 2024-04-20T13:09:34.179 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:09:34.190 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:09:34.190 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:34.559 INFO:teuthology.orchestra.run.smithi121.stdout:generic/334 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:09:34.566 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/334 2024-04-20T13:09:34.566 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/334 2024-04-20T13:09:34.566 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:09:34.573 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:34.598 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:09:34.598 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/335) 2024-04-20T13:09:35.066 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:09:35.083 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:09:35.083 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:35.448 INFO:teuthology.orchestra.run.smithi121.stdout:generic/335 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:09:35.458 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/335 2024-04-20T13:09:35.459 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/335 2024-04-20T13:09:35.459 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:09:35.467 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:35.497 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:09:35.497 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/336) 2024-04-20T13:09:35.976 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:09:35.987 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:09:35.987 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:36.384 INFO:teuthology.orchestra.run.smithi121.stdout:generic/336 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:09:36.393 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/336 2024-04-20T13:09:36.394 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/336 2024-04-20T13:09:36.394 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:09:36.401 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:36.425 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:09:36.426 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/337) 2024-04-20T13:09:36.882 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:09:36.895 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:09:36.895 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:37.261 INFO:teuthology.orchestra.run.smithi121.stdout:generic/337 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:09:37.271 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/337 2024-04-20T13:09:37.271 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/337 2024-04-20T13:09:37.271 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:09:37.279 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:37.304 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:09:37.304 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/338) 2024-04-20T13:09:37.754 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:09:37.765 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:09:37.765 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:38.146 INFO:teuthology.orchestra.run.smithi121.stdout:generic/338 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:09:38.157 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/338 2024-04-20T13:09:38.157 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/338 2024-04-20T13:09:38.157 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:09:38.166 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:38.195 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:09:38.195 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/339) 2024-04-20T13:09:38.668 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:09:38.681 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:09:38.682 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:39.074 INFO:teuthology.orchestra.run.smithi121.stdout:generic/339 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:09:39.086 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/339 2024-04-20T13:09:39.086 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/339 2024-04-20T13:09:39.086 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:09:39.097 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:39.125 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:09:39.125 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/340) 2024-04-20T13:09:39.627 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:09:39.638 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:09:39.638 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:40.780 INFO:teuthology.orchestra.run.smithi121.stdout:generic/340 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:09:40.789 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/340 2024-04-20T13:09:40.789 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/340 2024-04-20T13:09:40.789 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:09:40.797 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:40.821 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:09:40.822 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/341) 2024-04-20T13:09:41.285 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:09:41.297 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:09:41.297 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:42.117 INFO:teuthology.orchestra.run.smithi121.stdout:generic/341 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:09:42.128 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/341 2024-04-20T13:09:42.128 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/341 2024-04-20T13:09:42.128 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:09:42.136 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:42.167 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:09:42.167 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/342) 2024-04-20T13:09:42.708 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:09:42.720 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:09:42.720 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:43.256 INFO:teuthology.orchestra.run.smithi121.stdout:generic/342 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:09:43.268 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/342 2024-04-20T13:09:43.268 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/342 2024-04-20T13:09:43.268 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:09:43.276 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:43.304 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:09:43.305 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/343) 2024-04-20T13:09:43.798 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:09:43.811 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:09:43.811 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:44.781 INFO:teuthology.orchestra.run.smithi121.stdout:generic/343 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:09:44.793 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/343 2024-04-20T13:09:44.794 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/343 2024-04-20T13:09:44.794 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:09:44.807 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:44.838 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:09:44.838 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/344) 2024-04-20T13:09:45.359 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:09:45.371 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:09:45.371 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:46.286 INFO:teuthology.orchestra.run.smithi121.stdout:generic/344 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:09:46.296 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/344 2024-04-20T13:09:46.296 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/344 2024-04-20T13:09:46.296 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:09:46.304 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:46.330 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:09:46.330 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/345) 2024-04-20T13:09:46.814 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:09:46.825 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:09:46.825 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:48.156 INFO:teuthology.orchestra.run.smithi121.stdout:generic/345 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:09:48.167 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/345 2024-04-20T13:09:48.167 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/345 2024-04-20T13:09:48.167 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:09:48.174 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:48.198 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:09:48.198 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/346) 2024-04-20T13:09:48.676 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:09:48.689 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:09:48.689 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:49.066 INFO:teuthology.orchestra.run.smithi121.stdout:generic/346 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:09:49.077 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/346 2024-04-20T13:09:49.077 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/346 2024-04-20T13:09:49.077 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:09:49.086 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:49.117 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:09:49.117 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/347) 2024-04-20T13:09:49.591 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:09:49.601 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:09:49.601 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:50.108 INFO:teuthology.orchestra.run.smithi121.stdout:generic/347 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:09:50.119 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/347 2024-04-20T13:09:50.119 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/347 2024-04-20T13:09:50.119 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:09:50.128 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:50.154 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:09:50.155 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/348) 2024-04-20T13:09:50.607 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:09:50.618 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:09:50.618 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:51.246 INFO:teuthology.orchestra.run.smithi121.stdout:generic/348 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:09:51.260 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/348 2024-04-20T13:09:51.260 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/348 2024-04-20T13:09:51.260 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:09:51.272 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:51.303 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:09:51.304 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/349) 2024-04-20T13:09:51.782 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:09:51.793 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:09:51.794 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:54.747 INFO:teuthology.orchestra.run.smithi121.stdout:generic/349 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-04-20T13:09:54.758 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/349 2024-04-20T13:09:54.758 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/349 2024-04-20T13:09:54.758 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:09:54.766 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:09:54.794 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:09:54.794 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/350) 2024-04-20T13:09:55.236 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:09:55.248 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:09:55.248 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:00.010 INFO:teuthology.orchestra.run.smithi121.stdout:generic/350 4s 2024-04-20T13:10:00.025 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/350 2024-04-20T13:10:00.026 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:00.033 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:00.056 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:00.057 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/351) 2024-04-20T13:10:00.564 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:00.578 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:00.578 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:03.932 INFO:teuthology.orchestra.run.smithi121.stdout:generic/351 [not run] xfs_io finsert failed (old kernel/wrong fs?) 2024-04-20T13:10:03.943 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/351 2024-04-20T13:10:03.943 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/351 2024-04-20T13:10:03.944 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:03.952 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:04.000 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:04.000 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/352) 2024-04-20T13:10:04.707 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:04.719 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:04.719 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:05.108 INFO:teuthology.orchestra.run.smithi121.stdout:generic/352 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:10:05.119 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/352 2024-04-20T13:10:05.119 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/352 2024-04-20T13:10:05.119 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:05.126 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:05.149 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:05.149 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/353) 2024-04-20T13:10:05.625 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:05.636 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:05.637 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:05.985 INFO:teuthology.orchestra.run.smithi121.stdout:generic/353 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:10:05.995 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/353 2024-04-20T13:10:05.996 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/353 2024-04-20T13:10:05.996 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:06.007 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:06.035 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:06.035 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/354) 2024-04-20T13:10:06.512 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:06.523 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:06.523 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:06.887 INFO:teuthology.orchestra.run.smithi121.stdout:generic/354 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:10:06.898 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/354 2024-04-20T13:10:06.898 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/354 2024-04-20T13:10:06.898 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:06.906 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:06.930 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:06.930 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/355) 2024-04-20T13:10:07.408 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:07.419 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:07.419 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:09.743 INFO:teuthology.orchestra.run.smithi121.stdout:generic/355 2s 2024-04-20T13:10:09.760 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/355 2024-04-20T13:10:09.760 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:09.768 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:09.796 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:09.796 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/356) 2024-04-20T13:10:10.527 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:10.540 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:10.540 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:10.962 INFO:teuthology.orchestra.run.smithi121.stdout:generic/356 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:10:10.975 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/356 2024-04-20T13:10:10.976 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/356 2024-04-20T13:10:10.976 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:10.988 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:11.015 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:11.015 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/357) 2024-04-20T13:10:11.504 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:11.516 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:11.516 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:11.887 INFO:teuthology.orchestra.run.smithi121.stdout:generic/357 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:10:11.903 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/357 2024-04-20T13:10:11.903 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/357 2024-04-20T13:10:11.903 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:11.917 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:11.945 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:11.945 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/358) 2024-04-20T13:10:12.439 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:12.451 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:12.451 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:12.834 INFO:teuthology.orchestra.run.smithi121.stdout:generic/358 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:10:12.845 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/358 2024-04-20T13:10:12.845 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/358 2024-04-20T13:10:12.845 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:12.853 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:12.880 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:12.880 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/359) 2024-04-20T13:10:13.375 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:13.389 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:13.389 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:13.760 INFO:teuthology.orchestra.run.smithi121.stdout:generic/359 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:10:13.770 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/359 2024-04-20T13:10:13.771 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/359 2024-04-20T13:10:13.771 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:13.780 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:13.813 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:13.813 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/360) 2024-04-20T13:10:14.288 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:14.299 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:14.299 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:14.962 INFO:teuthology.orchestra.run.smithi121.stdout:generic/360 0s 2024-04-20T13:10:14.977 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/360 2024-04-20T13:10:14.977 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:14.985 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:15.013 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:15.013 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/361) 2024-04-20T13:10:15.501 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:15.513 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:15.513 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:15.939 INFO:teuthology.orchestra.run.smithi121.stdout:generic/361 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:10:15.950 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/361 2024-04-20T13:10:15.950 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/361 2024-04-20T13:10:15.950 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:15.959 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:15.988 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:15.988 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/362) 2024-04-20T13:10:16.461 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:16.472 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:16.472 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:16.835 INFO:teuthology.orchestra.run.smithi121.stdout:generic/362 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:10:16.846 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/362 2024-04-20T13:10:16.846 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/362 2024-04-20T13:10:16.846 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:16.854 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:16.881 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:16.881 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/363) 2024-04-20T13:10:17.364 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:17.376 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:17.377 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:17.742 INFO:teuthology.orchestra.run.smithi121.stdout:generic/363 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:10:17.752 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/363 2024-04-20T13:10:17.752 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/363 2024-04-20T13:10:17.752 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:17.759 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:17.785 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:17.785 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/364) 2024-04-20T13:10:18.284 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:18.295 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:18.295 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:18.667 INFO:teuthology.orchestra.run.smithi121.stdout:generic/364 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:10:18.677 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/364 2024-04-20T13:10:18.678 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/364 2024-04-20T13:10:18.678 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:18.687 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:18.712 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:18.712 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/365) 2024-04-20T13:10:19.189 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:19.201 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:19.202 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:19.607 INFO:teuthology.orchestra.run.smithi121.stdout:generic/365 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:10:19.620 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/365 2024-04-20T13:10:19.620 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/365 2024-04-20T13:10:19.621 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:19.633 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:19.666 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:19.666 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/366) 2024-04-20T13:10:20.137 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:20.147 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:20.148 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:20.511 INFO:teuthology.orchestra.run.smithi121.stdout:generic/366 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:10:20.521 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/366 2024-04-20T13:10:20.521 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/366 2024-04-20T13:10:20.521 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:20.531 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:20.558 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:20.558 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/367) 2024-04-20T13:10:21.007 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:21.019 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:21.019 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:21.434 INFO:teuthology.orchestra.run.smithi121.stdout:generic/367 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:10:21.448 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/367 2024-04-20T13:10:21.448 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/367 2024-04-20T13:10:21.448 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:21.457 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:21.486 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:21.487 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/368) 2024-04-20T13:10:21.988 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:22.000 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:22.000 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:22.317 INFO:teuthology.orchestra.run.smithi121.stdout:generic/368 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:10:22.326 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/368 2024-04-20T13:10:22.326 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/368 2024-04-20T13:10:22.326 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:22.333 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:22.355 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:22.355 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/369) 2024-04-20T13:10:22.788 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:22.799 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:22.799 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:23.206 INFO:teuthology.orchestra.run.smithi121.stdout:generic/369 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:10:23.218 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/369 2024-04-20T13:10:23.218 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/369 2024-04-20T13:10:23.218 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:23.228 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:23.252 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:23.253 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/370) 2024-04-20T13:10:23.703 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:23.716 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:23.716 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:24.084 INFO:teuthology.orchestra.run.smithi121.stdout:generic/370 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:10:24.095 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/370 2024-04-20T13:10:24.095 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/370 2024-04-20T13:10:24.095 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:24.102 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:24.124 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:24.125 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/371) 2024-04-20T13:10:24.612 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:24.623 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:24.623 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:25.013 INFO:teuthology.orchestra.run.smithi121.stdout:generic/371 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:10:25.022 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/371 2024-04-20T13:10:25.022 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/371 2024-04-20T13:10:25.023 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:25.032 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:25.060 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:25.060 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/372) 2024-04-20T13:10:25.554 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:25.568 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:25.568 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:26.006 INFO:teuthology.orchestra.run.smithi121.stdout:generic/372 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:10:26.020 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/372 2024-04-20T13:10:26.020 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/372 2024-04-20T13:10:26.021 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:26.031 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:26.058 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:26.058 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/373) 2024-04-20T13:10:26.507 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:26.520 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:26.520 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:26.904 INFO:teuthology.orchestra.run.smithi121.stdout:generic/373 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:10:26.915 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/373 2024-04-20T13:10:26.915 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/373 2024-04-20T13:10:26.915 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:26.923 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:26.950 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:26.950 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/374) 2024-04-20T13:10:27.415 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:27.426 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:27.426 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:27.836 INFO:teuthology.orchestra.run.smithi121.stdout:generic/374 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:10:27.848 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/374 2024-04-20T13:10:27.848 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/374 2024-04-20T13:10:27.848 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:27.856 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:27.882 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:27.882 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/375) 2024-04-20T13:10:28.358 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:28.369 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:28.369 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:28.865 INFO:teuthology.orchestra.run.smithi121.stdout:generic/375 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-04-20T13:10:28.876 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/375 2024-04-20T13:10:28.877 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/375 2024-04-20T13:10:28.877 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:28.887 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:28.916 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:28.916 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/376) 2024-04-20T13:10:29.394 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:29.406 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:29.406 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:29.920 INFO:teuthology.orchestra.run.smithi121.stdout:generic/376 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:10:29.932 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/376 2024-04-20T13:10:29.933 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/376 2024-04-20T13:10:29.933 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:29.942 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:29.969 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:29.969 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/377) 2024-04-20T13:10:30.441 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:30.453 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:30.453 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:30.827 INFO:teuthology.orchestra.run.smithi121.stdout:generic/377 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:10:30.837 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/377 2024-04-20T13:10:30.837 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/377 2024-04-20T13:10:30.837 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:30.845 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:30.869 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:30.869 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/378) 2024-04-20T13:10:31.320 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:31.332 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:31.332 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:33.330 INFO:teuthology.orchestra.run.smithi121.stdout:generic/378 2s 2024-04-20T13:10:33.347 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/378 2024-04-20T13:10:33.347 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:33.355 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:33.402 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:33.403 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/379) 2024-04-20T13:10:34.125 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:34.137 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:34.137 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:34.582 INFO:teuthology.orchestra.run.smithi121.stdout:generic/379 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:10:34.593 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/379 2024-04-20T13:10:34.593 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/379 2024-04-20T13:10:34.593 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:34.602 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:34.629 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:34.629 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/380) 2024-04-20T13:10:35.062 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:35.073 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:35.073 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:35.482 INFO:teuthology.orchestra.run.smithi121.stdout:generic/380 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:10:35.491 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/380 2024-04-20T13:10:35.491 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/380 2024-04-20T13:10:35.491 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:35.499 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:35.523 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:35.523 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/381) 2024-04-20T13:10:36.033 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:36.047 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:36.047 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:36.458 INFO:teuthology.orchestra.run.smithi121.stdout:generic/381 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:10:36.469 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/381 2024-04-20T13:10:36.470 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/381 2024-04-20T13:10:36.470 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:36.478 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:36.509 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:36.509 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/382) 2024-04-20T13:10:37.013 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:37.024 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:37.025 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:37.419 INFO:teuthology.orchestra.run.smithi121.stdout:generic/382 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:10:37.430 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/382 2024-04-20T13:10:37.430 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/382 2024-04-20T13:10:37.430 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:37.439 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:37.467 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:37.468 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/383) 2024-04-20T13:10:37.946 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:37.957 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:37.957 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:38.389 INFO:teuthology.orchestra.run.smithi121.stdout:generic/383 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:10:38.400 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/383 2024-04-20T13:10:38.400 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/383 2024-04-20T13:10:38.400 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:38.409 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:38.436 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:38.437 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/384) 2024-04-20T13:10:38.910 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:38.921 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:38.921 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:39.279 INFO:teuthology.orchestra.run.smithi121.stdout:generic/384 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2024-04-20T13:10:39.288 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/384 2024-04-20T13:10:39.289 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/384 2024-04-20T13:10:39.289 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:39.296 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:39.321 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:39.322 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/385) 2024-04-20T13:10:39.796 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:39.808 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:39.808 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:40.156 INFO:teuthology.orchestra.run.smithi121.stdout:generic/385 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:10:40.166 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/385 2024-04-20T13:10:40.167 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/385 2024-04-20T13:10:40.167 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:40.174 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:40.199 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:40.199 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/386) 2024-04-20T13:10:40.709 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:40.720 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:40.720 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:41.063 INFO:teuthology.orchestra.run.smithi121.stdout:generic/386 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2024-04-20T13:10:41.075 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/386 2024-04-20T13:10:41.075 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/386 2024-04-20T13:10:41.076 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:41.084 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:41.110 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:41.110 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/387) 2024-04-20T13:10:41.574 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:41.585 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:41.585 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:41.969 INFO:teuthology.orchestra.run.smithi121.stdout:generic/387 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:10:41.980 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/387 2024-04-20T13:10:41.981 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/387 2024-04-20T13:10:41.981 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:41.989 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:42.016 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:42.016 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/388) 2024-04-20T13:10:42.504 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:42.515 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:42.515 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:42.896 INFO:teuthology.orchestra.run.smithi121.stdout:generic/388 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:10:42.907 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/388 2024-04-20T13:10:42.907 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/388 2024-04-20T13:10:42.907 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:42.918 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:42.945 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:42.946 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/389) 2024-04-20T13:10:43.407 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:43.419 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:43.419 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:44.080 INFO:teuthology.orchestra.run.smithi121.stdout:generic/389 [not run] O_TMPFILE is not supported 2024-04-20T13:10:44.092 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/389 2024-04-20T13:10:44.093 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/389 2024-04-20T13:10:44.093 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:44.104 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:44.132 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:44.133 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/390) 2024-04-20T13:10:44.622 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:44.634 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:44.635 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:45.056 INFO:teuthology.orchestra.run.smithi121.stdout:generic/390 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:10:45.066 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/390 2024-04-20T13:10:45.067 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/390 2024-04-20T13:10:45.067 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:10:45.076 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:10:45.103 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:10:45.103 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/391) 2024-04-20T13:10:45.588 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:10:45.599 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:10:45.599 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:11:54.939 INFO:teuthology.orchestra.run.smithi121.stdout:generic/391 69s 2024-04-20T13:11:55.132 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/391 2024-04-20T13:11:55.132 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:11:55.144 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:11:55.351 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:11:55.351 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/392) 2024-04-20T13:11:59.645 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:11:59.659 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:11:59.659 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:00.543 INFO:teuthology.orchestra.run.smithi121.stdout:generic/392 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:12:00.556 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/392 2024-04-20T13:12:00.556 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/392 2024-04-20T13:12:00.556 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:12:00.567 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:00.597 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:12:00.597 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/393) 2024-04-20T13:12:01.092 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:12:01.103 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:12:01.103 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:01.498 INFO:teuthology.orchestra.run.smithi121.stdout:generic/393 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:12:01.508 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/393 2024-04-20T13:12:01.509 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/393 2024-04-20T13:12:01.509 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:12:01.518 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:01.545 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:12:01.545 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/394) 2024-04-20T13:12:02.053 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:12:02.065 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:12:02.065 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:02.573 INFO:teuthology.orchestra.run.smithi121.stdout:generic/394 0s 2024-04-20T13:12:02.588 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/394 2024-04-20T13:12:02.588 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:12:02.596 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:02.624 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:12:02.624 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/395) 2024-04-20T13:12:03.498 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:12:03.513 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:12:03.513 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:03.971 INFO:teuthology.orchestra.run.smithi121.stdout:generic/395 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:12:03.981 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/395 2024-04-20T13:12:03.981 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/395 2024-04-20T13:12:03.981 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:12:03.988 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:04.016 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:12:04.016 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/396) 2024-04-20T13:12:04.507 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:12:04.518 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:12:04.518 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:04.988 INFO:teuthology.orchestra.run.smithi121.stdout:generic/396 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:12:05.002 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/396 2024-04-20T13:12:05.002 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/396 2024-04-20T13:12:05.002 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:12:05.013 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:05.041 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:12:05.042 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/397) 2024-04-20T13:12:05.532 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:12:05.544 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:12:05.544 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:06.048 INFO:teuthology.orchestra.run.smithi121.stdout:generic/397 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:12:06.061 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/397 2024-04-20T13:12:06.061 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/397 2024-04-20T13:12:06.061 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:12:06.072 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:06.105 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:12:06.106 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/398) 2024-04-20T13:12:06.582 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:12:06.594 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:12:06.594 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:07.036 INFO:teuthology.orchestra.run.smithi121.stdout:generic/398 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:12:07.049 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/398 2024-04-20T13:12:07.050 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/398 2024-04-20T13:12:07.050 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:12:07.059 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:07.086 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:12:07.086 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/399) 2024-04-20T13:12:07.575 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:12:07.586 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:12:07.586 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:08.035 INFO:teuthology.orchestra.run.smithi121.stdout:generic/399 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:12:08.047 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/399 2024-04-20T13:12:08.047 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/399 2024-04-20T13:12:08.047 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:12:08.058 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:08.087 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:12:08.088 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/400) 2024-04-20T13:12:08.560 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:12:08.571 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:12:08.571 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:08.991 INFO:teuthology.orchestra.run.smithi121.stdout:generic/400 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2024-04-20T13:12:09.001 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/400 2024-04-20T13:12:09.001 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/400 2024-04-20T13:12:09.001 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:12:09.009 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:09.031 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:12:09.031 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/401) 2024-04-20T13:12:09.523 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:12:09.534 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:12:09.534 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:09.907 INFO:teuthology.orchestra.run.smithi121.stdout:generic/401 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:12:09.917 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/401 2024-04-20T13:12:09.917 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/401 2024-04-20T13:12:09.917 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:12:09.925 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:09.951 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:12:09.951 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/402) 2024-04-20T13:12:10.456 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:12:10.469 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:12:10.469 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:10.892 INFO:teuthology.orchestra.run.smithi121.stdout:generic/402 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:12:10.904 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/402 2024-04-20T13:12:10.904 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/402 2024-04-20T13:12:10.905 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:12:10.914 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:10.942 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:12:10.943 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/403) 2024-04-20T13:12:11.460 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:12:11.472 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:12:11.472 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:11.895 INFO:teuthology.orchestra.run.smithi121.stdout:generic/403 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:12:11.906 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/403 2024-04-20T13:12:11.906 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/403 2024-04-20T13:12:11.906 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:12:11.914 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:11.941 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:12:11.941 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/404) 2024-04-20T13:12:12.500 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:12:12.514 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:12:12.514 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:17.157 INFO:teuthology.orchestra.run.smithi121.stdout:generic/404 [not run] xfs_io finsert failed (old kernel/wrong fs?) 2024-04-20T13:12:17.169 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/404 2024-04-20T13:12:17.169 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/404 2024-04-20T13:12:17.169 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:12:17.178 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:17.207 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:12:17.207 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/405) 2024-04-20T13:12:17.912 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:12:17.923 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:12:17.924 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:18.501 INFO:teuthology.orchestra.run.smithi121.stdout:generic/405 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:12:18.515 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/405 2024-04-20T13:12:18.515 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/405 2024-04-20T13:12:18.515 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:12:18.529 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:18.557 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:12:18.557 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/406) 2024-04-20T13:12:19.103 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:12:19.115 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:12:19.115 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:19.557 INFO:teuthology.orchestra.run.smithi121.stdout:generic/406 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:12:19.571 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/406 2024-04-20T13:12:19.571 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/406 2024-04-20T13:12:19.571 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:12:19.583 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:19.614 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:12:19.614 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/407) 2024-04-20T13:12:20.151 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:12:20.165 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:12:20.165 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:20.745 INFO:teuthology.orchestra.run.smithi121.stdout:generic/407 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-20T13:12:20.755 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/407 2024-04-20T13:12:20.756 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/407 2024-04-20T13:12:20.756 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:12:20.764 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:20.795 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:12:20.795 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/408) 2024-04-20T13:12:21.466 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:12:21.479 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:12:21.479 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:22.071 INFO:teuthology.orchestra.run.smithi121.stdout:generic/408 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2024-04-20T13:12:22.084 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/408 2024-04-20T13:12:22.084 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/408 2024-04-20T13:12:22.084 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:12:22.094 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:22.140 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:12:22.140 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/409) 2024-04-20T13:12:22.813 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:12:22.825 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:12:22.825 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:23.268 INFO:teuthology.orchestra.run.smithi121.stdout:generic/409 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:12:23.279 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/409 2024-04-20T13:12:23.279 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/409 2024-04-20T13:12:23.279 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:12:23.288 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:23.319 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:12:23.320 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/410) 2024-04-20T13:12:23.845 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:12:23.859 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:12:23.859 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:24.340 INFO:teuthology.orchestra.run.smithi121.stdout:generic/410 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:12:24.351 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/410 2024-04-20T13:12:24.351 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/410 2024-04-20T13:12:24.351 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:12:24.361 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:24.391 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:12:24.391 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/411) 2024-04-20T13:12:24.936 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:12:24.948 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:12:24.948 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:25.363 INFO:teuthology.orchestra.run.smithi121.stdout:generic/411 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:12:25.375 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/411 2024-04-20T13:12:25.375 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/411 2024-04-20T13:12:25.375 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:12:25.384 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:25.414 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:12:25.414 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/412) 2024-04-20T13:12:25.925 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:12:25.939 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:12:25.939 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:26.391 INFO:teuthology.orchestra.run.smithi121.stdout:generic/412 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:12:26.404 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/412 2024-04-20T13:12:26.404 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/412 2024-04-20T13:12:26.404 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:12:26.414 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:26.443 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:12:26.443 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/413) 2024-04-20T13:12:27.002 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:12:27.018 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:12:27.018 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:27.448 INFO:teuthology.orchestra.run.smithi121.stdout:generic/413 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:12:27.462 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/413 2024-04-20T13:12:27.462 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/413 2024-04-20T13:12:27.462 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:12:27.474 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:27.504 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:12:27.504 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/414) 2024-04-20T13:12:28.042 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:12:28.056 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:12:28.056 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:28.457 INFO:teuthology.orchestra.run.smithi121.stdout:generic/414 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:12:28.468 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/414 2024-04-20T13:12:28.468 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/414 2024-04-20T13:12:28.468 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:12:28.476 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:28.504 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:12:28.504 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/415) 2024-04-20T13:12:29.071 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:12:29.084 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:12:29.084 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:29.509 INFO:teuthology.orchestra.run.smithi121.stdout:generic/415 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:12:29.520 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/415 2024-04-20T13:12:29.521 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/415 2024-04-20T13:12:29.521 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:12:29.531 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:29.561 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:12:29.561 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/416) 2024-04-20T13:12:30.114 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:12:30.128 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:12:30.129 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:30.592 INFO:teuthology.orchestra.run.smithi121.stdout:generic/416 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:12:30.604 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/416 2024-04-20T13:12:30.604 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/416 2024-04-20T13:12:30.604 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:12:30.614 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:30.649 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:12:30.649 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/417) 2024-04-20T13:12:31.185 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:12:31.199 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:12:31.199 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:31.621 INFO:teuthology.orchestra.run.smithi121.stdout:generic/417 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:12:31.629 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/417 2024-04-20T13:12:31.630 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/417 2024-04-20T13:12:31.630 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:12:31.638 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:31.664 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:12:31.664 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/418) 2024-04-20T13:12:32.198 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:12:32.213 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:12:32.213 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:32.752 INFO:teuthology.orchestra.run.smithi121.stdout:generic/418 [not run] require ceph-fuse to be valid block disk 2024-04-20T13:12:32.763 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/418 2024-04-20T13:12:32.764 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/418 2024-04-20T13:12:32.764 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:12:32.773 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:32.805 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:12:32.805 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/419) 2024-04-20T13:12:33.362 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:12:33.376 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:12:33.376 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:33.901 INFO:teuthology.orchestra.run.smithi121.stdout:generic/419 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:12:33.916 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/419 2024-04-20T13:12:33.916 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/419 2024-04-20T13:12:33.916 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:12:33.928 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:33.963 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:12:33.963 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/420) 2024-04-20T13:12:34.485 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:12:34.500 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:12:34.500 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:37.861 INFO:teuthology.orchestra.run.smithi121.stdout:generic/420 3s 2024-04-20T13:12:37.877 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/420 2024-04-20T13:12:37.877 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:12:37.885 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:37.917 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:12:37.918 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/421) 2024-04-20T13:12:38.598 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:12:38.610 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:12:38.610 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:39.097 INFO:teuthology.orchestra.run.smithi121.stdout:generic/421 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:12:39.108 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/421 2024-04-20T13:12:39.109 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/421 2024-04-20T13:12:39.109 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:12:39.119 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:39.148 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:12:39.149 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/422) 2024-04-20T13:12:39.672 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:12:39.684 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:12:39.684 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:40.080 INFO:teuthology.orchestra.run.smithi121.stdout:generic/422 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:12:40.094 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/422 2024-04-20T13:12:40.094 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/422 2024-04-20T13:12:40.094 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:12:40.106 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:40.139 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:12:40.140 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/423) 2024-04-20T13:12:40.540 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:12:40.544 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:12:40.673 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:12:40.685 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:12:40.685 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:41.532 INFO:teuthology.orchestra.run.smithi121.stdout:generic/423 1s 2024-04-20T13:12:41.553 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/423 2024-04-20T13:12:41.553 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:12:41.565 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:41.611 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:12:41.612 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/424) 2024-04-20T13:12:42.302 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:12:42.314 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:12:42.314 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:42.813 INFO:teuthology.orchestra.run.smithi121.stdout:generic/424 [not run] file system doesn't support any of /bin/chattr +a/+c/+d/+i 2024-04-20T13:12:42.824 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/424 2024-04-20T13:12:42.824 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/424 2024-04-20T13:12:42.824 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:12:42.834 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:42.867 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:12:42.867 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/425) 2024-04-20T13:12:43.399 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:12:43.413 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:12:43.413 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:43.833 INFO:teuthology.orchestra.run.smithi121.stdout:generic/425 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:12:43.845 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/425 2024-04-20T13:12:43.845 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/425 2024-04-20T13:12:43.845 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:12:43.854 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:12:43.883 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:12:43.884 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/426) 2024-04-20T13:12:44.412 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:12:44.425 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:12:44.425 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:13:10.677 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:13:10.685 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:13:41.463 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:13:41.472 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:14:09.170 INFO:teuthology.orchestra.run.smithi121.stdout:generic/426 - output mismatch (see /tmp/tmp.yONpfXTHudxfstests-dev/results//generic/426.out.bad) 2024-04-20T13:14:09.198 INFO:teuthology.orchestra.run.smithi121.stdout: --- tests/generic/426.out 2024-04-20 10:23:05.953112330 +0000 2024-04-20T13:14:09.198 INFO:teuthology.orchestra.run.smithi121.stdout: +++ /tmp/tmp.yONpfXTHudxfstests-dev/results//generic/426.out.bad 2024-04-20 13:14:09.118446179 +0000 2024-04-20T13:14:09.198 INFO:teuthology.orchestra.run.smithi121.stdout: @@ -1,5 +1,384 @@ 2024-04-20T13:14:09.198 INFO:teuthology.orchestra.run.smithi121.stdout: QA output created by 426 2024-04-20T13:14:09.198 INFO:teuthology.orchestra.run.smithi121.stdout: test_file_handles TEST_DIR/426-dir -d 2024-04-20T13:14:09.198 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000645) opened an unlinked file! 2024-04-20T13:14:09.198 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000646) opened an unlinked file! 2024-04-20T13:14:09.198 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000647) opened an unlinked file! 2024-04-20T13:14:09.199 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000648) opened an unlinked file! 2024-04-20T13:14:09.199 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000649) opened an unlinked file! 2024-04-20T13:14:09.199 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000650) opened an unlinked file! 2024-04-20T13:14:09.199 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000651) opened an unlinked file! 2024-04-20T13:14:09.199 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000652) opened an unlinked file! 2024-04-20T13:14:09.199 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000653) opened an unlinked file! 2024-04-20T13:14:09.199 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000654) opened an unlinked file! 2024-04-20T13:14:09.199 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000655) opened an unlinked file! 2024-04-20T13:14:09.199 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000656) opened an unlinked file! 2024-04-20T13:14:09.199 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000657) opened an unlinked file! 2024-04-20T13:14:09.199 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000658) opened an unlinked file! 2024-04-20T13:14:09.199 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000659) opened an unlinked file! 2024-04-20T13:14:09.199 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000660) opened an unlinked file! 2024-04-20T13:14:09.199 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000661) opened an unlinked file! 2024-04-20T13:14:09.200 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000662) opened an unlinked file! 2024-04-20T13:14:09.200 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000663) opened an unlinked file! 2024-04-20T13:14:09.200 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000664) opened an unlinked file! 2024-04-20T13:14:09.200 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000665) opened an unlinked file! 2024-04-20T13:14:09.200 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000666) opened an unlinked file! 2024-04-20T13:14:09.200 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000667) opened an unlinked file! 2024-04-20T13:14:09.200 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000668) opened an unlinked file! 2024-04-20T13:14:09.200 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000669) opened an unlinked file! 2024-04-20T13:14:09.200 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000670) opened an unlinked file! 2024-04-20T13:14:09.200 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000671) opened an unlinked file! 2024-04-20T13:14:09.200 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000672) opened an unlinked file! 2024-04-20T13:14:09.200 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000673) opened an unlinked file! 2024-04-20T13:14:09.200 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000674) opened an unlinked file! 2024-04-20T13:14:09.200 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000675) opened an unlinked file! 2024-04-20T13:14:09.201 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000676) opened an unlinked file! 2024-04-20T13:14:09.201 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000677) opened an unlinked file! 2024-04-20T13:14:09.201 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000678) opened an unlinked file! 2024-04-20T13:14:09.201 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000679) opened an unlinked file! 2024-04-20T13:14:09.201 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000680) opened an unlinked file! 2024-04-20T13:14:09.201 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000681) opened an unlinked file! 2024-04-20T13:14:09.201 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000682) opened an unlinked file! 2024-04-20T13:14:09.201 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000683) opened an unlinked file! 2024-04-20T13:14:09.201 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000684) opened an unlinked file! 2024-04-20T13:14:09.201 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000685) opened an unlinked file! 2024-04-20T13:14:09.201 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000686) opened an unlinked file! 2024-04-20T13:14:09.201 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000687) opened an unlinked file! 2024-04-20T13:14:09.201 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000688) opened an unlinked file! 2024-04-20T13:14:09.201 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000689) opened an unlinked file! 2024-04-20T13:14:09.201 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000690) opened an unlinked file! 2024-04-20T13:14:09.202 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000691) opened an unlinked file! 2024-04-20T13:14:09.202 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000692) opened an unlinked file! 2024-04-20T13:14:09.202 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000693) opened an unlinked file! 2024-04-20T13:14:09.202 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000694) opened an unlinked file! 2024-04-20T13:14:09.202 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000695) opened an unlinked file! 2024-04-20T13:14:09.202 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000696) opened an unlinked file! 2024-04-20T13:14:09.202 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000697) opened an unlinked file! 2024-04-20T13:14:09.202 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000698) opened an unlinked file! 2024-04-20T13:14:09.202 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000699) opened an unlinked file! 2024-04-20T13:14:09.202 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000700) opened an unlinked file! 2024-04-20T13:14:09.202 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000701) opened an unlinked file! 2024-04-20T13:14:09.202 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000702) opened an unlinked file! 2024-04-20T13:14:09.202 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000703) opened an unlinked file! 2024-04-20T13:14:09.202 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000704) opened an unlinked file! 2024-04-20T13:14:09.203 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000705) opened an unlinked file! 2024-04-20T13:14:09.203 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000706) opened an unlinked file! 2024-04-20T13:14:09.203 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000707) opened an unlinked file! 2024-04-20T13:14:09.203 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000708) opened an unlinked file! 2024-04-20T13:14:09.203 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000709) opened an unlinked file! 2024-04-20T13:14:09.203 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000710) opened an unlinked file! 2024-04-20T13:14:09.203 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000711) opened an unlinked file! 2024-04-20T13:14:09.203 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000712) opened an unlinked file! 2024-04-20T13:14:09.203 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000713) opened an unlinked file! 2024-04-20T13:14:09.203 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000714) opened an unlinked file! 2024-04-20T13:14:09.203 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000715) opened an unlinked file! 2024-04-20T13:14:09.203 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000716) opened an unlinked file! 2024-04-20T13:14:09.203 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000717) opened an unlinked file! 2024-04-20T13:14:09.203 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000718) opened an unlinked file! 2024-04-20T13:14:09.204 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000719) opened an unlinked file! 2024-04-20T13:14:09.204 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000720) opened an unlinked file! 2024-04-20T13:14:09.204 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000721) opened an unlinked file! 2024-04-20T13:14:09.204 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000722) opened an unlinked file! 2024-04-20T13:14:09.204 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000723) opened an unlinked file! 2024-04-20T13:14:09.204 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000724) opened an unlinked file! 2024-04-20T13:14:09.204 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000725) opened an unlinked file! 2024-04-20T13:14:09.204 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000726) opened an unlinked file! 2024-04-20T13:14:09.204 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000727) opened an unlinked file! 2024-04-20T13:14:09.204 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000728) opened an unlinked file! 2024-04-20T13:14:09.204 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000729) opened an unlinked file! 2024-04-20T13:14:09.204 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000730) opened an unlinked file! 2024-04-20T13:14:09.204 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000731) opened an unlinked file! 2024-04-20T13:14:09.204 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000732) opened an unlinked file! 2024-04-20T13:14:09.204 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000733) opened an unlinked file! 2024-04-20T13:14:09.205 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000734) opened an unlinked file! 2024-04-20T13:14:09.205 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000735) opened an unlinked file! 2024-04-20T13:14:09.205 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000736) opened an unlinked file! 2024-04-20T13:14:09.205 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000737) opened an unlinked file! 2024-04-20T13:14:09.205 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000738) opened an unlinked file! 2024-04-20T13:14:09.205 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000739) opened an unlinked file! 2024-04-20T13:14:09.205 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000740) opened an unlinked file! 2024-04-20T13:14:09.205 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000741) opened an unlinked file! 2024-04-20T13:14:09.205 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000742) opened an unlinked file! 2024-04-20T13:14:09.205 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000743) opened an unlinked file! 2024-04-20T13:14:09.205 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000744) opened an unlinked file! 2024-04-20T13:14:09.205 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000745) opened an unlinked file! 2024-04-20T13:14:09.205 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000746) opened an unlinked file! 2024-04-20T13:14:09.205 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000747) opened an unlinked file! 2024-04-20T13:14:09.206 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000748) opened an unlinked file! 2024-04-20T13:14:09.206 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000749) opened an unlinked file! 2024-04-20T13:14:09.206 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000750) opened an unlinked file! 2024-04-20T13:14:09.206 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000751) opened an unlinked file! 2024-04-20T13:14:09.206 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000752) opened an unlinked file! 2024-04-20T13:14:09.206 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000753) opened an unlinked file! 2024-04-20T13:14:09.206 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000754) opened an unlinked file! 2024-04-20T13:14:09.206 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000755) opened an unlinked file! 2024-04-20T13:14:09.206 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000756) opened an unlinked file! 2024-04-20T13:14:09.206 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000757) opened an unlinked file! 2024-04-20T13:14:09.206 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000758) opened an unlinked file! 2024-04-20T13:14:09.206 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000759) opened an unlinked file! 2024-04-20T13:14:09.206 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000760) opened an unlinked file! 2024-04-20T13:14:09.206 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000761) opened an unlinked file! 2024-04-20T13:14:09.207 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000762) opened an unlinked file! 2024-04-20T13:14:09.207 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000763) opened an unlinked file! 2024-04-20T13:14:09.207 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000764) opened an unlinked file! 2024-04-20T13:14:09.207 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000765) opened an unlinked file! 2024-04-20T13:14:09.207 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000766) opened an unlinked file! 2024-04-20T13:14:09.207 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000767) opened an unlinked file! 2024-04-20T13:14:09.207 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000768) opened an unlinked file! 2024-04-20T13:14:09.207 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000769) opened an unlinked file! 2024-04-20T13:14:09.207 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000770) opened an unlinked file! 2024-04-20T13:14:09.207 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000771) opened an unlinked file! 2024-04-20T13:14:09.207 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000772) opened an unlinked file! 2024-04-20T13:14:09.207 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000773) opened an unlinked file! 2024-04-20T13:14:09.207 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000774) opened an unlinked file! 2024-04-20T13:14:09.207 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000775) opened an unlinked file! 2024-04-20T13:14:09.208 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000776) opened an unlinked file! 2024-04-20T13:14:09.208 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000777) opened an unlinked file! 2024-04-20T13:14:09.208 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000778) opened an unlinked file! 2024-04-20T13:14:09.208 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000779) opened an unlinked file! 2024-04-20T13:14:09.208 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000780) opened an unlinked file! 2024-04-20T13:14:09.208 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000781) opened an unlinked file! 2024-04-20T13:14:09.208 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000782) opened an unlinked file! 2024-04-20T13:14:09.208 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000783) opened an unlinked file! 2024-04-20T13:14:09.208 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000784) opened an unlinked file! 2024-04-20T13:14:09.208 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000785) opened an unlinked file! 2024-04-20T13:14:09.208 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000786) opened an unlinked file! 2024-04-20T13:14:09.208 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000787) opened an unlinked file! 2024-04-20T13:14:09.208 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000788) opened an unlinked file! 2024-04-20T13:14:09.208 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000789) opened an unlinked file! 2024-04-20T13:14:09.208 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000790) opened an unlinked file! 2024-04-20T13:14:09.209 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000791) opened an unlinked file! 2024-04-20T13:14:09.209 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000792) opened an unlinked file! 2024-04-20T13:14:09.209 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000793) opened an unlinked file! 2024-04-20T13:14:09.209 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000794) opened an unlinked file! 2024-04-20T13:14:09.209 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000795) opened an unlinked file! 2024-04-20T13:14:09.209 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000796) opened an unlinked file! 2024-04-20T13:14:09.209 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000797) opened an unlinked file! 2024-04-20T13:14:09.209 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000798) opened an unlinked file! 2024-04-20T13:14:09.209 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000799) opened an unlinked file! 2024-04-20T13:14:09.209 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000800) opened an unlinked file! 2024-04-20T13:14:09.209 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000801) opened an unlinked file! 2024-04-20T13:14:09.209 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000802) opened an unlinked file! 2024-04-20T13:14:09.209 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000803) opened an unlinked file! 2024-04-20T13:14:09.209 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000804) opened an unlinked file! 2024-04-20T13:14:09.210 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000805) opened an unlinked file! 2024-04-20T13:14:09.210 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000806) opened an unlinked file! 2024-04-20T13:14:09.210 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000807) opened an unlinked file! 2024-04-20T13:14:09.210 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000808) opened an unlinked file! 2024-04-20T13:14:09.210 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000809) opened an unlinked file! 2024-04-20T13:14:09.210 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000810) opened an unlinked file! 2024-04-20T13:14:09.210 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000811) opened an unlinked file! 2024-04-20T13:14:09.210 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000812) opened an unlinked file! 2024-04-20T13:14:09.210 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000813) opened an unlinked file! 2024-04-20T13:14:09.210 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000814) opened an unlinked file! 2024-04-20T13:14:09.210 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000815) opened an unlinked file! 2024-04-20T13:14:09.210 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000816) opened an unlinked file! 2024-04-20T13:14:09.210 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000817) opened an unlinked file! 2024-04-20T13:14:09.210 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000818) opened an unlinked file! 2024-04-20T13:14:09.211 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000819) opened an unlinked file! 2024-04-20T13:14:09.211 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000820) opened an unlinked file! 2024-04-20T13:14:09.211 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000821) opened an unlinked file! 2024-04-20T13:14:09.211 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000822) opened an unlinked file! 2024-04-20T13:14:09.211 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000823) opened an unlinked file! 2024-04-20T13:14:09.211 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000824) opened an unlinked file! 2024-04-20T13:14:09.211 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000825) opened an unlinked file! 2024-04-20T13:14:09.211 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000826) opened an unlinked file! 2024-04-20T13:14:09.211 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000827) opened an unlinked file! 2024-04-20T13:14:09.211 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000828) opened an unlinked file! 2024-04-20T13:14:09.211 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000829) opened an unlinked file! 2024-04-20T13:14:09.211 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000830) opened an unlinked file! 2024-04-20T13:14:09.211 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000831) opened an unlinked file! 2024-04-20T13:14:09.211 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000832) opened an unlinked file! 2024-04-20T13:14:09.212 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000833) opened an unlinked file! 2024-04-20T13:14:09.212 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000834) opened an unlinked file! 2024-04-20T13:14:09.212 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000835) opened an unlinked file! 2024-04-20T13:14:09.212 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000836) opened an unlinked file! 2024-04-20T13:14:09.212 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000837) opened an unlinked file! 2024-04-20T13:14:09.212 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000838) opened an unlinked file! 2024-04-20T13:14:09.212 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000839) opened an unlinked file! 2024-04-20T13:14:09.212 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000840) opened an unlinked file! 2024-04-20T13:14:09.212 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000841) opened an unlinked file! 2024-04-20T13:14:09.212 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000842) opened an unlinked file! 2024-04-20T13:14:09.212 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000843) opened an unlinked file! 2024-04-20T13:14:09.212 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000844) opened an unlinked file! 2024-04-20T13:14:09.212 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000845) opened an unlinked file! 2024-04-20T13:14:09.212 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000846) opened an unlinked file! 2024-04-20T13:14:09.213 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000847) opened an unlinked file! 2024-04-20T13:14:09.213 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000848) opened an unlinked file! 2024-04-20T13:14:09.213 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000849) opened an unlinked file! 2024-04-20T13:14:09.213 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000850) opened an unlinked file! 2024-04-20T13:14:09.213 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000851) opened an unlinked file! 2024-04-20T13:14:09.213 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000852) opened an unlinked file! 2024-04-20T13:14:09.213 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000853) opened an unlinked file! 2024-04-20T13:14:09.213 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000854) opened an unlinked file! 2024-04-20T13:14:09.213 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000855) opened an unlinked file! 2024-04-20T13:14:09.213 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000856) opened an unlinked file! 2024-04-20T13:14:09.213 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000857) opened an unlinked file! 2024-04-20T13:14:09.213 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000858) opened an unlinked file! 2024-04-20T13:14:09.213 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000859) opened an unlinked file! 2024-04-20T13:14:09.213 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000860) opened an unlinked file! 2024-04-20T13:14:09.213 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000861) opened an unlinked file! 2024-04-20T13:14:09.214 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000862) opened an unlinked file! 2024-04-20T13:14:09.214 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000863) opened an unlinked file! 2024-04-20T13:14:09.214 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000864) opened an unlinked file! 2024-04-20T13:14:09.214 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000865) opened an unlinked file! 2024-04-20T13:14:09.214 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000866) opened an unlinked file! 2024-04-20T13:14:09.214 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000867) opened an unlinked file! 2024-04-20T13:14:09.214 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000868) opened an unlinked file! 2024-04-20T13:14:09.214 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000869) opened an unlinked file! 2024-04-20T13:14:09.214 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000870) opened an unlinked file! 2024-04-20T13:14:09.214 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000871) opened an unlinked file! 2024-04-20T13:14:09.214 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000872) opened an unlinked file! 2024-04-20T13:14:09.214 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000873) opened an unlinked file! 2024-04-20T13:14:09.214 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000874) opened an unlinked file! 2024-04-20T13:14:09.214 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000875) opened an unlinked file! 2024-04-20T13:14:09.215 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000876) opened an unlinked file! 2024-04-20T13:14:09.215 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000877) opened an unlinked file! 2024-04-20T13:14:09.215 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000878) opened an unlinked file! 2024-04-20T13:14:09.215 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000879) opened an unlinked file! 2024-04-20T13:14:09.215 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000880) opened an unlinked file! 2024-04-20T13:14:09.215 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000881) opened an unlinked file! 2024-04-20T13:14:09.215 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000882) opened an unlinked file! 2024-04-20T13:14:09.215 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000883) opened an unlinked file! 2024-04-20T13:14:09.215 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000884) opened an unlinked file! 2024-04-20T13:14:09.215 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000885) opened an unlinked file! 2024-04-20T13:14:09.215 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000886) opened an unlinked file! 2024-04-20T13:14:09.215 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000887) opened an unlinked file! 2024-04-20T13:14:09.215 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000888) opened an unlinked file! 2024-04-20T13:14:09.215 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000889) opened an unlinked file! 2024-04-20T13:14:09.216 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000890) opened an unlinked file! 2024-04-20T13:14:09.216 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000891) opened an unlinked file! 2024-04-20T13:14:09.216 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000892) opened an unlinked file! 2024-04-20T13:14:09.216 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000893) opened an unlinked file! 2024-04-20T13:14:09.216 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000894) opened an unlinked file! 2024-04-20T13:14:09.216 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000895) opened an unlinked file! 2024-04-20T13:14:09.216 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000896) opened an unlinked file! 2024-04-20T13:14:09.216 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000897) opened an unlinked file! 2024-04-20T13:14:09.216 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000898) opened an unlinked file! 2024-04-20T13:14:09.216 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000899) opened an unlinked file! 2024-04-20T13:14:09.216 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000900) opened an unlinked file! 2024-04-20T13:14:09.216 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000901) opened an unlinked file! 2024-04-20T13:14:09.216 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000902) opened an unlinked file! 2024-04-20T13:14:09.216 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000903) opened an unlinked file! 2024-04-20T13:14:09.216 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000904) opened an unlinked file! 2024-04-20T13:14:09.217 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000905) opened an unlinked file! 2024-04-20T13:14:09.217 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000906) opened an unlinked file! 2024-04-20T13:14:09.217 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000907) opened an unlinked file! 2024-04-20T13:14:09.217 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000908) opened an unlinked file! 2024-04-20T13:14:09.217 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000909) opened an unlinked file! 2024-04-20T13:14:09.217 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000910) opened an unlinked file! 2024-04-20T13:14:09.217 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000911) opened an unlinked file! 2024-04-20T13:14:09.217 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000912) opened an unlinked file! 2024-04-20T13:14:09.217 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000913) opened an unlinked file! 2024-04-20T13:14:09.217 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000914) opened an unlinked file! 2024-04-20T13:14:09.217 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000915) opened an unlinked file! 2024-04-20T13:14:09.217 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000916) opened an unlinked file! 2024-04-20T13:14:09.217 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000917) opened an unlinked file! 2024-04-20T13:14:09.218 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000918) opened an unlinked file! 2024-04-20T13:14:09.218 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000919) opened an unlinked file! 2024-04-20T13:14:09.218 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000920) opened an unlinked file! 2024-04-20T13:14:09.218 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000921) opened an unlinked file! 2024-04-20T13:14:09.218 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000922) opened an unlinked file! 2024-04-20T13:14:09.218 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000923) opened an unlinked file! 2024-04-20T13:14:09.218 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000924) opened an unlinked file! 2024-04-20T13:14:09.218 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000925) opened an unlinked file! 2024-04-20T13:14:09.218 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000926) opened an unlinked file! 2024-04-20T13:14:09.218 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000927) opened an unlinked file! 2024-04-20T13:14:09.218 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000928) opened an unlinked file! 2024-04-20T13:14:09.218 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000929) opened an unlinked file! 2024-04-20T13:14:09.218 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000930) opened an unlinked file! 2024-04-20T13:14:09.218 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000931) opened an unlinked file! 2024-04-20T13:14:09.219 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000932) opened an unlinked file! 2024-04-20T13:14:09.219 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000933) opened an unlinked file! 2024-04-20T13:14:09.219 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000934) opened an unlinked file! 2024-04-20T13:14:09.219 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000935) opened an unlinked file! 2024-04-20T13:14:09.219 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000936) opened an unlinked file! 2024-04-20T13:14:09.219 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000937) opened an unlinked file! 2024-04-20T13:14:09.219 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000938) opened an unlinked file! 2024-04-20T13:14:09.219 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000939) opened an unlinked file! 2024-04-20T13:14:09.219 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000940) opened an unlinked file! 2024-04-20T13:14:09.219 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000941) opened an unlinked file! 2024-04-20T13:14:09.219 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000942) opened an unlinked file! 2024-04-20T13:14:09.219 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000943) opened an unlinked file! 2024-04-20T13:14:09.219 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000944) opened an unlinked file! 2024-04-20T13:14:09.219 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000945) opened an unlinked file! 2024-04-20T13:14:09.219 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000946) opened an unlinked file! 2024-04-20T13:14:09.220 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000947) opened an unlinked file! 2024-04-20T13:14:09.220 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000948) opened an unlinked file! 2024-04-20T13:14:09.220 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000949) opened an unlinked file! 2024-04-20T13:14:09.220 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000950) opened an unlinked file! 2024-04-20T13:14:09.220 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000951) opened an unlinked file! 2024-04-20T13:14:09.220 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000952) opened an unlinked file! 2024-04-20T13:14:09.220 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000953) opened an unlinked file! 2024-04-20T13:14:09.220 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000954) opened an unlinked file! 2024-04-20T13:14:09.220 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000955) opened an unlinked file! 2024-04-20T13:14:09.220 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000956) opened an unlinked file! 2024-04-20T13:14:09.220 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000957) opened an unlinked file! 2024-04-20T13:14:09.220 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000958) opened an unlinked file! 2024-04-20T13:14:09.220 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000959) opened an unlinked file! 2024-04-20T13:14:09.220 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000960) opened an unlinked file! 2024-04-20T13:14:09.221 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000961) opened an unlinked file! 2024-04-20T13:14:09.221 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000962) opened an unlinked file! 2024-04-20T13:14:09.221 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000963) opened an unlinked file! 2024-04-20T13:14:09.221 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000964) opened an unlinked file! 2024-04-20T13:14:09.221 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000965) opened an unlinked file! 2024-04-20T13:14:09.221 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000966) opened an unlinked file! 2024-04-20T13:14:09.221 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000967) opened an unlinked file! 2024-04-20T13:14:09.221 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000968) opened an unlinked file! 2024-04-20T13:14:09.221 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000969) opened an unlinked file! 2024-04-20T13:14:09.221 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000970) opened an unlinked file! 2024-04-20T13:14:09.221 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000971) opened an unlinked file! 2024-04-20T13:14:09.221 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000972) opened an unlinked file! 2024-04-20T13:14:09.221 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000973) opened an unlinked file! 2024-04-20T13:14:09.221 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000974) opened an unlinked file! 2024-04-20T13:14:09.221 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000975) opened an unlinked file! 2024-04-20T13:14:09.222 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000976) opened an unlinked file! 2024-04-20T13:14:09.222 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000977) opened an unlinked file! 2024-04-20T13:14:09.222 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000978) opened an unlinked file! 2024-04-20T13:14:09.222 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000979) opened an unlinked file! 2024-04-20T13:14:09.222 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000980) opened an unlinked file! 2024-04-20T13:14:09.222 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000981) opened an unlinked file! 2024-04-20T13:14:09.222 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000982) opened an unlinked file! 2024-04-20T13:14:09.222 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000983) opened an unlinked file! 2024-04-20T13:14:09.222 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000984) opened an unlinked file! 2024-04-20T13:14:09.222 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000985) opened an unlinked file! 2024-04-20T13:14:09.222 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000986) opened an unlinked file! 2024-04-20T13:14:09.222 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000987) opened an unlinked file! 2024-04-20T13:14:09.222 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000988) opened an unlinked file! 2024-04-20T13:14:09.222 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000989) opened an unlinked file! 2024-04-20T13:14:09.222 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000990) opened an unlinked file! 2024-04-20T13:14:09.223 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000991) opened an unlinked file! 2024-04-20T13:14:09.223 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000992) opened an unlinked file! 2024-04-20T13:14:09.223 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000993) opened an unlinked file! 2024-04-20T13:14:09.223 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000994) opened an unlinked file! 2024-04-20T13:14:09.223 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000995) opened an unlinked file! 2024-04-20T13:14:09.223 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000996) opened an unlinked file! 2024-04-20T13:14:09.223 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000997) opened an unlinked file! 2024-04-20T13:14:09.223 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000998) opened an unlinked file! 2024-04-20T13:14:09.223 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file000999) opened an unlinked file! 2024-04-20T13:14:09.223 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file001000) opened an unlinked file! 2024-04-20T13:14:09.223 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file001001) opened an unlinked file! 2024-04-20T13:14:09.223 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file001002) opened an unlinked file! 2024-04-20T13:14:09.223 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file001003) opened an unlinked file! 2024-04-20T13:14:09.223 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file001004) opened an unlinked file! 2024-04-20T13:14:09.224 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file001005) opened an unlinked file! 2024-04-20T13:14:09.224 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file001006) opened an unlinked file! 2024-04-20T13:14:09.224 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file001007) opened an unlinked file! 2024-04-20T13:14:09.224 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file001008) opened an unlinked file! 2024-04-20T13:14:09.224 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file001009) opened an unlinked file! 2024-04-20T13:14:09.224 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file001010) opened an unlinked file! 2024-04-20T13:14:09.224 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file001011) opened an unlinked file! 2024-04-20T13:14:09.224 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file001012) opened an unlinked file! 2024-04-20T13:14:09.224 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file001013) opened an unlinked file! 2024-04-20T13:14:09.224 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file001014) opened an unlinked file! 2024-04-20T13:14:09.224 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file001015) opened an unlinked file! 2024-04-20T13:14:09.224 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file001016) opened an unlinked file! 2024-04-20T13:14:09.224 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file001017) opened an unlinked file! 2024-04-20T13:14:09.224 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file001018) opened an unlinked file! 2024-04-20T13:14:09.225 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file001019) opened an unlinked file! 2024-04-20T13:14:09.225 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file001020) opened an unlinked file! 2024-04-20T13:14:09.225 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file001021) opened an unlinked file! 2024-04-20T13:14:09.225 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file001022) opened an unlinked file! 2024-04-20T13:14:09.225 INFO:teuthology.orchestra.run.smithi121.stdout: +open_by_handle(/tmp/tmp.ru8ajEsoUutest/426-dir/file001023) opened an unlinked file! 2024-04-20T13:14:09.225 INFO:teuthology.orchestra.run.smithi121.stdout: test_file_handles TEST_DIR/426-dir 2024-04-20T13:14:09.225 INFO:teuthology.orchestra.run.smithi121.stdout: test_file_handles TEST_DIR/426-dir -l 2024-04-20T13:14:09.225 INFO:teuthology.orchestra.run.smithi121.stdout: test_file_handles TEST_DIR/426-dir -u 2024-04-20T13:14:09.226 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/426 2024-04-20T13:14:09.226 INFO:teuthology.orchestra.run.smithi121.stdout:Failures: generic/426 2024-04-20T13:14:09.226 INFO:teuthology.orchestra.run.smithi121.stdout:Failed 1 of 1 tests 2024-04-20T13:14:09.259 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:14:09.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T13:14:09.385 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-20T13:14:09.385 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-20T13:14:09.386 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/427) 2024-04-20T13:14:10.952 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:14:10.965 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:14:10.965 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:14:11.455 INFO:teuthology.orchestra.run.smithi121.stdout:generic/427 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:14:11.466 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/427 2024-04-20T13:14:11.466 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/427 2024-04-20T13:14:11.466 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:14:11.475 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:14:11.503 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:14:11.503 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/428) 2024-04-20T13:14:12.023 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:14:12.037 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:14:12.037 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:14:12.376 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:14:12.384 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:14:12.626 INFO:teuthology.orchestra.run.smithi121.stdout:generic/428 0s 2024-04-20T13:14:12.643 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/428 2024-04-20T13:14:12.643 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:14:12.652 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:14:12.678 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:14:12.678 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/429) 2024-04-20T13:14:13.228 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:14:13.241 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:14:13.242 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:14:13.771 INFO:teuthology.orchestra.run.smithi121.stdout:generic/429 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:14:13.786 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/429 2024-04-20T13:14:13.786 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/429 2024-04-20T13:14:13.786 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:14:13.797 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:14:13.826 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:14:13.827 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/430) 2024-04-20T13:14:14.387 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:14:14.400 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:14:14.400 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:14:15.223 INFO:teuthology.orchestra.run.smithi121.stdout:generic/430 1s 2024-04-20T13:14:15.240 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/430 2024-04-20T13:14:15.241 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:14:15.250 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:14:15.282 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:14:15.283 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/431) 2024-04-20T13:14:15.975 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:14:15.989 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:14:15.989 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:14:16.796 INFO:teuthology.orchestra.run.smithi121.stdout:generic/431 0s 2024-04-20T13:14:16.812 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/431 2024-04-20T13:14:16.812 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:14:16.821 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:14:16.847 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:14:16.847 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/432) 2024-04-20T13:14:17.536 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:14:17.550 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:14:17.551 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:14:18.315 INFO:teuthology.orchestra.run.smithi121.stdout:generic/432 1s 2024-04-20T13:14:18.330 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/432 2024-04-20T13:14:18.330 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:14:18.338 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:14:18.365 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:14:18.366 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/433) 2024-04-20T13:14:19.063 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:14:19.076 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:14:19.076 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:14:19.792 INFO:teuthology.orchestra.run.smithi121.stdout:generic/433 0s 2024-04-20T13:14:19.832 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/433 2024-04-20T13:14:19.833 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:14:19.845 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:14:19.874 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:14:19.874 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/434) 2024-04-20T13:14:20.578 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:14:20.589 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:14:20.589 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:14:21.324 INFO:teuthology.orchestra.run.smithi121.stdout:generic/434 - output mismatch (see /tmp/tmp.yONpfXTHudxfstests-dev/results//generic/434.out.bad) 2024-04-20T13:14:21.331 INFO:teuthology.orchestra.run.smithi121.stdout: --- tests/generic/434.out 2024-04-20 10:23:05.954112317 +0000 2024-04-20T13:14:21.331 INFO:teuthology.orchestra.run.smithi121.stdout: +++ /tmp/tmp.yONpfXTHudxfstests-dev/results//generic/434.out.bad 2024-04-20 13:14:21.321287444 +0000 2024-04-20T13:14:21.331 INFO:teuthology.orchestra.run.smithi121.stdout: @@ -9,6 +9,6 @@ 2024-04-20T13:14:21.332 INFO:teuthology.orchestra.run.smithi121.stdout: copy_range: Bad file descriptor 2024-04-20T13:14:21.332 INFO:teuthology.orchestra.run.smithi121.stdout: d41d8cd98f00b204e9800998ecf8427e TEST_DIR/test-434/copy 2024-04-20T13:14:21.332 INFO:teuthology.orchestra.run.smithi121.stdout: Try to copy to a device 2024-04-20T13:14:21.332 INFO:teuthology.orchestra.run.smithi121.stdout: -copy_range: Invalid argument 2024-04-20T13:14:21.332 INFO:teuthology.orchestra.run.smithi121.stdout: +/tmp/tmp.ru8ajEsoUutest/test-434/dev1: Permission denied 2024-04-20T13:14:21.332 INFO:teuthology.orchestra.run.smithi121.stdout: Try to copy to a fifo 2024-04-20T13:14:21.332 INFO:teuthology.orchestra.run.smithi121.stdout: copy_range: Invalid argument 2024-04-20T13:14:21.339 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/434 2024-04-20T13:14:21.339 INFO:teuthology.orchestra.run.smithi121.stdout:Failures: generic/434 2024-04-20T13:14:21.339 INFO:teuthology.orchestra.run.smithi121.stdout:Failed 1 of 1 tests 2024-04-20T13:14:21.347 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:14:21.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T13:14:21.378 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-20T13:14:21.378 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-20T13:14:21.378 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/435) 2024-04-20T13:14:22.078 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:14:22.089 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:14:22.089 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:14:22.620 INFO:teuthology.orchestra.run.smithi121.stdout:generic/435 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:14:22.631 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/435 2024-04-20T13:14:22.631 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/435 2024-04-20T13:14:22.631 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:14:22.641 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:14:22.670 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:14:22.671 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/436) 2024-04-20T13:14:23.232 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:14:23.245 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:14:23.245 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:14:26.001 INFO:teuthology.orchestra.run.smithi121.stdout:generic/436 2s 2024-04-20T13:14:26.020 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/436 2024-04-20T13:14:26.020 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:14:26.029 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:14:26.062 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:14:26.062 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/437) 2024-04-20T13:14:26.770 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:14:26.782 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:14:26.782 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:14:28.129 INFO:teuthology.orchestra.run.smithi121.stdout:generic/437 2s 2024-04-20T13:14:28.145 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/437 2024-04-20T13:14:28.145 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:14:28.154 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:14:28.183 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:14:28.183 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/438) 2024-04-20T13:14:28.726 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:14:28.739 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:14:28.739 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:14:42.585 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:14:42.590 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:15:12.874 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:15:12.883 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:15:43.011 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:15:43.017 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:16:13.517 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:16:13.527 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:16:43.675 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:16:43.681 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:17:13.801 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:17:13.807 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:17:43.924 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:17:43.930 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:17:46.608 INFO:teuthology.orchestra.run.smithi121.stdout:generic/438 198s 2024-04-20T13:17:46.627 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/438 2024-04-20T13:17:46.627 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:17:46.637 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:17:46.668 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:17:46.669 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/439) 2024-04-20T13:17:47.202 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:17:47.214 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:17:47.214 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:17:47.648 INFO:teuthology.orchestra.run.smithi121.stdout:generic/439 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:17:47.658 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/439 2024-04-20T13:17:47.658 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/439 2024-04-20T13:17:47.659 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:17:47.666 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:17:47.693 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:17:47.693 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/440) 2024-04-20T13:17:48.214 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:17:48.227 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:17:48.227 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:17:48.737 INFO:teuthology.orchestra.run.smithi121.stdout:generic/440 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:17:48.750 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/440 2024-04-20T13:17:48.750 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/440 2024-04-20T13:17:48.750 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:17:48.760 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:17:48.791 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:17:48.791 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/441) 2024-04-20T13:17:49.334 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:17:49.348 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:17:49.348 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:17:49.921 INFO:teuthology.orchestra.run.smithi121.stdout:generic/441 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:17:49.936 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/441 2024-04-20T13:17:49.936 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/441 2024-04-20T13:17:49.936 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:17:49.948 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:17:49.983 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:17:49.983 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/442) 2024-04-20T13:17:50.537 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:17:50.551 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:17:50.551 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:17:51.014 INFO:teuthology.orchestra.run.smithi121.stdout:generic/442 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:17:51.026 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/442 2024-04-20T13:17:51.027 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/442 2024-04-20T13:17:51.027 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:17:51.037 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:17:51.070 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:17:51.071 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/443) 2024-04-20T13:17:51.628 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:17:51.642 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:17:51.643 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:17:52.158 INFO:teuthology.orchestra.run.smithi121.stdout:generic/443 1s 2024-04-20T13:17:52.176 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/443 2024-04-20T13:17:52.176 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:17:52.185 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:17:52.215 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:17:52.215 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/444) 2024-04-20T13:17:52.708 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:17:52.720 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:17:52.720 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:17:53.261 INFO:teuthology.orchestra.run.smithi121.stdout:generic/444 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-04-20T13:17:53.271 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/444 2024-04-20T13:17:53.271 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/444 2024-04-20T13:17:53.271 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:17:53.280 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:17:53.306 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:17:53.307 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/445) 2024-04-20T13:17:53.836 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:17:53.848 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:17:53.848 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:17:56.620 INFO:teuthology.orchestra.run.smithi121.stdout:generic/445 3s 2024-04-20T13:17:56.637 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/445 2024-04-20T13:17:56.637 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:17:56.646 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:17:56.677 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:17:56.677 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/446) 2024-04-20T13:17:57.386 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:17:57.400 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:17:57.400 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:17:57.820 INFO:teuthology.orchestra.run.smithi121.stdout:generic/446 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:17:57.833 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/446 2024-04-20T13:17:57.833 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/446 2024-04-20T13:17:57.833 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:17:57.842 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:17:57.873 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:17:57.873 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/447) 2024-04-20T13:17:58.378 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:17:58.391 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:17:58.391 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:17:58.791 INFO:teuthology.orchestra.run.smithi121.stdout:generic/447 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:17:58.802 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/447 2024-04-20T13:17:58.802 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/447 2024-04-20T13:17:58.802 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:17:58.812 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:17:58.841 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:17:58.842 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/448) 2024-04-20T13:17:59.386 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:17:59.397 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:17:59.397 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:18:02.056 INFO:teuthology.orchestra.run.smithi121.stdout:generic/448 3s 2024-04-20T13:18:02.076 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/448 2024-04-20T13:18:02.077 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:18:02.088 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:18:02.139 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:18:02.139 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/449) 2024-04-20T13:18:02.851 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:18:02.862 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:18:02.863 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:18:03.288 INFO:teuthology.orchestra.run.smithi121.stdout:generic/449 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:18:03.300 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/449 2024-04-20T13:18:03.300 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/449 2024-04-20T13:18:03.300 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:18:03.311 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:18:03.341 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:18:03.341 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/450) 2024-04-20T13:18:03.914 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:18:03.928 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:18:03.928 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:18:07.811 INFO:teuthology.orchestra.run.smithi121.stdout:generic/450 4s 2024-04-20T13:18:07.829 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/450 2024-04-20T13:18:07.830 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:18:07.839 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:18:07.889 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:18:07.890 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/451) 2024-04-20T13:18:08.568 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:18:08.581 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:18:08.581 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:18:09.454 INFO:teuthology.orchestra.run.smithi121.stdout:generic/451 - output mismatch (see /tmp/tmp.yONpfXTHudxfstests-dev/results//generic/451.out.bad) 2024-04-20T13:18:09.461 INFO:teuthology.orchestra.run.smithi121.stdout: --- tests/generic/451.out 2024-04-20 10:23:05.957112278 +0000 2024-04-20T13:18:09.461 INFO:teuthology.orchestra.run.smithi121.stdout: +++ /tmp/tmp.yONpfXTHudxfstests-dev/results//generic/451.out.bad 2024-04-20 13:18:09.451348658 +0000 2024-04-20T13:18:09.461 INFO:teuthology.orchestra.run.smithi121.stdout: @@ -1,2 +1,5 @@ 2024-04-20T13:18:09.461 INFO:teuthology.orchestra.run.smithi121.stdout: QA output created by 451 2024-04-20T13:18:09.461 INFO:teuthology.orchestra.run.smithi121.stdout: +get stale data from buffer read 2024-04-20T13:18:09.461 INFO:teuthology.orchestra.run.smithi121.stdout: +00000000 55 55 55 55 55 55 55 55 55 55 55 55 55 55 55 55 UUUUUUUUUUUUUUUU 2024-04-20T13:18:09.461 INFO:teuthology.orchestra.run.smithi121.stdout: +000a0000 2024-04-20T13:18:09.461 INFO:teuthology.orchestra.run.smithi121.stdout: Silence is golden 2024-04-20T13:18:09.467 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/451 2024-04-20T13:18:09.468 INFO:teuthology.orchestra.run.smithi121.stdout:Failures: generic/451 2024-04-20T13:18:09.468 INFO:teuthology.orchestra.run.smithi121.stdout:Failed 1 of 1 tests 2024-04-20T13:18:09.475 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:18:09.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T13:18:09.502 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-20T13:18:09.503 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-20T13:18:09.503 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/452) 2024-04-20T13:18:10.320 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:18:10.338 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:18:10.338 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:18:10.642 INFO:teuthology.orchestra.run.smithi121.stdout:generic/452 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:18:10.653 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/452 2024-04-20T13:18:10.653 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/452 2024-04-20T13:18:10.653 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:18:10.663 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:18:10.693 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:18:10.693 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/453) 2024-04-20T13:18:11.196 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:18:11.208 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:18:11.208 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:18:11.625 INFO:teuthology.orchestra.run.smithi121.stdout:generic/453 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:18:11.636 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/453 2024-04-20T13:18:11.636 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/453 2024-04-20T13:18:11.636 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:18:11.645 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:18:11.673 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:18:11.674 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/454) 2024-04-20T13:18:12.227 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:18:12.240 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:18:12.240 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:18:12.644 INFO:teuthology.orchestra.run.smithi121.stdout:generic/454 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:18:12.656 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/454 2024-04-20T13:18:12.656 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/454 2024-04-20T13:18:12.656 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:18:12.666 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:18:12.696 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:18:12.696 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/455) 2024-04-20T13:18:13.227 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:18:13.241 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:18:13.241 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:18:13.830 INFO:teuthology.orchestra.run.smithi121.stdout:generic/455 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:18:13.844 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/455 2024-04-20T13:18:13.844 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/455 2024-04-20T13:18:13.844 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:18:13.856 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:18:13.891 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:18:13.891 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/456) 2024-04-20T13:18:14.071 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:18:14.080 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:18:14.383 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:18:14.395 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:18:14.395 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:18:14.859 INFO:teuthology.orchestra.run.smithi121.stdout:generic/456 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:18:14.872 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/456 2024-04-20T13:18:14.872 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/456 2024-04-20T13:18:14.872 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:18:14.883 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:18:14.910 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:18:14.911 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/457) 2024-04-20T13:18:15.433 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:18:15.447 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:18:15.448 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:18:16.005 INFO:teuthology.orchestra.run.smithi121.stdout:generic/457 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:18:16.021 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/457 2024-04-20T13:18:16.021 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/457 2024-04-20T13:18:16.021 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:18:16.032 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:18:16.066 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:18:16.067 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/458) 2024-04-20T13:18:16.604 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:18:16.617 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:18:16.617 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:18:17.086 INFO:teuthology.orchestra.run.smithi121.stdout:generic/458 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:18:17.100 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/458 2024-04-20T13:18:17.100 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/458 2024-04-20T13:18:17.100 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:18:17.112 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:18:17.145 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:18:17.145 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/459) 2024-04-20T13:18:17.692 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:18:17.706 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:18:17.706 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:18:18.347 INFO:teuthology.orchestra.run.smithi121.stdout:generic/459 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:18:18.359 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/459 2024-04-20T13:18:18.359 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/459 2024-04-20T13:18:18.360 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:18:18.371 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:18:18.405 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:18:18.405 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/460) 2024-04-20T13:18:18.956 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:18:18.970 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:18:18.970 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:18:19.394 INFO:teuthology.orchestra.run.smithi121.stdout:generic/460 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:18:19.407 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/460 2024-04-20T13:18:19.407 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/460 2024-04-20T13:18:19.407 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:18:19.420 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:18:19.450 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:18:19.450 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/461) 2024-04-20T13:18:19.976 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:18:19.989 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:18:19.989 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:18:20.408 INFO:teuthology.orchestra.run.smithi121.stdout:generic/461 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:18:20.418 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/461 2024-04-20T13:18:20.419 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/461 2024-04-20T13:18:20.419 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:18:20.429 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:18:20.458 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:18:20.458 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/462) 2024-04-20T13:18:20.976 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:18:20.988 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:18:20.988 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:18:21.426 INFO:teuthology.orchestra.run.smithi121.stdout:generic/462 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:18:21.437 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/462 2024-04-20T13:18:21.438 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/462 2024-04-20T13:18:21.438 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:18:21.447 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:18:21.477 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:18:21.478 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/463) 2024-04-20T13:18:21.996 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:18:22.009 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:18:22.009 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:18:22.583 INFO:teuthology.orchestra.run.smithi121.stdout:generic/463 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-20T13:18:22.594 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/463 2024-04-20T13:18:22.594 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/463 2024-04-20T13:18:22.595 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:18:22.603 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:18:22.643 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:18:22.644 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/464) 2024-04-20T13:18:23.347 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:18:23.358 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:18:23.358 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:18:23.784 INFO:teuthology.orchestra.run.smithi121.stdout:generic/464 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:18:23.795 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/464 2024-04-20T13:18:23.795 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/464 2024-04-20T13:18:23.795 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:18:23.805 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:18:23.834 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:18:23.834 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/465) 2024-04-20T13:18:24.341 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:18:24.354 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:18:24.354 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:18:44.464 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:18:44.473 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:18:54.144 INFO:teuthology.orchestra.run.smithi121.stdout:generic/465 30s 2024-04-20T13:18:54.180 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/465 2024-04-20T13:18:54.184 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:18:54.199 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:18:54.241 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:18:54.242 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/466) 2024-04-20T13:18:54.843 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:18:54.857 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:18:54.857 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:18:55.181 INFO:teuthology.orchestra.run.smithi121.stdout:generic/466 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:18:55.193 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/466 2024-04-20T13:18:55.193 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/466 2024-04-20T13:18:55.193 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:18:55.202 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:18:55.230 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:18:55.230 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/467) 2024-04-20T13:18:55.762 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:18:55.775 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:18:55.775 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:19:04.923 INFO:teuthology.orchestra.run.smithi121.stdout:generic/467 9s 2024-04-20T13:19:05.014 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/467 2024-04-20T13:19:05.014 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:19:05.024 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:19:05.221 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:19:05.221 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/468) 2024-04-20T13:19:06.728 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:19:06.743 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:19:06.743 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:19:07.217 INFO:teuthology.orchestra.run.smithi121.stdout:generic/468 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:19:07.230 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/468 2024-04-20T13:19:07.230 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/468 2024-04-20T13:19:07.231 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:19:07.242 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:19:07.276 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:19:07.276 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/469) 2024-04-20T13:19:07.801 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:19:07.815 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:19:07.815 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:19:14.582 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:19:14.592 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:19:15.183 INFO:teuthology.orchestra.run.smithi121.stdout:generic/469 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-04-20T13:19:15.198 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/469 2024-04-20T13:19:15.198 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/469 2024-04-20T13:19:15.198 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:19:15.209 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:19:15.253 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:19:15.253 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/470) 2024-04-20T13:19:15.950 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:19:15.961 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:19:15.961 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:19:16.439 INFO:teuthology.orchestra.run.smithi121.stdout:generic/470 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:19:16.451 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/470 2024-04-20T13:19:16.451 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/470 2024-04-20T13:19:16.451 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:19:16.461 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:19:16.490 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:19:16.490 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/471) 2024-04-20T13:19:17.002 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:19:17.015 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:19:17.015 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:19:44.862 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:19:44.872 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:20:15.018 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:20:15.027 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:20:45.198 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:20:45.205 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:21:15.867 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:21:15.872 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:21:37.798 INFO:teuthology.orchestra.run.smithi121.stdout:generic/471 140s 2024-04-20T13:21:37.812 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/471 2024-04-20T13:21:37.812 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:21:37.818 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:21:37.872 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:21:37.872 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/472) 2024-04-20T13:21:38.536 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:21:38.551 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:21:38.551 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:21:39.114 INFO:teuthology.orchestra.run.smithi121.stdout:generic/472 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:21:39.132 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/472 2024-04-20T13:21:39.133 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/472 2024-04-20T13:21:39.133 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:21:39.145 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:21:39.196 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:21:39.197 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/473) 2024-04-20T13:21:39.671 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:21:39.683 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:21:39.683 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:21:40.256 INFO:teuthology.orchestra.run.smithi121.stdout:generic/473 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-20T13:21:40.268 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/473 2024-04-20T13:21:40.268 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/473 2024-04-20T13:21:40.268 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:21:40.277 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:21:40.310 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:21:40.311 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/474) 2024-04-20T13:21:40.852 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:21:40.865 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:21:40.865 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:21:41.308 INFO:teuthology.orchestra.run.smithi121.stdout:generic/474 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:21:41.322 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/474 2024-04-20T13:21:41.322 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/474 2024-04-20T13:21:41.322 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:21:41.335 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:21:41.366 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:21:41.366 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/475) 2024-04-20T13:21:41.891 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:21:41.904 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:21:41.904 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:21:42.447 INFO:teuthology.orchestra.run.smithi121.stdout:generic/475 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:21:42.461 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/475 2024-04-20T13:21:42.461 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/475 2024-04-20T13:21:42.461 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:21:42.473 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:21:42.508 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:21:42.508 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/476) 2024-04-20T13:21:43.029 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:21:43.043 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:21:43.043 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:21:43.489 INFO:teuthology.orchestra.run.smithi121.stdout:generic/476 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:21:43.504 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/476 2024-04-20T13:21:43.504 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/476 2024-04-20T13:21:43.504 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:21:43.515 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:21:43.549 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:21:43.549 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/477) 2024-04-20T13:21:44.097 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:21:44.112 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:21:44.112 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:21:45.990 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:21:45.998 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:21:58.122 INFO:teuthology.orchestra.run.smithi121.stdout:generic/477 13s 2024-04-20T13:21:58.275 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/477 2024-04-20T13:21:58.275 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:21:58.308 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:21:58.390 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:21:58.390 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/478) 2024-04-20T13:21:59.867 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:21:59.879 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:21:59.879 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:12.249 INFO:teuthology.orchestra.run.smithi121.stdout:generic/478 - output mismatch (see /tmp/tmp.yONpfXTHudxfstests-dev/results//generic/478.out.bad) 2024-04-20T13:22:12.258 INFO:teuthology.orchestra.run.smithi121.stdout: --- tests/generic/478.out 2024-04-20 10:23:05.962112212 +0000 2024-04-20T13:22:12.258 INFO:teuthology.orchestra.run.smithi121.stdout: +++ /tmp/tmp.yONpfXTHudxfstests-dev/results//generic/478.out.bad 2024-04-20 13:22:12.240283555 +0000 2024-04-20T13:22:12.258 INFO:teuthology.orchestra.run.smithi121.stdout: @@ -10,7 +10,7 @@ 2024-04-20T13:22:12.258 INFO:teuthology.orchestra.run.smithi121.stdout: lock could be placed 2024-04-20T13:22:12.259 INFO:teuthology.orchestra.run.smithi121.stdout: get wrlck 2024-04-20T13:22:12.259 INFO:teuthology.orchestra.run.smithi121.stdout: lock could be placed 2024-04-20T13:22:12.259 INFO:teuthology.orchestra.run.smithi121.stdout: -lock could be placed 2024-04-20T13:22:12.259 INFO:teuthology.orchestra.run.smithi121.stdout: +get wrlck 2024-04-20T13:22:12.259 INFO:teuthology.orchestra.run.smithi121.stdout: lock could be placed 2024-04-20T13:22:12.259 INFO:teuthology.orchestra.run.smithi121.stdout: lock could be placed 2024-04-20T13:22:12.259 INFO:teuthology.orchestra.run.smithi121.stdout: lock could be placed 2024-04-20T13:22:12.259 INFO:teuthology.orchestra.run.smithi121.stdout: @@ -25,7 +25,7 @@ 2024-04-20T13:22:12.259 INFO:teuthology.orchestra.run.smithi121.stdout: lock could be placed 2024-04-20T13:22:12.259 INFO:teuthology.orchestra.run.smithi121.stdout: get wrlck 2024-04-20T13:22:12.259 INFO:teuthology.orchestra.run.smithi121.stdout: lock could be placed 2024-04-20T13:22:12.260 INFO:teuthology.orchestra.run.smithi121.stdout: -lock could be placed 2024-04-20T13:22:12.260 INFO:teuthology.orchestra.run.smithi121.stdout: +get wrlck 2024-04-20T13:22:12.260 INFO:teuthology.orchestra.run.smithi121.stdout: lock could be placed 2024-04-20T13:22:12.260 INFO:teuthology.orchestra.run.smithi121.stdout: lock could be placed 2024-04-20T13:22:12.260 INFO:teuthology.orchestra.run.smithi121.stdout: lock could be placed 2024-04-20T13:22:12.260 INFO:teuthology.orchestra.run.smithi121.stdout: @@ -37,7 +37,7 @@ 2024-04-20T13:22:12.260 INFO:teuthology.orchestra.run.smithi121.stdout: get rdlck 2024-04-20T13:22:12.260 INFO:teuthology.orchestra.run.smithi121.stdout: get rdlck 2024-04-20T13:22:12.260 INFO:teuthology.orchestra.run.smithi121.stdout: lock could be placed 2024-04-20T13:22:12.261 INFO:teuthology.orchestra.run.smithi121.stdout: -lock could be placed 2024-04-20T13:22:12.261 INFO:teuthology.orchestra.run.smithi121.stdout: +get rdlck 2024-04-20T13:22:12.261 INFO:teuthology.orchestra.run.smithi121.stdout: get rdlck 2024-04-20T13:22:12.261 INFO:teuthology.orchestra.run.smithi121.stdout: lock could be placed 2024-04-20T13:22:12.261 INFO:teuthology.orchestra.run.smithi121.stdout: get rdlck 2024-04-20T13:22:12.261 INFO:teuthology.orchestra.run.smithi121.stdout: @@ -46,7 +46,7 @@ 2024-04-20T13:22:12.261 INFO:teuthology.orchestra.run.smithi121.stdout: get rdlck 2024-04-20T13:22:12.261 INFO:teuthology.orchestra.run.smithi121.stdout: get rdlck 2024-04-20T13:22:12.261 INFO:teuthology.orchestra.run.smithi121.stdout: lock could be placed 2024-04-20T13:22:12.261 INFO:teuthology.orchestra.run.smithi121.stdout: -lock could be placed 2024-04-20T13:22:12.261 INFO:teuthology.orchestra.run.smithi121.stdout: +get rdlck 2024-04-20T13:22:12.261 INFO:teuthology.orchestra.run.smithi121.stdout: lock could be placed 2024-04-20T13:22:12.262 INFO:teuthology.orchestra.run.smithi121.stdout: lock could be placed 2024-04-20T13:22:12.262 INFO:teuthology.orchestra.run.smithi121.stdout: lock could be placed 2024-04-20T13:22:12.267 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/478 2024-04-20T13:22:12.267 INFO:teuthology.orchestra.run.smithi121.stdout:Failures: generic/478 2024-04-20T13:22:12.267 INFO:teuthology.orchestra.run.smithi121.stdout:Failed 1 of 1 tests 2024-04-20T13:22:12.278 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:12.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T13:22:12.312 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-20T13:22:12.312 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-20T13:22:12.312 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/479) 2024-04-20T13:22:12.834 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:22:12.848 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:22:12.848 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:13.328 INFO:teuthology.orchestra.run.smithi121.stdout:generic/479 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:22:13.339 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/479 2024-04-20T13:22:13.339 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/479 2024-04-20T13:22:13.339 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:22:13.349 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:13.385 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:22:13.385 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/480) 2024-04-20T13:22:13.926 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:22:13.939 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:22:13.939 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:14.381 INFO:teuthology.orchestra.run.smithi121.stdout:generic/480 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:22:14.393 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/480 2024-04-20T13:22:14.393 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/480 2024-04-20T13:22:14.394 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:22:14.406 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:14.437 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:22:14.437 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/481) 2024-04-20T13:22:14.989 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:22:15.002 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:22:15.002 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:15.465 INFO:teuthology.orchestra.run.smithi121.stdout:generic/481 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:22:15.479 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/481 2024-04-20T13:22:15.479 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/481 2024-04-20T13:22:15.479 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:22:15.490 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:15.523 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:22:15.523 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/482) 2024-04-20T13:22:16.064 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:22:16.076 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:22:16.076 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:16.615 INFO:teuthology.orchestra.run.smithi121.stdout:generic/482 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:22:16.630 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/482 2024-04-20T13:22:16.630 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/482 2024-04-20T13:22:16.630 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:22:16.642 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:16.675 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:22:16.675 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/483) 2024-04-20T13:22:17.203 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:22:17.219 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:22:17.219 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:17.471 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:22:17.477 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:22:17.647 INFO:teuthology.orchestra.run.smithi121.stdout:generic/483 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:22:17.659 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/483 2024-04-20T13:22:17.659 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/483 2024-04-20T13:22:17.660 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:22:17.670 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:17.703 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:22:17.704 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/484) 2024-04-20T13:22:18.300 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:22:18.315 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:22:18.315 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:18.795 INFO:teuthology.orchestra.run.smithi121.stdout:generic/484 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:22:18.807 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/484 2024-04-20T13:22:18.808 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/484 2024-04-20T13:22:18.808 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:22:18.816 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:18.848 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:22:18.848 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/485) 2024-04-20T13:22:19.355 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:22:19.368 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:22:19.369 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:22.597 INFO:teuthology.orchestra.run.smithi121.stdout:generic/485 [not run] xfs_io finsert failed (old kernel/wrong fs?) 2024-04-20T13:22:22.611 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/485 2024-04-20T13:22:22.611 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/485 2024-04-20T13:22:22.611 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:22:22.622 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:22.686 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:22:22.686 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/486) 2024-04-20T13:22:23.357 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:22:23.369 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:22:23.369 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:23.819 INFO:teuthology.orchestra.run.smithi121.stdout:generic/486 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:22:23.830 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/486 2024-04-20T13:22:23.830 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/486 2024-04-20T13:22:23.830 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:22:23.839 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:23.868 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:22:23.868 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/487) 2024-04-20T13:22:24.378 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:22:24.391 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:22:24.392 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:24.845 INFO:teuthology.orchestra.run.smithi121.stdout:generic/487 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:22:24.859 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/487 2024-04-20T13:22:24.859 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/487 2024-04-20T13:22:24.859 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:22:24.870 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:24.901 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:22:24.902 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/488) 2024-04-20T13:22:25.432 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:22:25.445 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:22:25.445 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:25.860 INFO:teuthology.orchestra.run.smithi121.stdout:generic/488 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:22:25.871 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/488 2024-04-20T13:22:25.871 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/488 2024-04-20T13:22:25.871 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:22:25.880 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:25.908 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:22:25.908 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/489) 2024-04-20T13:22:26.428 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:22:26.442 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:22:26.442 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:26.895 INFO:teuthology.orchestra.run.smithi121.stdout:generic/489 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:22:26.908 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/489 2024-04-20T13:22:26.908 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/489 2024-04-20T13:22:26.908 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:22:26.921 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:26.952 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:22:26.952 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/490) 2024-04-20T13:22:27.486 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:22:27.500 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:22:27.500 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:29.810 INFO:teuthology.orchestra.run.smithi121.stdout:generic/490 2s 2024-04-20T13:22:29.827 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/490 2024-04-20T13:22:29.827 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:22:29.835 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:29.873 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:22:29.874 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/491) 2024-04-20T13:22:30.550 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:22:30.563 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:22:30.584 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:30.971 INFO:teuthology.orchestra.run.smithi121.stdout:generic/491 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:22:30.984 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/491 2024-04-20T13:22:30.984 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/491 2024-04-20T13:22:30.984 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:22:30.995 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:31.025 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:22:31.025 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/492) 2024-04-20T13:22:31.558 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:22:31.570 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:22:31.570 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:31.982 INFO:teuthology.orchestra.run.smithi121.stdout:generic/492 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:22:31.996 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/492 2024-04-20T13:22:31.996 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/492 2024-04-20T13:22:31.996 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:22:32.008 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:32.042 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:22:32.043 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/493) 2024-04-20T13:22:32.568 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:22:32.582 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:22:32.582 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:33.036 INFO:teuthology.orchestra.run.smithi121.stdout:generic/493 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:22:33.046 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/493 2024-04-20T13:22:33.046 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/493 2024-04-20T13:22:33.046 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:22:33.054 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:33.084 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:22:33.085 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/494) 2024-04-20T13:22:33.614 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:22:33.626 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:22:33.626 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:34.031 INFO:teuthology.orchestra.run.smithi121.stdout:generic/494 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:22:34.041 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/494 2024-04-20T13:22:34.041 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/494 2024-04-20T13:22:34.041 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:22:34.050 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:34.079 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:22:34.079 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/495) 2024-04-20T13:22:34.616 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:22:34.629 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:22:34.629 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:35.083 INFO:teuthology.orchestra.run.smithi121.stdout:generic/495 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:22:35.096 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/495 2024-04-20T13:22:35.096 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/495 2024-04-20T13:22:35.096 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:22:35.108 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:35.138 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:22:35.138 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/496) 2024-04-20T13:22:35.662 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:22:35.673 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:22:35.673 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:36.085 INFO:teuthology.orchestra.run.smithi121.stdout:generic/496 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:22:36.099 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/496 2024-04-20T13:22:36.099 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/496 2024-04-20T13:22:36.099 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:22:36.112 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:36.145 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:22:36.146 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/497) 2024-04-20T13:22:36.662 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:22:36.676 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:22:36.676 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:37.106 INFO:teuthology.orchestra.run.smithi121.stdout:generic/497 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:22:37.118 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/497 2024-04-20T13:22:37.118 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/497 2024-04-20T13:22:37.119 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:22:37.131 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:37.162 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:22:37.163 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/498) 2024-04-20T13:22:37.712 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:22:37.726 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:22:37.726 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:38.197 INFO:teuthology.orchestra.run.smithi121.stdout:generic/498 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:22:38.211 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/498 2024-04-20T13:22:38.212 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/498 2024-04-20T13:22:38.212 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:22:38.223 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:38.255 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:22:38.256 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/499) 2024-04-20T13:22:38.774 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:22:38.786 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:22:38.786 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:39.184 INFO:teuthology.orchestra.run.smithi121.stdout:generic/499 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:22:39.195 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/499 2024-04-20T13:22:39.195 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/499 2024-04-20T13:22:39.195 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:22:39.203 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:39.231 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:22:39.231 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/500) 2024-04-20T13:22:39.795 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:22:39.808 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:22:39.808 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:40.346 INFO:teuthology.orchestra.run.smithi121.stdout:generic/500 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:22:40.360 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/500 2024-04-20T13:22:40.361 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/500 2024-04-20T13:22:40.361 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:22:40.372 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:40.408 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:22:40.409 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/501) 2024-04-20T13:22:40.950 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:22:40.963 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:22:40.963 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:41.404 INFO:teuthology.orchestra.run.smithi121.stdout:generic/501 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:22:41.414 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/501 2024-04-20T13:22:41.414 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/501 2024-04-20T13:22:41.414 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:22:41.422 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:41.454 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:22:41.454 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/502) 2024-04-20T13:22:41.960 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:22:41.974 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:22:41.974 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:42.467 INFO:teuthology.orchestra.run.smithi121.stdout:generic/502 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:22:42.482 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/502 2024-04-20T13:22:42.483 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/502 2024-04-20T13:22:42.483 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:22:42.494 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:42.529 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:22:42.529 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/503) 2024-04-20T13:22:43.053 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:22:43.067 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:22:43.067 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:43.496 INFO:teuthology.orchestra.run.smithi121.stdout:generic/503 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:22:43.507 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/503 2024-04-20T13:22:43.507 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/503 2024-04-20T13:22:43.507 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:22:43.516 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:43.543 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:22:43.543 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/504) 2024-04-20T13:22:44.043 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:22:44.056 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:22:44.057 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:44.581 INFO:teuthology.orchestra.run.smithi121.stdout:generic/504 - output mismatch (see /tmp/tmp.yONpfXTHudxfstests-dev/results//generic/504.out.bad) 2024-04-20T13:22:44.589 INFO:teuthology.orchestra.run.smithi121.stdout: --- tests/generic/504.out 2024-04-20 10:23:05.966112160 +0000 2024-04-20T13:22:44.589 INFO:teuthology.orchestra.run.smithi121.stdout: +++ /tmp/tmp.yONpfXTHudxfstests-dev/results//generic/504.out.bad 2024-04-20 13:22:44.577878125 +0000 2024-04-20T13:22:44.589 INFO:teuthology.orchestra.run.smithi121.stdout: @@ -1,2 +1,3 @@ 2024-04-20T13:22:44.589 INFO:teuthology.orchestra.run.smithi121.stdout: QA output created by 504 2024-04-20T13:22:44.589 INFO:teuthology.orchestra.run.smithi121.stdout: +lock info not found 2024-04-20T13:22:44.589 INFO:teuthology.orchestra.run.smithi121.stdout: Silence is golden 2024-04-20T13:22:44.597 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/504 2024-04-20T13:22:44.597 INFO:teuthology.orchestra.run.smithi121.stdout:Failures: generic/504 2024-04-20T13:22:44.597 INFO:teuthology.orchestra.run.smithi121.stdout:Failed 1 of 1 tests 2024-04-20T13:22:44.606 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:44.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T13:22:44.634 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-20T13:22:44.635 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-20T13:22:44.635 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/505) 2024-04-20T13:22:45.139 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:22:45.152 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:22:45.152 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:45.582 INFO:teuthology.orchestra.run.smithi121.stdout:generic/505 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:22:45.594 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/505 2024-04-20T13:22:45.594 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/505 2024-04-20T13:22:45.594 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:22:45.604 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:45.635 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:22:45.635 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/506) 2024-04-20T13:22:46.179 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:22:46.191 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:22:46.192 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:46.610 INFO:teuthology.orchestra.run.smithi121.stdout:generic/506 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:22:46.622 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/506 2024-04-20T13:22:46.623 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/506 2024-04-20T13:22:46.623 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:22:46.632 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:46.662 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:22:46.663 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/507) 2024-04-20T13:22:47.162 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:22:47.176 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:22:47.176 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:47.703 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:22:47.713 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:22:47.748 INFO:teuthology.orchestra.run.smithi121.stdout:generic/507 [not run] file system doesn't support chattr +ASai 2024-04-20T13:22:47.758 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/507 2024-04-20T13:22:47.758 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/507 2024-04-20T13:22:47.758 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:22:47.765 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:47.790 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:22:47.790 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/508) 2024-04-20T13:22:48.387 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:22:48.400 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:22:48.400 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:48.827 INFO:teuthology.orchestra.run.smithi121.stdout:generic/508 [not run] lsattr not supported by test filesystem type: ceph-fuse 2024-04-20T13:22:48.841 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/508 2024-04-20T13:22:48.841 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/508 2024-04-20T13:22:48.841 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:22:48.854 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:48.884 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:22:48.884 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/509) 2024-04-20T13:22:49.414 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:22:49.428 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:22:49.428 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:49.881 INFO:teuthology.orchestra.run.smithi121.stdout:generic/509 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:22:49.893 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/509 2024-04-20T13:22:49.893 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/509 2024-04-20T13:22:49.893 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:22:49.902 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:49.932 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:22:49.932 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/510) 2024-04-20T13:22:50.483 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:22:50.497 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:22:50.497 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:50.964 INFO:teuthology.orchestra.run.smithi121.stdout:generic/510 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:22:50.978 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/510 2024-04-20T13:22:50.979 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/510 2024-04-20T13:22:50.979 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:22:50.990 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:51.025 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:22:51.025 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/511) 2024-04-20T13:22:51.543 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:22:51.558 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:22:51.558 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:52.015 INFO:teuthology.orchestra.run.smithi121.stdout:generic/511 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:22:52.026 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/511 2024-04-20T13:22:52.026 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/511 2024-04-20T13:22:52.026 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:22:52.035 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:52.066 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:22:52.066 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/512) 2024-04-20T13:22:52.608 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:22:52.622 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:22:52.622 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:53.094 INFO:teuthology.orchestra.run.smithi121.stdout:generic/512 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:22:53.109 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/512 2024-04-20T13:22:53.109 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/512 2024-04-20T13:22:53.109 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:22:53.120 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:53.151 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:22:53.152 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/513) 2024-04-20T13:22:53.665 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:22:53.682 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:22:53.682 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:54.109 INFO:teuthology.orchestra.run.smithi121.stdout:generic/513 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:22:54.120 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/513 2024-04-20T13:22:54.120 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/513 2024-04-20T13:22:54.120 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:22:54.129 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:54.159 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:22:54.160 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/514) 2024-04-20T13:22:54.723 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:22:54.736 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:22:54.736 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:55.166 INFO:teuthology.orchestra.run.smithi121.stdout:generic/514 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:22:55.179 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/514 2024-04-20T13:22:55.180 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/514 2024-04-20T13:22:55.180 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:22:55.191 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:55.223 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:22:55.224 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/515) 2024-04-20T13:22:55.744 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:22:55.758 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:22:55.758 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:56.167 INFO:teuthology.orchestra.run.smithi121.stdout:generic/515 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:22:56.179 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/515 2024-04-20T13:22:56.179 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/515 2024-04-20T13:22:56.179 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:22:56.188 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:56.219 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:22:56.219 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/516) 2024-04-20T13:22:56.758 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:22:56.772 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:22:56.772 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:57.351 INFO:teuthology.orchestra.run.smithi121.stdout:generic/516 [not run] Dedupe not supported by test filesystem type: ceph-fuse 2024-04-20T13:22:57.366 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/516 2024-04-20T13:22:57.366 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/516 2024-04-20T13:22:57.366 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:22:57.377 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:57.424 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:22:57.424 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/517) 2024-04-20T13:22:58.091 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:22:58.103 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:22:58.103 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:58.448 INFO:teuthology.orchestra.run.smithi121.stdout:generic/517 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:22:58.459 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/517 2024-04-20T13:22:58.459 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/517 2024-04-20T13:22:58.459 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:22:58.470 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:58.496 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:22:58.497 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/518) 2024-04-20T13:22:59.023 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:22:59.036 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:22:59.036 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:59.453 INFO:teuthology.orchestra.run.smithi121.stdout:generic/518 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:22:59.464 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/518 2024-04-20T13:22:59.464 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/518 2024-04-20T13:22:59.464 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:22:59.473 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:22:59.502 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:22:59.502 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/519) 2024-04-20T13:23:00.009 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:23:00.022 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:23:00.022 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:23:00.468 INFO:teuthology.orchestra.run.smithi121.stdout:generic/519 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:23:00.483 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/519 2024-04-20T13:23:00.483 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/519 2024-04-20T13:23:00.483 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:23:00.494 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:23:00.524 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:23:00.524 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/520) 2024-04-20T13:23:01.047 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:23:01.060 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:23:01.060 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:23:01.543 INFO:teuthology.orchestra.run.smithi121.stdout:generic/520 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T13:23:01.557 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/520 2024-04-20T13:23:01.557 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/520 2024-04-20T13:23:01.558 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T13:23:01.569 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:23:01.600 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T13:23:01.600 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/521) 2024-04-20T13:23:02.104 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T13:23:02.119 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T13:23:02.120 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T13:23:17.924 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:23:17.935 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:23:48.176 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:23:48.184 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:24:18.344 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:24:18.354 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:24:48.549 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:24:48.559 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:25:18.687 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:25:18.696 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:25:48.860 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:25:48.868 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:26:19.073 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:26:19.082 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:26:49.254 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:26:49.263 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:27:19.534 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:27:19.545 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:27:49.664 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:27:49.672 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:28:19.984 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:28:19.993 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:28:50.125 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:28:50.134 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:29:20.272 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:29:20.281 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:29:50.438 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:29:50.447 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:30:21.310 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:30:21.319 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:30:51.457 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:30:51.465 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:31:21.650 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:31:21.659 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:31:51.806 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:31:51.815 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:32:21.955 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:32:21.967 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:32:52.133 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:32:52.142 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:33:22.340 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:33:22.350 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:33:52.554 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:33:52.566 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:34:22.725 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:34:22.737 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:34:52.914 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:34:52.923 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:35:23.094 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:35:23.105 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:35:53.307 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:35:53.318 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:36:23.496 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:36:23.506 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:36:53.703 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:36:53.712 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:37:23.933 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:37:23.941 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:37:54.069 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:37:54.078 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:38:24.201 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:38:24.212 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:38:54.391 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:38:54.400 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:39:24.538 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:39:24.549 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:39:54.668 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:39:54.678 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:40:24.814 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:40:24.824 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:40:55.003 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:40:55.014 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:41:25.150 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:41:25.161 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:41:55.306 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:41:55.316 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:42:25.566 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:42:25.577 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:42:55.709 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:42:55.718 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:43:25.829 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:43:25.839 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:43:55.994 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:43:56.003 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:44:26.199 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:44:26.208 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:44:56.405 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:44:56.413 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:45:26.576 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:45:26.584 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:45:56.756 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:45:56.765 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:46:26.894 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:46:26.903 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:46:57.084 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:46:57.093 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:47:27.249 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:47:27.258 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:47:57.423 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:47:57.431 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:48:27.628 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:48:27.637 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:48:57.825 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:48:57.833 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:49:27.954 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:49:27.962 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:49:58.135 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:49:58.144 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:50:28.308 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:50:28.318 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:50:58.513 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:50:58.521 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:51:28.785 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:51:28.796 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:51:58.949 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:51:58.959 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:52:29.146 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:52:29.155 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:52:59.593 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:52:59.603 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:53:29.722 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:53:29.733 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:53:59.947 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:53:59.955 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:54:30.100 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:54:30.113 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:55:00.265 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:55:00.275 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:55:30.394 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:55:30.403 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:56:00.557 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:56:00.566 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:56:30.723 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:56:30.733 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:57:00.973 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:57:00.987 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:57:31.120 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:57:31.132 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:58:01.262 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:58:01.271 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:58:31.401 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:58:31.409 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:59:01.614 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:59:01.622 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T13:59:01.729 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T13:59:01.730+0000 7f9c11a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 415503) UID: 0 2024-04-20T13:59:01.729 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-04-20T13:59:01.730+0000 7fb3ce400640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 415503) UID: 0 2024-04-20T13:59:01.729 INFO:tasks.ceph.osd.2.smithi121.stderr:2024-04-20T13:59:01.730+0000 7f11dfa00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 415503) UID: 0 2024-04-20T13:59:01.730 INFO:tasks.ceph.osd.3.smithi121.stderr:2024-04-20T13:59:01.730+0000 7fca40a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 415503) UID: 0 2024-04-20T14:02:13.684 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:02:13.693 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:02:43.837 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:02:43.850 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:03:14.041 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:03:14.050 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:03:44.191 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:03:44.202 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:04:14.351 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:04:14.359 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:04:44.489 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:04:44.498 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:05:14.645 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:05:14.656 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:05:44.885 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:05:44.894 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:06:15.089 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:06:15.097 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:06:45.228 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:06:45.238 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:07:15.400 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:07:15.410 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:07:45.535 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:07:45.545 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:08:15.693 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:08:15.703 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:08:45.849 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:08:45.857 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:09:16.004 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:09:16.013 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:09:46.152 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:09:46.162 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:10:16.306 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:10:16.316 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:10:46.461 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:10:46.470 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:11:16.583 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:11:16.594 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:11:46.732 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:11:46.741 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:12:16.944 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:12:16.953 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:12:47.091 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:12:47.101 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:13:17.273 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:13:17.282 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:13:47.485 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:13:47.493 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:14:17.690 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:14:17.700 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:14:47.871 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:14:47.880 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:15:18.069 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:15:18.078 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:15:48.303 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:15:48.312 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:16:18.732 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:16:18.742 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:16:48.867 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:16:48.881 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:17:19.031 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:17:19.046 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:17:49.269 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:17:49.283 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:18:19.424 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:18:19.433 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:18:49.621 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:18:49.631 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:19:19.772 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:19:19.783 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:19:49.916 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:19:49.928 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:20:20.061 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:20:20.072 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:20:50.275 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:20:50.287 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:21:20.479 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:21:20.488 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:21:50.668 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:21:50.677 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:22:20.833 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:22:20.843 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:22:51.021 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:22:51.030 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:23:21.152 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:23:21.162 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:23:51.349 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:23:51.385 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:24:21.546 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:24:21.556 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:24:51.774 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:24:51.783 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:25:21.955 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:25:21.964 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:25:52.169 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:25:52.178 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:26:22.352 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:26:22.362 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:26:52.505 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:26:52.518 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:27:22.694 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:27:22.703 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:27:52.839 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:27:52.849 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:28:22.961 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:28:22.970 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:28:53.134 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:28:53.144 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:29:23.289 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:29:23.299 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:29:53.469 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:29:53.479 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:30:23.633 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:30:23.642 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:30:53.839 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:30:53.848 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:31:24.036 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:31:24.044 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:31:54.232 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:31:54.241 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:32:24.471 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:32:24.481 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:32:54.685 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:32:54.697 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:33:24.866 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:33:24.875 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:33:55.062 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:33:55.071 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:34:25.199 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:34:25.209 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:34:55.460 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:34:55.469 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:35:25.765 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:35:25.775 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:35:55.909 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:35:55.921 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:36:26.095 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:36:26.104 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:36:56.243 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:36:56.254 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:37:26.381 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:37:26.390 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:37:56.561 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:37:56.570 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:38:26.769 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:38:26.777 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:38:56.957 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:38:56.967 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:39:27.099 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:39:27.108 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:39:57.230 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:39:57.240 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:40:27.434 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:40:27.443 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:40:57.564 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:40:57.575 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:40:57.667 INFO:tasks.ceph.mds.a.smithi121.stderr:2024-04-20T14:40:57.668+0000 7f95c3c00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 416976) UID: 0 2024-04-20T14:40:57.667 INFO:tasks.ceph.mds.c.smithi121.stderr:2024-04-20T14:40:57.668+0000 7f4d9da00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 416976) UID: 0 2024-04-20T14:43:57.844 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:43:57.855 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:44:28.436 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:44:28.446 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:44:58.564 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:44:58.574 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:45:28.746 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:45:28.756 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:45:58.961 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:45:58.974 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:46:29.123 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:46:29.133 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:46:59.353 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:46:59.363 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:47:29.474 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:47:29.482 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:47:59.638 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:47:59.649 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:48:29.793 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:48:29.803 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:48:59.983 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:48:59.993 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:49:30.155 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:49:30.164 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:50:00.330 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:50:00.339 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:50:30.501 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:50:30.509 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:51:00.648 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:51:00.658 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:51:31.377 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:51:31.386 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:52:01.517 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:52:01.527 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:52:31.706 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:52:31.717 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:53:01.911 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:53:01.918 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:53:32.107 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:53:32.116 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:54:02.229 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:54:02.238 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:54:32.434 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:54:32.443 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:55:02.674 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:55:02.685 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:55:32.812 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:55:32.823 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:56:03.010 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:56:03.021 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:56:33.147 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:56:33.156 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:57:03.270 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:57:03.280 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:57:33.474 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:57:33.484 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:58:03.605 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:58:03.614 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:58:33.744 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:58:33.754 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:59:03.983 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:59:03.991 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:59:34.154 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T14:59:34.164 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:00:04.277 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:00:04.288 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:00:34.423 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:00:34.432 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:01:04.587 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:01:04.597 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:01:34.810 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:01:34.820 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:01:34.922 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-04-20T15:01:34.922+0000 7fb3ce400640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 417652) UID: 0 2024-04-20T15:01:34.922 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T15:01:34.922+0000 7f9c11a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 417652) UID: 0 2024-04-20T15:01:34.923 INFO:tasks.ceph.osd.3.smithi121.stderr:2024-04-20T15:01:34.922+0000 7fca40a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 417652) UID: 0 2024-04-20T15:01:34.923 INFO:tasks.ceph.osd.2.smithi121.stderr:2024-04-20T15:01:34.922+0000 7f11dfa00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 417652) UID: 0 2024-04-20T15:04:40.094 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:04:40.105 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:05:10.301 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:05:10.310 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:05:40.441 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:05:40.449 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:06:10.568 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:06:10.578 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:06:40.782 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:06:40.791 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:07:10.904 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:07:10.913 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:07:41.035 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:07:41.044 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:08:11.224 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:08:11.232 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:08:41.556 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:08:41.565 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:09:11.700 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:09:11.711 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:09:41.899 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:09:41.909 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:10:12.096 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:10:12.104 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:10:42.422 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:10:42.431 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:11:12.563 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:11:12.572 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:11:42.693 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:11:42.703 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:12:12.866 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:12:12.876 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:12:43.017 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:12:43.026 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:13:13.254 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:13:13.262 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:13:43.390 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:13:43.401 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:14:13.596 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:14:13.604 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:14:43.785 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:14:43.798 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:15:13.923 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:15:13.933 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:15:44.060 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:15:44.070 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:16:14.233 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:16:14.242 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:16:44.404 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:16:44.414 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:16:44.506 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-04-20T15:16:44.507+0000 7fb3ce400640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 418119) UID: 0 2024-04-20T15:16:44.507 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T15:16:44.507+0000 7f9c11a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 418119) UID: 0 2024-04-20T15:16:44.507 INFO:tasks.ceph.osd.2.smithi121.stderr:2024-04-20T15:16:44.507+0000 7f11dfa00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 418119) UID: 0 2024-04-20T15:16:44.508 INFO:tasks.ceph.osd.3.smithi121.stderr:2024-04-20T15:16:44.507+0000 7fca40a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 418119) UID: 0 2024-04-20T15:19:37.476 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:19:37.486 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:20:07.681 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:20:07.689 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:20:37.811 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:20:37.821 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:21:08.008 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:21:08.018 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:21:38.139 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:21:38.150 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:22:08.263 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:22:08.271 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:22:38.743 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:22:38.754 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:23:08.908 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:23:08.919 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:23:39.137 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:23:39.147 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:24:09.309 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:24:09.319 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:24:39.472 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:24:39.482 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:25:09.637 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:25:09.645 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:25:39.867 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:25:39.875 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:26:10.074 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:26:10.083 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:26:40.470 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:26:40.480 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:27:10.591 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:27:10.601 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:27:40.782 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:27:40.792 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:28:10.995 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:28:11.007 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:28:41.125 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:28:41.134 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:29:11.332 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:29:11.340 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:29:41.544 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:29:41.553 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:30:11.676 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:30:11.685 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:30:41.846 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:30:41.854 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:31:12.035 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:31:12.045 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:31:42.165 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:31:42.175 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:32:12.388 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:32:12.398 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:32:42.670 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:32:42.680 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:33:12.849 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:33:12.858 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:33:43.029 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:33:43.038 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:34:13.209 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:34:13.219 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:34:43.416 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:34:43.424 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:35:10.881 INFO:teuthology.orchestra.run.smithi121.stdout:generic/521 7928s 2024-04-20T15:35:10.927 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/521 2024-04-20T15:35:10.927 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T15:35:10.937 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T15:35:10.968 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T15:35:10.968 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/522) 2024-04-20T15:35:11.866 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T15:35:11.877 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T15:35:11.877 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T15:35:13.542 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:35:13.554 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:35:43.751 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:35:43.761 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:36:14.066 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:36:14.075 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:36:44.196 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:36:44.206 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:37:14.326 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:37:14.335 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:37:44.463 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:37:44.472 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:38:14.586 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:38:14.595 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:38:44.783 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:38:44.793 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:39:14.913 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:39:14.922 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:39:45.035 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:39:45.044 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:40:15.165 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:40:15.173 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:40:45.313 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:40:45.322 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:41:15.460 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:41:15.468 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:41:45.803 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:41:45.814 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:42:15.954 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:42:15.966 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:42:46.077 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:42:46.085 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:43:16.199 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:43:16.209 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:43:46.328 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:43:46.338 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:44:16.468 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:44:16.476 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:44:46.614 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:44:46.623 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:45:16.875 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:45:16.885 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:45:47.004 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:45:47.014 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:46:17.133 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:46:17.142 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:46:47.281 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:46:47.291 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:47:17.410 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:47:17.419 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:47:47.538 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:47:47.548 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:48:17.761 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:48:17.771 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:48:47.885 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:48:47.895 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:49:18.014 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:49:18.023 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:49:48.193 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:49:48.202 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:50:18.309 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:50:18.318 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:50:48.443 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:50:48.452 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:51:18.582 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:51:18.591 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:51:48.702 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:51:48.712 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:52:19.024 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:52:19.033 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:52:49.177 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:52:49.186 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:53:19.303 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:53:19.312 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:53:49.502 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:53:49.513 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:53:49.609 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-04-20T15:53:49.608+0000 7fb3ce400640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 419883) UID: 0 2024-04-20T15:53:49.609 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T15:53:49.608+0000 7f9c11a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 419883) UID: 0 2024-04-20T15:53:49.610 INFO:tasks.ceph.osd.2.smithi121.stderr:2024-04-20T15:53:49.608+0000 7f11dfa00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 419883) UID: 0 2024-04-20T15:53:49.610 INFO:tasks.ceph.osd.3.smithi121.stderr:2024-04-20T15:53:49.608+0000 7fca40a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 419883) UID: 0 2024-04-20T15:57:05.762 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:57:05.777 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:57:36.189 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:57:36.887 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:58:07.014 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:58:07.024 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:58:37.376 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:58:37.387 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:59:07.499 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:59:07.509 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:59:37.655 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T15:59:37.665 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:00:07.798 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:00:07.807 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:00:37.925 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:00:37.934 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:01:08.053 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:01:08.065 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:01:38.194 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:01:38.202 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:02:08.366 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:02:08.376 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:02:38.574 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:02:38.583 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:03:08.700 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:03:08.711 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:03:38.832 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:03:38.841 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:04:08.961 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:04:08.970 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:04:39.144 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:04:39.153 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:05:09.255 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:05:09.264 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:05:39.392 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:05:39.403 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:06:09.523 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:06:09.532 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:06:39.650 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:06:39.660 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:07:09.771 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:07:09.780 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:07:40.042 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:07:40.050 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:08:10.196 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:08:10.206 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:08:40.333 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:08:40.344 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:09:10.459 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:09:10.468 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:09:40.613 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:09:40.622 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:10:10.734 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:10:10.745 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:10:40.885 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:10:40.895 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:11:11.062 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:11:11.071 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:11:41.194 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:11:41.203 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:12:11.332 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:12:11.342 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:12:41.462 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:12:41.471 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:13:11.641 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:13:11.651 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:13:41.848 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:13:41.857 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:14:11.974 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:14:11.983 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:14:42.099 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:14:42.109 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:15:12.247 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:15:12.257 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:15:42.379 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:15:42.388 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:16:12.516 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:16:12.525 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:16:42.679 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:16:42.688 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:17:12.808 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:17:12.818 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:17:42.948 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:17:42.958 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:18:13.095 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:18:13.104 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:18:43.489 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:18:43.498 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:19:13.622 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:19:13.631 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:19:43.769 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:19:43.778 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:20:13.900 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:20:13.910 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:20:44.022 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:20:44.033 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:21:14.176 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:21:14.186 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:21:44.357 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:21:44.367 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:22:14.490 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:22:14.500 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:22:44.617 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:22:44.626 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:23:15.286 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:23:15.297 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:23:45.481 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:23:45.489 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:24:15.626 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:24:15.635 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:24:45.773 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:24:45.781 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:25:15.953 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:25:15.961 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:25:46.100 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:25:46.111 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:26:16.242 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:26:16.251 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:26:46.369 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:26:46.378 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:27:16.499 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:27:16.508 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:27:46.621 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:27:46.630 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:28:16.751 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:28:16.761 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:28:46.881 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:28:46.891 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:29:17.667 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:29:17.677 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:29:47.803 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:29:47.813 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:29:47.918 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-04-20T16:29:47.916+0000 7f4dce800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 40530) UID: 0 2024-04-20T16:29:47.918 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-04-20T16:29:47.916+0000 7f1143800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 40530) UID: 0 2024-04-20T16:29:47.919 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T16:29:47.916+0000 7fad6ce00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 40530) UID: 0 2024-04-20T16:29:47.919 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-04-20T16:29:47.916+0000 7f5d77a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 40530) UID: 0 2024-04-20T16:32:32.822 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:32:32.833 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:33:03.013 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:33:03.025 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:33:03.120 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-04-20T16:33:03.120+0000 7f4dce800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 40578) UID: 0 2024-04-20T16:33:03.121 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-04-20T16:33:03.120+0000 7f1143800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 40578) UID: 0 2024-04-20T16:33:03.121 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-04-20T16:33:03.120+0000 7f5d77a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 40578) UID: 0 2024-04-20T16:33:03.122 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T16:33:03.120+0000 7fad6ce00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 40578) UID: 0 2024-04-20T16:36:06.831 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:36:06.840 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:36:37.028 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:36:37.037 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:37:07.166 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:37:07.176 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:37:37.292 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:37:37.304 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:38:07.425 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:38:07.434 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:38:37.553 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:38:37.562 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:39:07.682 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:39:07.691 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:39:37.807 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:39:37.816 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:40:07.937 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:40:07.949 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:40:38.087 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:40:38.096 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:41:08.217 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:41:08.229 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:41:38.345 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:41:38.354 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:42:08.484 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:42:08.493 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:42:38.648 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:42:38.659 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:43:08.805 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:43:08.814 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:43:38.937 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:43:38.947 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:44:09.082 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:44:09.092 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:44:39.244 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:44:39.252 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:45:09.384 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:45:09.394 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:45:39.504 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:45:39.545 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:46:09.676 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:46:09.687 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:46:39.808 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:46:39.817 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:47:09.933 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:47:09.942 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:47:40.096 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:47:40.105 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:48:10.249 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:48:10.264 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:48:40.386 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:48:40.396 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:49:10.525 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:49:10.537 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:49:40.655 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:49:40.664 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:50:10.776 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:50:10.786 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:50:40.918 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:50:40.927 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:51:11.046 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:51:11.056 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:51:41.186 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:51:41.196 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:52:11.333 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:52:11.341 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:52:41.479 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:52:41.488 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:53:11.625 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:53:11.636 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:53:41.784 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:53:41.793 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:54:11.913 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:54:11.921 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:54:42.034 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:54:42.044 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:55:12.389 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:55:12.397 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:55:42.519 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:55:42.528 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:56:12.644 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:56:12.653 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:56:42.790 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:56:42.799 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:57:12.909 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:57:12.920 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:57:43.032 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:57:43.041 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:58:13.162 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:58:13.172 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:58:43.301 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:58:43.310 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:59:13.432 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:59:13.441 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:59:43.553 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T16:59:43.563 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:00:13.701 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:00:13.717 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:00:43.842 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:00:43.851 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:01:13.971 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:01:13.981 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:01:44.169 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:01:44.179 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:02:14.329 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:02:14.340 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:02:44.463 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:02:44.473 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:03:14.603 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:03:14.612 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:03:44.739 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:03:44.749 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:04:14.868 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:04:14.876 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:04:45.200 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:04:45.210 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:05:15.329 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:05:15.342 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:05:45.455 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:05:45.467 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:06:15.591 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:06:15.600 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:06:45.728 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:06:45.737 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:07:16.099 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:07:16.108 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:07:16.203 INFO:tasks.ceph.mds.a.smithi121.stderr:2024-04-20T17:07:16.202+0000 7f95c3c00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 422393) UID: 0 2024-04-20T17:07:16.209 INFO:tasks.ceph.mds.c.smithi121.stderr:2024-04-20T17:07:16.202+0000 7f4d9da00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 422393) UID: 0 2024-04-20T17:09:28.601 INFO:teuthology.orchestra.run.smithi121.stdout:generic/522 5657s 2024-04-20T17:09:28.665 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/522 2024-04-20T17:09:28.665 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T17:09:28.675 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T17:09:28.707 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T17:09:28.707 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/523) 2024-04-20T17:09:29.348 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T17:09:29.359 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T17:09:29.359 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T17:09:29.754 INFO:teuthology.orchestra.run.smithi121.stdout:generic/523 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T17:09:29.764 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/523 2024-04-20T17:09:29.765 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/523 2024-04-20T17:09:29.765 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T17:09:29.773 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T17:09:29.805 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T17:09:29.806 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/524) 2024-04-20T17:09:30.288 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T17:09:30.299 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T17:09:30.299 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T17:09:30.673 INFO:teuthology.orchestra.run.smithi121.stdout:generic/524 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T17:09:30.683 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/524 2024-04-20T17:09:30.683 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/524 2024-04-20T17:09:30.683 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T17:09:30.692 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T17:09:30.720 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T17:09:30.720 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/525) 2024-04-20T17:09:31.199 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T17:09:31.210 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T17:09:31.210 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T17:09:31.601 INFO:teuthology.orchestra.run.smithi121.stdout:generic/525 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T17:09:31.614 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/525 2024-04-20T17:09:31.614 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/525 2024-04-20T17:09:31.615 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T17:09:31.626 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T17:09:31.652 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T17:09:31.653 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/526) 2024-04-20T17:09:32.118 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T17:09:32.130 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T17:09:32.130 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T17:09:32.648 INFO:teuthology.orchestra.run.smithi121.stdout:generic/526 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T17:09:32.660 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/526 2024-04-20T17:09:32.661 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/526 2024-04-20T17:09:32.661 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T17:09:32.670 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T17:09:32.699 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T17:09:32.699 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/527) 2024-04-20T17:09:33.167 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T17:09:33.178 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T17:09:33.178 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T17:09:33.548 INFO:teuthology.orchestra.run.smithi121.stdout:generic/527 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T17:09:33.559 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/527 2024-04-20T17:09:33.559 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/527 2024-04-20T17:09:33.559 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T17:09:33.567 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T17:09:33.593 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T17:09:33.594 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/528) 2024-04-20T17:09:34.035 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T17:09:34.046 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T17:09:34.046 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T17:09:34.489 INFO:teuthology.orchestra.run.smithi121.stdout:generic/528 [not run] inode creation time not supported by this filesystem 2024-04-20T17:09:34.499 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/528 2024-04-20T17:09:34.500 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/528 2024-04-20T17:09:34.500 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T17:09:34.508 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T17:09:34.531 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T17:09:34.531 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/529) 2024-04-20T17:09:34.969 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T17:09:34.981 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T17:09:34.981 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T17:09:35.411 INFO:teuthology.orchestra.run.smithi121.stdout:generic/529 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-04-20T17:09:35.421 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/529 2024-04-20T17:09:35.421 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/529 2024-04-20T17:09:35.421 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T17:09:35.428 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T17:09:35.453 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T17:09:35.453 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/530) 2024-04-20T17:09:35.894 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T17:09:35.907 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T17:09:35.907 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T17:09:36.302 INFO:teuthology.orchestra.run.smithi121.stdout:generic/530 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T17:09:36.312 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/530 2024-04-20T17:09:36.312 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/530 2024-04-20T17:09:36.312 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T17:09:36.320 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T17:09:36.349 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T17:09:36.349 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/531) 2024-04-20T17:09:36.886 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T17:09:36.901 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T17:09:36.901 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T17:09:37.337 INFO:teuthology.orchestra.run.smithi121.stdout:generic/531 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T17:09:37.352 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/531 2024-04-20T17:09:37.352 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/531 2024-04-20T17:09:37.352 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T17:09:37.364 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T17:09:37.398 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T17:09:37.398 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/532) 2024-04-20T17:09:37.945 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T17:09:37.959 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T17:09:37.959 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T17:09:38.538 INFO:teuthology.orchestra.run.smithi121.stdout:generic/532 1s 2024-04-20T17:09:38.556 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/532 2024-04-20T17:09:38.556 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T17:09:38.566 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T17:09:38.597 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T17:09:38.597 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/533) 2024-04-20T17:09:39.113 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T17:09:39.125 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T17:09:39.125 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T17:09:40.030 INFO:teuthology.orchestra.run.smithi121.stdout:generic/533 1s 2024-04-20T17:09:40.046 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/533 2024-04-20T17:09:40.046 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T17:09:40.054 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T17:09:40.080 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T17:09:40.080 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/534) 2024-04-20T17:09:40.626 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T17:09:40.640 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T17:09:40.640 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T17:09:41.103 INFO:teuthology.orchestra.run.smithi121.stdout:generic/534 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T17:09:41.114 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/534 2024-04-20T17:09:41.114 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/534 2024-04-20T17:09:41.114 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T17:09:41.121 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T17:09:41.150 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T17:09:41.150 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/535) 2024-04-20T17:09:41.695 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T17:09:41.709 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T17:09:41.709 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T17:09:42.165 INFO:teuthology.orchestra.run.smithi121.stdout:generic/535 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T17:09:42.179 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/535 2024-04-20T17:09:42.179 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/535 2024-04-20T17:09:42.179 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T17:09:42.190 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T17:09:42.224 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T17:09:42.224 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/536) 2024-04-20T17:09:42.758 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T17:09:42.771 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T17:09:42.772 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T17:09:43.210 INFO:teuthology.orchestra.run.smithi121.stdout:generic/536 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T17:09:43.224 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/536 2024-04-20T17:09:43.224 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/536 2024-04-20T17:09:43.224 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T17:09:43.235 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T17:09:43.270 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T17:09:43.270 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/537) 2024-04-20T17:09:43.788 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T17:09:43.802 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T17:09:43.802 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T17:09:44.200 INFO:teuthology.orchestra.run.smithi121.stdout:generic/537 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T17:09:44.212 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/537 2024-04-20T17:09:44.212 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/537 2024-04-20T17:09:44.212 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T17:09:44.222 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T17:09:44.255 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T17:09:44.255 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/538) 2024-04-20T17:09:44.785 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T17:09:44.799 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T17:09:44.799 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T17:10:05.664 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:10:05.672 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:10:35.859 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:10:35.868 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:11:05.982 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:11:05.992 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:11:36.115 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:11:36.127 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:12:06.244 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:12:06.253 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:12:36.380 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:12:36.390 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:13:06.520 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:13:06.533 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:13:36.668 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:13:36.677 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:14:06.796 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:14:06.805 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:14:36.937 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:14:36.946 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:15:07.142 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:15:07.151 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:15:37.278 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:15:37.287 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:16:07.423 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:16:07.433 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:16:37.574 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:16:37.586 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:17:07.765 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:17:07.776 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:17:37.913 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:17:37.922 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:18:08.064 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:18:08.075 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:18:38.195 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:18:38.204 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:19:08.321 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:19:08.333 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:19:38.474 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:19:38.483 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:20:08.601 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:20:08.611 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:20:38.742 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:20:38.750 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:21:08.879 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:21:08.888 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:21:39.044 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:21:39.053 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:22:09.179 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:22:09.190 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:22:39.328 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:22:39.339 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:23:09.453 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:23:09.461 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:23:39.598 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:23:39.607 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:24:09.745 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:24:09.754 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:24:39.895 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:24:39.915 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:25:10.038 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:25:10.046 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:25:40.181 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:25:40.192 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:26:10.306 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:26:10.315 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:26:40.431 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:26:40.440 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:27:10.770 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:27:10.779 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:27:40.906 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:27:40.915 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:28:11.034 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:28:11.044 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:28:41.175 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:28:41.184 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:29:11.375 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:29:11.384 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:29:41.527 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:29:41.537 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:30:11.679 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:30:11.690 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:30:41.805 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:30:41.814 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:31:11.950 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:31:11.960 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:31:42.271 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:31:42.280 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:32:12.633 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:32:12.645 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:32:42.869 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:32:42.879 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:33:12.997 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:33:13.007 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:33:43.128 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:33:43.137 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:34:13.265 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:34:13.275 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:34:43.403 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:34:43.414 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:35:13.592 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:35:13.601 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:35:43.719 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:35:43.728 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:36:13.871 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:36:13.880 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:36:44.000 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:36:44.009 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:37:14.129 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:37:14.138 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:37:44.261 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:37:44.271 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:38:14.399 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:38:14.409 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:38:44.537 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:38:44.547 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:39:14.663 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:39:14.671 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:39:44.798 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:39:44.808 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:40:14.929 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:40:14.938 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:40:45.052 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:40:45.064 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:41:15.217 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:41:15.226 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:41:45.346 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:41:45.356 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:42:15.521 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:42:15.532 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:42:45.641 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:42:45.650 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:43:15.762 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:43:15.772 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:43:45.889 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:43:45.898 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:44:16.040 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:44:16.048 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:44:46.181 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:44:46.190 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:45:16.300 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:45:16.309 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:45:46.431 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:45:46.440 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:46:16.779 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:46:16.788 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:46:46.914 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:46:46.925 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:47:17.072 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:47:17.082 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:47:47.189 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:47:47.198 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:48:17.316 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:48:17.324 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:48:47.438 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:48:47.447 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:49:17.577 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:49:17.587 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:49:47.739 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:49:47.750 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:50:17.871 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:50:17.881 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:50:48.077 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:50:48.088 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:51:18.216 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:51:18.225 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:51:48.336 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:51:48.346 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:52:18.475 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:52:18.484 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:52:48.636 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:52:48.645 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:53:18.760 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:53:18.769 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:53:48.882 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:53:48.891 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:54:19.005 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:54:19.014 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:54:49.529 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:54:49.538 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:55:19.663 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:55:19.672 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:55:49.810 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:55:49.819 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:56:19.946 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:56:19.955 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:56:50.089 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:56:50.098 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:57:20.228 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:57:20.238 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:57:50.359 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:57:50.368 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:58:20.603 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:58:20.613 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:58:50.736 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:58:50.746 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:59:20.906 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:59:20.915 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:59:51.063 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T17:59:51.073 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:00:21.204 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:00:21.213 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:00:51.357 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:00:51.367 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:01:21.479 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:01:21.488 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:01:51.644 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:01:51.653 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:02:21.798 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:02:21.810 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:02:51.936 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:02:51.945 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:03:22.057 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:03:22.096 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:03:52.216 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:03:52.224 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:04:22.360 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:04:22.370 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:04:52.485 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:04:52.494 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:05:22.630 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:05:22.639 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:05:52.841 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:05:52.851 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:06:22.986 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:06:22.994 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:06:53.164 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:06:53.176 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:07:23.310 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:07:23.318 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:07:53.432 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:07:53.441 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:08:23.572 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:08:23.579 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:08:53.691 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:08:53.700 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:09:23.912 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:09:23.921 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:09:54.043 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:09:54.052 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:10:24.207 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:10:24.216 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:10:54.338 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:10:54.346 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:11:24.535 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:11:24.544 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:11:54.673 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:11:54.682 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:12:25.220 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:12:25.230 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:12:55.341 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:12:55.350 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:13:25.480 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:13:25.489 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:13:55.614 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:13:55.623 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:14:25.741 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:14:25.750 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:14:55.897 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:14:55.905 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:15:26.108 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:15:26.119 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:15:56.256 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:15:56.266 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:16:26.384 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:16:26.394 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:16:56.522 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:16:56.531 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:17:26.655 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:17:26.664 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:17:26.753 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-04-20T18:17:26.753+0000 7fb3ce400640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 435979) UID: 0 2024-04-20T18:17:26.753 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T18:17:26.753+0000 7f9c11a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 435979) UID: 0 2024-04-20T18:17:26.753 INFO:tasks.ceph.osd.2.smithi121.stderr:2024-04-20T18:17:26.753+0000 7f11dfa00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 435979) UID: 0 2024-04-20T18:17:26.754 INFO:tasks.ceph.osd.3.smithi121.stderr:2024-04-20T18:17:26.753+0000 7fca40a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 435979) UID: 0 2024-04-20T18:20:30.972 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:20:30.982 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:21:01.300 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:21:01.309 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:21:31.437 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:21:31.447 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:22:01.592 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:22:01.602 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:22:31.724 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:22:31.733 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:23:01.853 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:23:01.862 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:23:32.007 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:23:32.016 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:24:02.155 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:24:02.165 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:24:32.299 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:24:32.308 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:25:02.423 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:25:02.435 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:25:32.564 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:25:32.573 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:26:02.686 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:26:02.722 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:26:32.840 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:26:32.850 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:27:03.199 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:27:03.208 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:27:33.331 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:27:33.340 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:28:03.476 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:28:03.488 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:28:33.597 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:28:33.606 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:29:03.743 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:29:03.752 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:29:33.890 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:29:33.900 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:30:04.181 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:30:04.193 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:30:34.324 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:30:34.332 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:30:34.440 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T18:30:34.440+0000 7f9c11a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 437147) UID: 0 2024-04-20T18:30:34.441 INFO:tasks.ceph.osd.3.smithi121.stderr:2024-04-20T18:30:34.440+0000 7fca40a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 437147) UID: 0 2024-04-20T18:30:34.441 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-04-20T18:30:34.440+0000 7fb3ce400640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 437147) UID: 0 2024-04-20T18:30:34.441 INFO:tasks.ceph.osd.2.smithi121.stderr:2024-04-20T18:30:34.440+0000 7f11dfa00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 437147) UID: 0 2024-04-20T18:33:39.797 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:33:39.807 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:34:10.130 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:34:10.139 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:34:40.256 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:34:40.266 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:35:10.379 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:35:10.388 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:35:40.500 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:35:40.511 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:36:10.632 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:36:10.642 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:36:40.760 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:36:40.766 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:37:10.892 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:37:10.903 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:37:41.029 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:37:41.038 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:38:11.160 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:38:11.170 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:38:41.290 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:38:41.302 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:39:11.497 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:39:11.506 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:39:41.625 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:39:41.634 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:40:11.756 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:40:11.766 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:40:41.897 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:40:41.906 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:41:12.033 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:41:12.043 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:41:42.214 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:41:42.223 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:42:12.353 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:42:12.365 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:42:42.491 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:42:42.501 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:43:12.649 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:43:12.660 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:43:42.788 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:43:42.799 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:44:12.932 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:44:12.942 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:44:13.047 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-04-20T18:44:13.047+0000 7f1143800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 45116) UID: 0 2024-04-20T18:44:13.048 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T18:44:13.047+0000 7fad6ce00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 45116) UID: 0 2024-04-20T18:44:13.048 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-04-20T18:44:13.047+0000 7f4dce800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 45116) UID: 0 2024-04-20T18:44:13.048 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-04-20T18:44:13.047+0000 7f5d77a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 45116) UID: 0 2024-04-20T18:47:10.724 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:47:10.733 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:47:40.932 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:47:40.943 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:48:11.101 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:48:11.111 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:48:41.231 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:48:41.242 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:49:11.365 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:49:11.373 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:49:41.491 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:49:41.500 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:50:11.634 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:50:11.643 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:50:41.790 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:50:41.801 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:51:11.936 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:51:11.945 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:51:42.053 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:51:42.067 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:52:12.222 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:52:12.237 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:52:42.386 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:52:42.395 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:53:12.506 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:53:12.515 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:53:42.660 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:53:42.669 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:54:12.789 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:54:12.801 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:54:42.950 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:54:42.962 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:55:00.688 INFO:teuthology.orchestra.run.smithi121.stdout:generic/538 6316s 2024-04-20T18:55:00.734 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/538 2024-04-20T18:55:00.735 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:00.744 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:00.781 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:00.781 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/539) 2024-04-20T18:55:01.672 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:01.685 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:01.686 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:06.917 INFO:teuthology.orchestra.run.smithi121.stdout:generic/539 5s 2024-04-20T18:55:06.933 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/539 2024-04-20T18:55:06.934 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:06.942 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:06.970 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:06.970 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/540) 2024-04-20T18:55:07.673 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:07.684 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:07.684 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:08.086 INFO:teuthology.orchestra.run.smithi121.stdout:generic/540 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:55:08.097 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/540 2024-04-20T18:55:08.097 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/540 2024-04-20T18:55:08.097 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:08.106 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:08.133 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:08.133 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/541) 2024-04-20T18:55:08.685 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:08.699 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:08.699 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:09.145 INFO:teuthology.orchestra.run.smithi121.stdout:generic/541 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:55:09.158 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/541 2024-04-20T18:55:09.159 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/541 2024-04-20T18:55:09.159 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:09.171 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:09.201 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:09.201 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/542) 2024-04-20T18:55:09.740 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:09.754 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:09.754 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:10.176 INFO:teuthology.orchestra.run.smithi121.stdout:generic/542 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:55:10.190 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/542 2024-04-20T18:55:10.190 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/542 2024-04-20T18:55:10.190 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:10.201 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:10.237 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:10.238 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/543) 2024-04-20T18:55:10.791 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:10.804 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:10.804 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:11.241 INFO:teuthology.orchestra.run.smithi121.stdout:generic/543 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:55:11.250 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/543 2024-04-20T18:55:11.250 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/543 2024-04-20T18:55:11.251 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:11.259 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:11.287 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:11.287 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/544) 2024-04-20T18:55:11.830 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:11.844 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:11.844 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:12.243 INFO:teuthology.orchestra.run.smithi121.stdout:generic/544 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:55:12.256 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/544 2024-04-20T18:55:12.256 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/544 2024-04-20T18:55:12.256 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:12.268 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:12.299 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:12.299 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/545) 2024-04-20T18:55:12.855 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:12.869 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:12.869 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:13.072 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:55:13.077 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:55:13.438 INFO:teuthology.orchestra.run.smithi121.stdout:generic/545 [not run] file system doesn't support chattr +i 2024-04-20T18:55:13.450 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/545 2024-04-20T18:55:13.451 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/545 2024-04-20T18:55:13.451 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:13.460 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:13.495 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:13.496 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/546) 2024-04-20T18:55:14.044 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:14.058 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:14.058 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:14.628 INFO:teuthology.orchestra.run.smithi121.stdout:generic/546 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:55:14.641 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/546 2024-04-20T18:55:14.641 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/546 2024-04-20T18:55:14.641 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:14.650 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:14.680 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:14.680 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/547) 2024-04-20T18:55:15.234 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:15.248 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:15.248 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:15.738 INFO:teuthology.orchestra.run.smithi121.stdout:generic/547 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:55:15.749 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/547 2024-04-20T18:55:15.749 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/547 2024-04-20T18:55:15.749 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:15.759 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:15.790 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:15.790 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/548) 2024-04-20T18:55:16.340 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:16.354 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:16.354 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:16.920 INFO:teuthology.orchestra.run.smithi121.stdout:generic/548 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:55:16.933 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/548 2024-04-20T18:55:16.933 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/548 2024-04-20T18:55:16.933 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:16.942 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:16.974 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:16.974 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/549) 2024-04-20T18:55:17.496 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:17.510 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:17.510 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:18.084 INFO:teuthology.orchestra.run.smithi121.stdout:generic/549 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:55:18.099 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/549 2024-04-20T18:55:18.099 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/549 2024-04-20T18:55:18.099 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:18.111 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:18.146 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:18.146 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/550) 2024-04-20T18:55:18.692 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:18.706 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:18.707 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:19.215 INFO:teuthology.orchestra.run.smithi121.stdout:generic/550 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:55:19.226 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/550 2024-04-20T18:55:19.226 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/550 2024-04-20T18:55:19.226 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:19.236 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:19.267 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:19.267 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/551) 2024-04-20T18:55:19.825 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:19.840 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:19.840 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:20.264 INFO:teuthology.orchestra.run.smithi121.stdout:generic/551 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:55:20.278 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/551 2024-04-20T18:55:20.279 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/551 2024-04-20T18:55:20.279 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:20.289 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:20.319 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:20.319 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/552) 2024-04-20T18:55:20.846 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:20.859 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:20.859 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:21.285 INFO:teuthology.orchestra.run.smithi121.stdout:generic/552 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:55:21.295 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/552 2024-04-20T18:55:21.295 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/552 2024-04-20T18:55:21.295 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:21.302 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:21.327 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:21.327 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/553) 2024-04-20T18:55:21.888 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:21.902 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:21.902 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:22.515 INFO:teuthology.orchestra.run.smithi121.stdout:generic/553 [not run] xfs_io chattr +i failed (old kernel/wrong fs/bad args?) 2024-04-20T18:55:22.525 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/553 2024-04-20T18:55:22.526 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/553 2024-04-20T18:55:22.526 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:22.534 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:22.563 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:22.563 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/554) 2024-04-20T18:55:23.255 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:23.267 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:23.267 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:23.615 INFO:teuthology.orchestra.run.smithi121.stdout:generic/554 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:55:23.626 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/554 2024-04-20T18:55:23.626 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/554 2024-04-20T18:55:23.626 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:23.634 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:23.659 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:23.659 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/555) 2024-04-20T18:55:24.188 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:24.202 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:24.203 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:24.746 INFO:teuthology.orchestra.run.smithi121.stdout:generic/555 [not run] xfs_io chattr +ia failed (old kernel/wrong fs/bad args?) 2024-04-20T18:55:24.760 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/555 2024-04-20T18:55:24.761 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/555 2024-04-20T18:55:24.761 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:24.773 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:24.813 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:24.813 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/556) 2024-04-20T18:55:25.495 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:25.506 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:25.506 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:25.893 INFO:teuthology.orchestra.run.smithi121.stdout:generic/556 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:55:25.903 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/556 2024-04-20T18:55:25.903 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/556 2024-04-20T18:55:25.903 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:25.912 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:25.940 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:25.940 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/557) 2024-04-20T18:55:26.490 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:26.505 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:26.505 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:26.940 INFO:teuthology.orchestra.run.smithi121.stdout:generic/557 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:55:26.954 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/557 2024-04-20T18:55:26.954 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/557 2024-04-20T18:55:26.954 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:26.961 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:26.994 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:26.994 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/558) 2024-04-20T18:55:27.534 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:27.548 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:27.548 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:27.958 INFO:teuthology.orchestra.run.smithi121.stdout:generic/558 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:55:27.969 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/558 2024-04-20T18:55:27.969 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/558 2024-04-20T18:55:27.969 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:27.978 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:28.006 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:28.007 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/559) 2024-04-20T18:55:28.565 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:28.577 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:28.577 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:28.976 INFO:teuthology.orchestra.run.smithi121.stdout:generic/559 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:55:28.990 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/559 2024-04-20T18:55:28.990 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/559 2024-04-20T18:55:28.990 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:29.001 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:29.034 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:29.035 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/560) 2024-04-20T18:55:29.607 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:29.624 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:29.624 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:30.038 INFO:teuthology.orchestra.run.smithi121.stdout:generic/560 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:55:30.050 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/560 2024-04-20T18:55:30.050 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/560 2024-04-20T18:55:30.050 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:30.059 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:30.089 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:30.089 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/561) 2024-04-20T18:55:30.664 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:30.679 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:30.679 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:31.137 INFO:teuthology.orchestra.run.smithi121.stdout:generic/561 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:55:31.149 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/561 2024-04-20T18:55:31.149 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/561 2024-04-20T18:55:31.150 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:31.159 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:31.189 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:31.190 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/562) 2024-04-20T18:55:31.744 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:31.757 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:31.757 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:32.155 INFO:teuthology.orchestra.run.smithi121.stdout:generic/562 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:55:32.166 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/562 2024-04-20T18:55:32.166 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/562 2024-04-20T18:55:32.167 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:32.174 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:32.204 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:32.205 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/563) 2024-04-20T18:55:32.733 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:32.749 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:32.749 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:33.186 INFO:teuthology.orchestra.run.smithi121.stdout:generic/563 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:55:33.198 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/563 2024-04-20T18:55:33.198 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/563 2024-04-20T18:55:33.198 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:33.207 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:33.237 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:33.237 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/564) 2024-04-20T18:55:33.777 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:33.792 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:33.792 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:34.962 INFO:teuthology.orchestra.run.smithi121.stdout:generic/564 1s 2024-04-20T18:55:34.980 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/564 2024-04-20T18:55:34.980 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:34.989 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:35.019 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:35.019 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/565) 2024-04-20T18:55:35.712 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:35.723 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:35.723 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:36.096 INFO:teuthology.orchestra.run.smithi121.stdout:generic/565 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:55:36.107 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/565 2024-04-20T18:55:36.107 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/565 2024-04-20T18:55:36.107 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:36.116 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:36.144 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:36.145 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/566) 2024-04-20T18:55:36.678 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:36.692 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:36.692 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:37.104 INFO:teuthology.orchestra.run.smithi121.stdout:generic/566 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:55:37.115 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/566 2024-04-20T18:55:37.116 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/566 2024-04-20T18:55:37.116 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:37.124 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:37.153 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:37.153 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/567) 2024-04-20T18:55:37.729 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:37.743 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:37.743 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:38.190 INFO:teuthology.orchestra.run.smithi121.stdout:generic/567 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:55:38.203 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/567 2024-04-20T18:55:38.204 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/567 2024-04-20T18:55:38.204 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:38.216 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:38.252 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:38.252 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/568) 2024-04-20T18:55:38.770 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:38.782 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:38.783 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:39.434 INFO:teuthology.orchestra.run.smithi121.stdout:generic/568 1s 2024-04-20T18:55:39.466 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/568 2024-04-20T18:55:39.469 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:39.491 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:39.534 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:39.534 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/569) 2024-04-20T18:55:40.130 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:40.143 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:40.143 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:40.632 INFO:teuthology.orchestra.run.smithi121.stdout:generic/569 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:55:40.646 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/569 2024-04-20T18:55:40.646 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/569 2024-04-20T18:55:40.646 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:40.658 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:40.690 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:40.691 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/570) 2024-04-20T18:55:41.283 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:41.296 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:41.297 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:41.730 INFO:teuthology.orchestra.run.smithi121.stdout:generic/570 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:55:41.742 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/570 2024-04-20T18:55:41.742 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/570 2024-04-20T18:55:41.742 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:41.752 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:41.781 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:41.781 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/571) 2024-04-20T18:55:42.367 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:42.379 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:42.379 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:43.236 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:55:43.245 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:55:49.067 INFO:teuthology.orchestra.run.smithi121.stdout:generic/571 7s 2024-04-20T18:55:49.083 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/571 2024-04-20T18:55:49.083 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:49.092 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:49.120 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:49.120 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/572) 2024-04-20T18:55:49.810 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:49.821 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:49.822 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:50.234 INFO:teuthology.orchestra.run.smithi121.stdout:generic/572 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:55:50.245 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/572 2024-04-20T18:55:50.246 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/572 2024-04-20T18:55:50.246 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:50.254 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:50.281 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:50.281 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/573) 2024-04-20T18:55:50.810 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:50.822 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:50.823 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:51.244 INFO:teuthology.orchestra.run.smithi121.stdout:generic/573 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:55:51.255 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/573 2024-04-20T18:55:51.255 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/573 2024-04-20T18:55:51.256 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:51.264 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:51.292 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:51.293 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/574) 2024-04-20T18:55:51.836 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:51.850 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:51.850 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:52.269 INFO:teuthology.orchestra.run.smithi121.stdout:generic/574 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:55:52.281 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/574 2024-04-20T18:55:52.281 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/574 2024-04-20T18:55:52.281 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:52.291 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:52.323 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:52.323 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/575) 2024-04-20T18:55:52.831 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:52.843 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:52.843 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:53.268 INFO:teuthology.orchestra.run.smithi121.stdout:generic/575 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:55:53.279 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/575 2024-04-20T18:55:53.279 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/575 2024-04-20T18:55:53.279 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:53.287 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:53.313 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:53.313 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/576) 2024-04-20T18:55:53.831 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:53.843 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:53.844 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:54.429 INFO:teuthology.orchestra.run.smithi121.stdout:generic/576 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:55:54.442 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/576 2024-04-20T18:55:54.442 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/576 2024-04-20T18:55:54.442 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:54.453 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:54.484 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:54.484 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/577) 2024-04-20T18:55:55.070 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:55.085 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:55.085 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:55.504 INFO:teuthology.orchestra.run.smithi121.stdout:generic/577 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:55:55.517 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/577 2024-04-20T18:55:55.517 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/577 2024-04-20T18:55:55.517 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:55.528 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:55.559 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:55.559 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/578) 2024-04-20T18:55:56.097 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:56.110 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:56.110 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:56.682 INFO:teuthology.orchestra.run.smithi121.stdout:generic/578 [not run] xfs_io fiemap failed (old kernel/wrong fs?) 2024-04-20T18:55:56.695 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/578 2024-04-20T18:55:56.696 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/578 2024-04-20T18:55:56.696 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:56.709 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:56.744 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:56.744 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/579) 2024-04-20T18:55:57.289 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:57.303 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:57.303 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:57.751 INFO:teuthology.orchestra.run.smithi121.stdout:generic/579 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:55:57.765 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/579 2024-04-20T18:55:57.766 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/579 2024-04-20T18:55:57.766 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:57.777 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:57.809 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:57.809 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/580) 2024-04-20T18:55:58.350 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:58.364 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:58.364 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:58.888 INFO:teuthology.orchestra.run.smithi121.stdout:generic/580 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:55:58.900 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/580 2024-04-20T18:55:58.901 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/580 2024-04-20T18:55:58.901 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:55:58.911 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:55:58.940 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:55:58.940 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/581) 2024-04-20T18:55:59.497 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:55:59.512 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:55:59.513 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:56:00.820 INFO:teuthology.orchestra.run.smithi121.stdout:generic/581 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:56:00.832 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/581 2024-04-20T18:56:00.832 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/581 2024-04-20T18:56:00.833 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:56:00.842 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:56:00.871 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:56:00.872 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/582) 2024-04-20T18:56:01.436 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:56:01.449 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:56:01.449 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:56:01.970 INFO:teuthology.orchestra.run.smithi121.stdout:generic/582 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:56:01.981 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/582 2024-04-20T18:56:01.981 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/582 2024-04-20T18:56:01.981 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:56:01.990 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:56:02.017 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:56:02.017 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/583) 2024-04-20T18:56:02.566 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:56:02.579 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:56:02.580 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:56:03.105 INFO:teuthology.orchestra.run.smithi121.stdout:generic/583 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:56:03.116 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/583 2024-04-20T18:56:03.117 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/583 2024-04-20T18:56:03.117 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:56:03.126 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:56:03.157 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:56:03.157 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/584) 2024-04-20T18:56:03.720 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:56:03.733 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:56:03.734 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:56:04.236 INFO:teuthology.orchestra.run.smithi121.stdout:generic/584 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:56:04.247 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/584 2024-04-20T18:56:04.247 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/584 2024-04-20T18:56:04.247 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:56:04.255 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:56:04.285 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:56:04.285 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/585) 2024-04-20T18:56:04.837 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:56:04.850 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:56:04.850 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:56:05.263 INFO:teuthology.orchestra.run.smithi121.stdout:generic/585 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:56:05.275 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/585 2024-04-20T18:56:05.275 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/585 2024-04-20T18:56:05.275 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:56:05.285 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:56:05.313 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:56:05.314 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/586) 2024-04-20T18:56:05.824 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:56:05.837 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:56:05.837 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:56:13.442 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:56:13.454 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:56:43.647 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:56:43.657 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:56:50.987 INFO:teuthology.orchestra.run.smithi121.stdout:generic/586 45s 2024-04-20T18:56:51.009 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/586 2024-04-20T18:56:51.010 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:56:51.021 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:56:51.054 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:56:51.054 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/587) 2024-04-20T18:56:51.834 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:56:51.860 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:56:51.860 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:56:52.498 INFO:teuthology.orchestra.run.smithi121.stdout:generic/587 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2024-04-20T18:56:52.508 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/587 2024-04-20T18:56:52.508 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/587 2024-04-20T18:56:52.508 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:56:52.517 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:56:52.543 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:56:52.543 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/588) 2024-04-20T18:56:53.074 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:56:53.086 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:56:53.086 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:56:53.576 INFO:teuthology.orchestra.run.smithi121.stdout:generic/588 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:56:53.589 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/588 2024-04-20T18:56:53.590 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/588 2024-04-20T18:56:53.590 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:56:53.602 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:56:53.634 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:56:53.634 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/589) 2024-04-20T18:56:54.198 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:56:54.211 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:56:54.211 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:56:54.654 INFO:teuthology.orchestra.run.smithi121.stdout:generic/589 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:56:54.665 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/589 2024-04-20T18:56:54.665 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/589 2024-04-20T18:56:54.665 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:56:54.674 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:56:54.697 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:56:54.697 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/590) 2024-04-20T18:56:55.245 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:56:55.258 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:56:55.258 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:56:55.712 INFO:teuthology.orchestra.run.smithi121.stdout:generic/590 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:56:55.724 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/590 2024-04-20T18:56:55.724 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/590 2024-04-20T18:56:55.724 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:56:55.733 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:56:55.764 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:56:55.765 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/591) 2024-04-20T18:56:56.280 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:56:56.291 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:56:56.291 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:56:57.130 INFO:teuthology.orchestra.run.smithi121.stdout:generic/591 1s 2024-04-20T18:56:57.148 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/591 2024-04-20T18:56:57.148 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:56:57.157 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:56:57.186 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:56:57.186 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/592) 2024-04-20T18:56:57.912 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:56:57.923 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:56:57.923 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:56:58.484 INFO:teuthology.orchestra.run.smithi121.stdout:generic/592 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:56:58.496 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/592 2024-04-20T18:56:58.496 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/592 2024-04-20T18:56:58.496 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:56:58.506 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:56:58.535 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:56:58.535 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/593) 2024-04-20T18:56:59.025 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:56:59.036 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:56:59.036 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:56:59.548 INFO:teuthology.orchestra.run.smithi121.stdout:generic/593 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:56:59.559 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/593 2024-04-20T18:56:59.559 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/593 2024-04-20T18:56:59.559 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:56:59.568 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:56:59.598 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:56:59.598 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/594) 2024-04-20T18:57:00.135 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:57:00.149 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:57:00.149 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:00.529 INFO:teuthology.orchestra.run.smithi121.stdout:generic/594 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:57:00.540 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/594 2024-04-20T18:57:00.540 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/594 2024-04-20T18:57:00.540 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:57:00.549 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:00.577 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:57:00.577 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/595) 2024-04-20T18:57:01.130 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:57:01.143 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:57:01.143 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:01.655 INFO:teuthology.orchestra.run.smithi121.stdout:generic/595 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:57:01.669 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/595 2024-04-20T18:57:01.669 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/595 2024-04-20T18:57:01.669 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:57:01.681 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:01.716 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:57:01.716 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/596) 2024-04-20T18:57:02.263 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:57:02.275 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:57:02.275 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:02.833 INFO:teuthology.orchestra.run.smithi121.stdout:generic/596 [not run] file system doesn't support chattr +S 2024-04-20T18:57:02.847 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/596 2024-04-20T18:57:02.847 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/596 2024-04-20T18:57:02.847 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:57:02.858 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:02.889 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:57:02.889 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/597) 2024-04-20T18:57:03.424 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:57:03.435 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:57:03.435 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:05.158 INFO:teuthology.orchestra.run.smithi121.stdout:generic/597 - output mismatch (see /tmp/tmp.yONpfXTHudxfstests-dev/results//generic/597.out.bad) 2024-04-20T18:57:05.165 INFO:teuthology.orchestra.run.smithi121.stdout: --- tests/generic/597.out 2024-04-20 10:23:05.981111963 +0000 2024-04-20T18:57:05.165 INFO:teuthology.orchestra.run.smithi121.stdout: +++ /tmp/tmp.yONpfXTHudxfstests-dev/results//generic/597.out.bad 2024-04-20 18:57:05.154166553 +0000 2024-04-20T18:57:05.165 INFO:teuthology.orchestra.run.smithi121.stdout: @@ -4,11 +4,11 @@ 2024-04-20T18:57:05.165 INFO:teuthology.orchestra.run.smithi121.stdout: fs.protected_symlinks = 0 2024-04-20T18:57:05.165 INFO:teuthology.orchestra.run.smithi121.stdout: successfully followed symlink 2024-04-20T18:57:05.165 INFO:teuthology.orchestra.run.smithi121.stdout: fs.protected_symlinks = 1 2024-04-20T18:57:05.166 INFO:teuthology.orchestra.run.smithi121.stdout: -Permission denied 2024-04-20T18:57:05.166 INFO:teuthology.orchestra.run.smithi121.stdout: +successfully followed symlink 2024-04-20T18:57:05.166 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:05.166 INFO:teuthology.orchestra.run.smithi121.stdout: == Test hardlink create protection when 2024-04-20T18:57:05.166 INFO:teuthology.orchestra.run.smithi121.stdout: == process != target owner and process cannot read target 2024-04-20T18:57:05.166 INFO:teuthology.orchestra.run.smithi121.stdout: fs.protected_hardlinks = 0 2024-04-20T18:57:05.166 INFO:teuthology.orchestra.run.smithi121.stdout: -successfully created hardlink 2024-04-20T18:57:05.166 INFO:teuthology.orchestra.run.smithi121.stdout: +Permission denied 2024-04-20T18:57:05.166 INFO:teuthology.orchestra.run.smithi121.stdout: fs.protected_hardlinks = 1 2024-04-20T18:57:05.166 INFO:teuthology.orchestra.run.smithi121.stdout: -ln: failed to create hard link 'TEST_DIR/597/sticky_dir/hardlink' => 'TEST_DIR/597/target': Operation not permitted 2024-04-20T18:57:05.166 INFO:teuthology.orchestra.run.smithi121.stdout: +Permission denied 2024-04-20T18:57:05.173 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/597 2024-04-20T18:57:05.173 INFO:teuthology.orchestra.run.smithi121.stdout:Failures: generic/597 2024-04-20T18:57:05.173 INFO:teuthology.orchestra.run.smithi121.stdout:Failed 1 of 1 tests 2024-04-20T18:57:05.182 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:05.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T18:57:05.216 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-20T18:57:05.216 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-20T18:57:05.217 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/598) 2024-04-20T18:57:05.936 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:57:05.948 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:57:05.948 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:07.859 INFO:teuthology.orchestra.run.smithi121.stdout:generic/598 - output mismatch (see /tmp/tmp.yONpfXTHudxfstests-dev/results//generic/598.out.bad) 2024-04-20T18:57:07.869 INFO:teuthology.orchestra.run.smithi121.stdout: --- tests/generic/598.out 2024-04-20 10:23:05.981111963 +0000 2024-04-20T18:57:07.870 INFO:teuthology.orchestra.run.smithi121.stdout: +++ /tmp/tmp.yONpfXTHudxfstests-dev/results//generic/598.out.bad 2024-04-20 18:57:07.735133175 +0000 2024-04-20T18:57:07.870 INFO:teuthology.orchestra.run.smithi121.stdout: @@ -5,13 +5,10 @@ 2024-04-20T18:57:07.870 INFO:teuthology.orchestra.run.smithi121.stdout: = only group writable dir 2024-04-20T18:57:07.870 INFO:teuthology.orchestra.run.smithi121.stdout: fs.protected_regular = 1 2024-04-20T18:57:07.870 INFO:teuthology.orchestra.run.smithi121.stdout: = group & world writable dir 2024-04-20T18:57:07.870 INFO:teuthology.orchestra.run.smithi121.stdout: -Permission denied 2024-04-20T18:57:07.870 INFO:teuthology.orchestra.run.smithi121.stdout: = only group writable dir 2024-04-20T18:57:07.870 INFO:teuthology.orchestra.run.smithi121.stdout: fs.protected_regular = 2 2024-04-20T18:57:07.870 INFO:teuthology.orchestra.run.smithi121.stdout: = group & world writable dir 2024-04-20T18:57:07.870 INFO:teuthology.orchestra.run.smithi121.stdout: -Permission denied 2024-04-20T18:57:07.870 INFO:teuthology.orchestra.run.smithi121.stdout: = only group writable dir 2024-04-20T18:57:07.870 INFO:teuthology.orchestra.run.smithi121.stdout: -Permission denied 2024-04-20T18:57:07.870 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:07.871 INFO:teuthology.orchestra.run.smithi121.stdout: == Test fifo open when owned by another and fifo owner != dir owner 2024-04-20T18:57:07.871 INFO:teuthology.orchestra.run.smithi121.stdout: fs.protected_fifos = 0 2024-04-20T18:57:07.871 INFO:teuthology.orchestra.run.smithi121.stdout: @@ -19,10 +16,7 @@ 2024-04-20T18:57:07.871 INFO:teuthology.orchestra.run.smithi121.stdout: = only group writable dir 2024-04-20T18:57:07.871 INFO:teuthology.orchestra.run.smithi121.stdout: fs.protected_fifos = 1 2024-04-20T18:57:07.871 INFO:teuthology.orchestra.run.smithi121.stdout: = group & world writable dir 2024-04-20T18:57:07.871 INFO:teuthology.orchestra.run.smithi121.stdout: -Permission denied 2024-04-20T18:57:07.871 INFO:teuthology.orchestra.run.smithi121.stdout: = only group writable dir 2024-04-20T18:57:07.871 INFO:teuthology.orchestra.run.smithi121.stdout: fs.protected_fifos = 2 2024-04-20T18:57:07.871 INFO:teuthology.orchestra.run.smithi121.stdout: = group & world writable dir 2024-04-20T18:57:07.871 INFO:teuthology.orchestra.run.smithi121.stdout: -Permission denied 2024-04-20T18:57:07.871 INFO:teuthology.orchestra.run.smithi121.stdout: = only group writable dir 2024-04-20T18:57:07.871 INFO:teuthology.orchestra.run.smithi121.stdout: -Permission denied 2024-04-20T18:57:07.878 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/598 2024-04-20T18:57:07.878 INFO:teuthology.orchestra.run.smithi121.stdout:Failures: generic/598 2024-04-20T18:57:07.879 INFO:teuthology.orchestra.run.smithi121.stdout:Failed 1 of 1 tests 2024-04-20T18:57:07.889 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:07.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T18:57:07.920 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-20T18:57:07.920 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-20T18:57:07.920 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/599) 2024-04-20T18:57:08.630 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:57:08.642 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:57:08.643 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:09.063 INFO:teuthology.orchestra.run.smithi121.stdout:generic/599 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:57:09.075 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/599 2024-04-20T18:57:09.075 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/599 2024-04-20T18:57:09.075 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:57:09.084 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:09.111 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:57:09.112 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/600) 2024-04-20T18:57:09.653 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:57:09.665 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:57:09.665 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:10.111 INFO:teuthology.orchestra.run.smithi121.stdout:generic/600 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:57:10.122 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/600 2024-04-20T18:57:10.122 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/600 2024-04-20T18:57:10.122 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:57:10.132 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:10.162 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:57:10.162 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/601) 2024-04-20T18:57:10.698 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:57:10.710 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:57:10.710 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:11.118 INFO:teuthology.orchestra.run.smithi121.stdout:generic/601 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:57:11.133 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/601 2024-04-20T18:57:11.133 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/601 2024-04-20T18:57:11.133 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:57:11.146 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:11.182 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:57:11.182 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/602) 2024-04-20T18:57:11.745 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:57:11.757 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:57:11.758 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:12.245 INFO:teuthology.orchestra.run.smithi121.stdout:generic/602 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:57:12.259 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/602 2024-04-20T18:57:12.259 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/602 2024-04-20T18:57:12.260 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:57:12.271 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:12.304 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:57:12.304 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/603) 2024-04-20T18:57:12.860 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:57:12.872 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:57:12.872 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:13.324 INFO:teuthology.orchestra.run.smithi121.stdout:generic/603 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:57:13.334 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/603 2024-04-20T18:57:13.334 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/603 2024-04-20T18:57:13.335 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:57:13.343 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:13.372 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:57:13.372 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/604) 2024-04-20T18:57:13.777 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:57:13.781 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:57:13.907 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:57:13.919 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:57:13.919 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:14.310 INFO:teuthology.orchestra.run.smithi121.stdout:generic/604 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:57:14.321 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/604 2024-04-20T18:57:14.321 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/604 2024-04-20T18:57:14.321 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:57:14.329 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:14.356 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:57:14.357 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/605) 2024-04-20T18:57:14.886 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:57:14.897 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:57:14.898 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:15.249 INFO:teuthology.orchestra.run.smithi121.stdout:generic/605 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:57:15.258 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/605 2024-04-20T18:57:15.258 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/605 2024-04-20T18:57:15.258 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:57:15.264 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:15.287 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:57:15.287 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/606) 2024-04-20T18:57:15.757 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:57:15.769 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:57:15.769 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:16.169 INFO:teuthology.orchestra.run.smithi121.stdout:generic/606 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:57:16.181 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/606 2024-04-20T18:57:16.181 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/606 2024-04-20T18:57:16.182 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:57:16.193 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:16.222 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:57:16.223 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/607) 2024-04-20T18:57:16.743 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:57:16.755 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:57:16.755 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:17.175 INFO:teuthology.orchestra.run.smithi121.stdout:generic/607 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:57:17.186 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/607 2024-04-20T18:57:17.187 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/607 2024-04-20T18:57:17.187 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:57:17.196 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:17.226 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:57:17.226 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/608) 2024-04-20T18:57:17.755 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:57:17.768 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:57:17.768 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:18.174 INFO:teuthology.orchestra.run.smithi121.stdout:generic/608 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:57:18.189 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/608 2024-04-20T18:57:18.189 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/608 2024-04-20T18:57:18.189 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:57:18.200 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:18.235 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:57:18.235 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/609) 2024-04-20T18:57:18.769 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:57:18.782 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:57:18.782 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:19.856 INFO:teuthology.orchestra.run.smithi121.stdout:generic/609 1s 2024-04-20T18:57:19.875 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/609 2024-04-20T18:57:19.875 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:57:19.885 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:19.961 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:57:19.961 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/610) 2024-04-20T18:57:20.629 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:57:20.642 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:57:20.642 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:21.080 INFO:teuthology.orchestra.run.smithi121.stdout:generic/610 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:57:21.091 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/610 2024-04-20T18:57:21.092 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/610 2024-04-20T18:57:21.092 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:57:21.101 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:21.133 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:57:21.133 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/611) 2024-04-20T18:57:21.676 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:57:21.690 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:57:21.690 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:22.116 INFO:teuthology.orchestra.run.smithi121.stdout:generic/611 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:57:22.126 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/611 2024-04-20T18:57:22.126 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/611 2024-04-20T18:57:22.126 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:57:22.135 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:22.162 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:57:22.162 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/612) 2024-04-20T18:57:22.719 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:57:22.732 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:57:22.732 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:23.306 INFO:teuthology.orchestra.run.smithi121.stdout:generic/612 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-20T18:57:23.320 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/612 2024-04-20T18:57:23.320 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/612 2024-04-20T18:57:23.320 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:57:23.332 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:23.379 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:57:23.379 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/613) 2024-04-20T18:57:24.072 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:57:24.084 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:57:24.084 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:24.618 INFO:teuthology.orchestra.run.smithi121.stdout:generic/613 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:57:24.630 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/613 2024-04-20T18:57:24.630 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/613 2024-04-20T18:57:24.631 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:57:24.640 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:24.670 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:57:24.670 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/614) 2024-04-20T18:57:25.204 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:57:25.217 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:57:25.217 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:25.649 INFO:teuthology.orchestra.run.smithi121.stdout:generic/614 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:57:25.664 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/614 2024-04-20T18:57:25.664 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/614 2024-04-20T18:57:25.665 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:57:25.678 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:25.709 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:57:25.709 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/615) 2024-04-20T18:57:26.219 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:57:26.233 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:57:26.233 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:26.649 INFO:teuthology.orchestra.run.smithi121.stdout:generic/615 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:57:26.662 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/615 2024-04-20T18:57:26.662 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/615 2024-04-20T18:57:26.662 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:57:26.672 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:26.700 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:57:26.700 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/616) 2024-04-20T18:57:27.212 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:57:27.223 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:57:27.223 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:27.652 INFO:teuthology.orchestra.run.smithi121.stdout:generic/616 [not run] kernel does not support IO_URING 2024-04-20T18:57:27.662 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/616 2024-04-20T18:57:27.662 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/616 2024-04-20T18:57:27.663 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:57:27.672 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:27.702 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:57:27.702 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/617) 2024-04-20T18:57:28.255 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:57:28.269 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:57:28.269 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:28.810 INFO:teuthology.orchestra.run.smithi121.stdout:generic/617 [not run] kernel does not support IO_URING 2024-04-20T18:57:28.821 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/617 2024-04-20T18:57:28.821 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/617 2024-04-20T18:57:28.821 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:57:28.831 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:28.861 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:57:28.861 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/618) 2024-04-20T18:57:29.379 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:57:29.393 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:57:29.393 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:29.826 INFO:teuthology.orchestra.run.smithi121.stdout:generic/618 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:57:29.838 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/618 2024-04-20T18:57:29.838 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/618 2024-04-20T18:57:29.839 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:57:29.850 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:29.883 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:57:29.884 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/619) 2024-04-20T18:57:30.451 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:57:30.464 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:57:30.464 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:30.886 INFO:teuthology.orchestra.run.smithi121.stdout:generic/619 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:57:30.900 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/619 2024-04-20T18:57:30.900 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/619 2024-04-20T18:57:30.900 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:57:30.912 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:30.942 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:57:30.943 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/620) 2024-04-20T18:57:31.469 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:57:31.483 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:57:31.483 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:31.980 INFO:teuthology.orchestra.run.smithi121.stdout:generic/620 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:57:31.994 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/620 2024-04-20T18:57:31.995 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/620 2024-04-20T18:57:31.995 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:57:32.007 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:32.042 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:57:32.042 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/621) 2024-04-20T18:57:32.572 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:57:32.584 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:57:32.584 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:33.111 INFO:teuthology.orchestra.run.smithi121.stdout:generic/621 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:57:33.124 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/621 2024-04-20T18:57:33.124 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/621 2024-04-20T18:57:33.124 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:57:33.134 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:33.164 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:57:33.165 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/622) 2024-04-20T18:57:33.717 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:57:33.731 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:57:33.731 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:34.168 INFO:teuthology.orchestra.run.smithi121.stdout:generic/622 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:57:34.180 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/622 2024-04-20T18:57:34.180 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/622 2024-04-20T18:57:34.180 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:57:34.189 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:34.218 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:57:34.218 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/623) 2024-04-20T18:57:34.749 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:57:34.763 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:57:34.763 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:35.182 INFO:teuthology.orchestra.run.smithi121.stdout:generic/623 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:57:35.195 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/623 2024-04-20T18:57:35.195 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/623 2024-04-20T18:57:35.195 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:57:35.206 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:35.236 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:57:35.236 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/624) 2024-04-20T18:57:35.766 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:57:35.780 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:57:35.780 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:36.202 INFO:teuthology.orchestra.run.smithi121.stdout:generic/624 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:57:36.218 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/624 2024-04-20T18:57:36.218 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/624 2024-04-20T18:57:36.218 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:57:36.230 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:36.261 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:57:36.262 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/625) 2024-04-20T18:57:36.782 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:57:36.796 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:57:36.796 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:37.215 INFO:teuthology.orchestra.run.smithi121.stdout:generic/625 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:57:37.228 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/625 2024-04-20T18:57:37.229 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/625 2024-04-20T18:57:37.229 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:57:37.241 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:37.273 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:57:37.273 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/626) 2024-04-20T18:57:37.820 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:57:37.834 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:57:37.834 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:38.309 INFO:teuthology.orchestra.run.smithi121.stdout:generic/626 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:57:38.323 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/626 2024-04-20T18:57:38.323 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/626 2024-04-20T18:57:38.323 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:57:38.334 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:38.368 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:57:38.368 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/627) 2024-04-20T18:57:38.911 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:57:38.925 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:57:38.925 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:39.396 INFO:teuthology.orchestra.run.smithi121.stdout:generic/627 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:57:39.409 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/627 2024-04-20T18:57:39.409 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/627 2024-04-20T18:57:39.409 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:57:39.420 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:39.455 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:57:39.455 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/628) 2024-04-20T18:57:40.009 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:57:40.021 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:57:40.022 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:40.466 INFO:teuthology.orchestra.run.smithi121.stdout:generic/628 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:57:40.480 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/628 2024-04-20T18:57:40.480 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/628 2024-04-20T18:57:40.481 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:57:40.491 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:40.524 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:57:40.524 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/629) 2024-04-20T18:57:41.048 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:57:41.062 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:57:41.062 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:41.509 INFO:teuthology.orchestra.run.smithi121.stdout:generic/629 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:57:41.522 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/629 2024-04-20T18:57:41.523 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/629 2024-04-20T18:57:41.523 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:57:41.535 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:41.567 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:57:41.567 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/630) 2024-04-20T18:57:42.105 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:57:42.119 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:57:42.119 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:42.545 INFO:teuthology.orchestra.run.smithi121.stdout:generic/630 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:57:42.556 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/630 2024-04-20T18:57:42.556 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/630 2024-04-20T18:57:42.556 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:57:42.565 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:42.597 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:57:42.597 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/631) 2024-04-20T18:57:43.116 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:57:43.128 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:57:43.128 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:43.544 INFO:teuthology.orchestra.run.smithi121.stdout:generic/631 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:57:43.557 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/631 2024-04-20T18:57:43.557 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/631 2024-04-20T18:57:43.557 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:57:43.567 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:57:43.596 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:57:43.597 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/632) 2024-04-20T18:57:43.907 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:57:43.913 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:57:44.133 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:57:44.146 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:57:44.147 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:58:03.997 INFO:teuthology.orchestra.run.smithi121.stdout:generic/632 19s 2024-04-20T18:58:04.018 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/632 2024-04-20T18:58:04.019 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:58:04.031 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:58:04.068 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:58:04.068 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/633) 2024-04-20T18:58:04.578 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:58:04.592 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:58:04.592 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:58:05.554 INFO:teuthology.orchestra.run.smithi121.stdout:generic/633 [failed, exit status 1]- output mismatch (see /tmp/tmp.yONpfXTHudxfstests-dev/results//generic/633.out.bad) 2024-04-20T18:58:05.578 INFO:teuthology.orchestra.run.smithi121.stdout: --- tests/generic/633.out 2024-04-20 10:23:05.987111885 +0000 2024-04-20T18:58:05.578 INFO:teuthology.orchestra.run.smithi121.stdout: +++ /tmp/tmp.yONpfXTHudxfstests-dev/results//generic/633.out.bad 2024-04-20 18:58:05.541385599 +0000 2024-04-20T18:58:05.578 INFO:teuthology.orchestra.run.smithi121.stdout: @@ -1,2 +1,5 @@ 2024-04-20T18:58:05.578 INFO:teuthology.orchestra.run.smithi121.stdout: QA output created by 633 2024-04-20T18:58:05.578 INFO:teuthology.orchestra.run.smithi121.stdout: Silence is golden 2024-04-20T18:58:05.578 INFO:teuthology.orchestra.run.smithi121.stdout: +vfstest.c: 1164: protected_symlinks - Success - failure: openat 2024-04-20T18:58:05.578 INFO:teuthology.orchestra.run.smithi121.stdout: +vfstest.c: 1171: protected_symlinks - Success - failure: wait_for_pid 2024-04-20T18:58:05.578 INFO:teuthology.orchestra.run.smithi121.stdout: +vfstest.c: 2418: run_test - Success - failure: following protected symlinks on regular mounts 2024-04-20T18:58:05.597 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/633 2024-04-20T18:58:05.597 INFO:teuthology.orchestra.run.smithi121.stdout:Failures: generic/633 2024-04-20T18:58:05.597 INFO:teuthology.orchestra.run.smithi121.stdout:Failed 1 of 1 tests 2024-04-20T18:58:05.616 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:58:05.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T18:58:05.678 INFO:tasks.cephfs.xfstests_dev:Command failed 2024-04-20T18:58:05.678 INFO:tasks.cephfs.xfstests_dev:Command return value: 1 2024-04-20T18:58:05.678 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/634) 2024-04-20T18:58:06.253 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:58:06.265 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:58:06.265 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:58:06.664 INFO:teuthology.orchestra.run.smithi121.stdout:generic/634 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:58:06.674 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/634 2024-04-20T18:58:06.674 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/634 2024-04-20T18:58:06.675 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:58:06.683 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:58:06.712 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:58:06.712 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/635) 2024-04-20T18:58:07.245 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:58:07.260 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:58:07.260 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:58:07.675 INFO:teuthology.orchestra.run.smithi121.stdout:generic/635 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:58:07.685 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/635 2024-04-20T18:58:07.685 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/635 2024-04-20T18:58:07.685 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:58:07.694 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:58:07.724 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:58:07.724 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/636) 2024-04-20T18:58:08.243 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:58:08.259 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:58:08.259 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:58:08.671 INFO:teuthology.orchestra.run.smithi121.stdout:generic/636 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:58:08.684 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/636 2024-04-20T18:58:08.684 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/636 2024-04-20T18:58:08.684 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:58:08.696 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:58:08.728 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:58:08.728 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/637) 2024-04-20T18:58:09.266 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:58:09.282 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:58:09.282 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:58:12.451 INFO:teuthology.orchestra.run.smithi121.stdout:generic/637 3s 2024-04-20T18:58:12.466 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/637 2024-04-20T18:58:12.466 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:58:12.473 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:58:12.500 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:58:12.501 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/638) 2024-04-20T18:58:13.231 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:58:13.243 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:58:13.243 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:58:13.756 INFO:teuthology.orchestra.run.smithi121.stdout:generic/638 0s 2024-04-20T18:58:13.771 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/638 2024-04-20T18:58:13.771 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:58:13.780 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:58:13.812 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:58:13.813 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/639) 2024-04-20T18:58:14.054 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:58:14.059 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:58:14.334 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:58:14.347 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:58:14.347 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:58:19.950 INFO:teuthology.orchestra.run.smithi121.stdout:generic/639 5s 2024-04-20T18:58:19.966 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/639 2024-04-20T18:58:19.966 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:58:19.975 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:58:20.001 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:58:20.001 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/640) 2024-04-20T18:58:20.565 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:58:20.577 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:58:20.577 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:58:21.001 INFO:teuthology.orchestra.run.smithi121.stdout:generic/640 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:58:21.015 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/640 2024-04-20T18:58:21.016 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/640 2024-04-20T18:58:21.016 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:58:21.027 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:58:21.061 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:58:21.061 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/641) 2024-04-20T18:58:21.586 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:58:21.600 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:58:21.600 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:58:22.013 INFO:teuthology.orchestra.run.smithi121.stdout:generic/641 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:58:22.027 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/641 2024-04-20T18:58:22.027 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/641 2024-04-20T18:58:22.027 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:58:22.039 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:58:22.071 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:58:22.071 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/642) 2024-04-20T18:58:22.600 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:58:22.612 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:58:22.612 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:58:23.045 INFO:teuthology.orchestra.run.smithi121.stdout:generic/642 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:58:23.059 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/642 2024-04-20T18:58:23.059 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/642 2024-04-20T18:58:23.059 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:58:23.071 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:58:23.108 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:58:23.108 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/643) 2024-04-20T18:58:23.662 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:58:23.675 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:58:23.675 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:58:24.062 INFO:teuthology.orchestra.run.smithi121.stdout:generic/643 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:58:24.074 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/643 2024-04-20T18:58:24.074 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/643 2024-04-20T18:58:24.074 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:58:24.083 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:58:24.115 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:58:24.115 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/644) 2024-04-20T18:58:24.650 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:58:24.664 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:58:24.664 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:58:25.120 INFO:teuthology.orchestra.run.smithi121.stdout:generic/644 [not run] vfstest not support by ceph-fuse 2024-04-20T18:58:25.135 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/644 2024-04-20T18:58:25.135 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/644 2024-04-20T18:58:25.135 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:58:25.147 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:58:25.180 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:58:25.180 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/645) 2024-04-20T18:58:25.741 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:58:25.755 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:58:25.755 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:58:26.154 INFO:teuthology.orchestra.run.smithi121.stdout:generic/645 [not run] vfstest not support by ceph-fuse 2024-04-20T18:58:26.165 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/645 2024-04-20T18:58:26.165 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/645 2024-04-20T18:58:26.165 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:58:26.174 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:58:26.203 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:58:26.203 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/646) 2024-04-20T18:58:26.743 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:58:26.757 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:58:26.757 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:58:27.188 INFO:teuthology.orchestra.run.smithi121.stdout:generic/646 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:58:27.199 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/646 2024-04-20T18:58:27.199 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/646 2024-04-20T18:58:27.199 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:58:27.209 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:58:27.239 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:58:27.239 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/647) 2024-04-20T18:58:27.772 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:58:27.787 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:58:27.787 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:58:28.634 INFO:teuthology.orchestra.run.smithi121.stdout:generic/647 1s 2024-04-20T18:58:28.652 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/647 2024-04-20T18:58:28.652 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:58:28.661 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:58:28.700 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:58:28.700 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/648) 2024-04-20T18:58:29.393 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:58:29.404 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:58:29.404 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:58:29.833 INFO:teuthology.orchestra.run.smithi121.stdout:generic/648 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T18:58:29.844 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/648 2024-04-20T18:58:29.844 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/648 2024-04-20T18:58:29.844 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:58:29.853 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:58:29.881 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:58:29.881 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/649) 2024-04-20T18:58:30.415 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:58:30.428 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:58:30.428 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:58:30.965 INFO:teuthology.orchestra.run.smithi121.stdout:generic/649 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-20T18:58:30.980 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/649 2024-04-20T18:58:30.980 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/649 2024-04-20T18:58:30.980 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T18:58:30.993 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:58:31.036 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T18:58:31.036 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/650) 2024-04-20T18:58:31.738 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T18:58:31.751 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T18:58:31.751 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T18:58:44.174 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:58:44.183 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:59:14.353 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:59:14.365 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:59:44.521 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:59:44.530 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T18:59:44.630 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-04-20T18:59:44.630+0000 7f4dce800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 45618) UID: 0 2024-04-20T18:59:44.631 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-04-20T18:59:44.630+0000 7f1143800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 45618) UID: 0 2024-04-20T18:59:44.632 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-04-20T18:59:44.630+0000 7f5d77a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 45618) UID: 0 2024-04-20T18:59:44.632 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T18:59:44.630+0000 7fad6ce00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 45618) UID: 0 2024-04-20T19:03:27.958 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:03:27.973 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:03:28.148 INFO:tasks.ceph.osd.3.smithi121.stderr:2024-04-20T19:03:28.149+0000 7fca40a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 490294) UID: 0 2024-04-20T19:03:28.149 INFO:tasks.ceph.osd.2.smithi121.stderr:2024-04-20T19:03:28.150+0000 7f11dfa00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 490294) UID: 0 2024-04-20T19:03:28.152 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-04-20T19:03:28.153+0000 7fb3ce400640 -1 Fail to open '/proc/490294/cmdline' error = (2) No such file or directory 2024-04-20T19:03:28.152 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-04-20T19:03:28.153+0000 7fb3ce400640 -1 received signal: Hangup from (PID: 490294) UID: 0 2024-04-20T19:03:28.160 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T19:03:28.159+0000 7f9c11a00640 -1 Fail to open '/proc/490294/cmdline' error = (2) No such file or directory 2024-04-20T19:03:28.160 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T19:03:28.159+0000 7f9c11a00640 -1 received signal: Hangup from (PID: 490294) UID: 0 2024-04-20T19:08:51.458 INFO:teuthology.orchestra.run.smithi121.stderr:error: Compressing program wrote following message to stderr when compressing log /var/log/ceph/ceph-osd.1.log.1: 2024-04-20T19:08:51.458 INFO:teuthology.orchestra.run.smithi121.stderr:gzip: stdin: file size changed while zipping 2024-04-20T19:09:25.785 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:09:25.801 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:09:25.877 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-04-20T19:09:25.877+0000 7f4dce800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 45686) UID: 0 2024-04-20T19:09:25.878 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-04-20T19:09:25.877+0000 7f5d77a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 45686) UID: 0 2024-04-20T19:09:25.878 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T19:09:25.877+0000 7fad6ce00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 45686) UID: 0 2024-04-20T19:09:25.878 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-04-20T19:09:25.877+0000 7f1143800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 45686) UID: 0 2024-04-20T19:09:26.006 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-04-20T19:09:26.005+0000 7fb3ce400640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 491332) UID: 0 2024-04-20T19:09:26.006 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T19:09:26.005+0000 7f9c11a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 491332) UID: 0 2024-04-20T19:09:26.007 INFO:tasks.ceph.osd.2.smithi121.stderr:2024-04-20T19:09:26.006+0000 7f11dfa00640 -1 received signal: Hangup from (PID: 491332) UID: 0 2024-04-20T19:09:26.008 INFO:tasks.ceph.osd.3.smithi121.stderr:2024-04-20T19:09:26.006+0000 7fca40a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 491332) UID: 0 2024-04-20T19:15:48.961 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:15:48.975 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:15:49.134 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-04-20T19:15:49.134+0000 7f4dce800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 45726) UID: 0 2024-04-20T19:15:49.134 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-04-20T19:15:49.134+0000 7f1143800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 45726) UID: 0 2024-04-20T19:15:49.135 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T19:15:49.135+0000 7fad6ce00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 45726) UID: 0 2024-04-20T19:15:49.135 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-04-20T19:15:49.134+0000 7f5d77a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 45726) UID: 0 2024-04-20T19:19:37.678 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:19:37.684 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:19:37.783 INFO:tasks.ceph.mds.a.smithi121.stderr:2024-04-20T19:19:37.784+0000 7f95c3c00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 493244) UID: 0 2024-04-20T19:19:37.783 INFO:tasks.ceph.mds.c.smithi121.stderr:2024-04-20T19:19:37.784+0000 7f4d9da00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 493244) UID: 0 2024-04-20T19:25:11.960 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:25:11.972 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:25:42.290 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:25:42.298 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:26:13.015 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:26:13.020 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:26:43.342 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:26:43.350 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:27:14.241 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:27:14.253 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:27:45.322 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:27:45.330 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:28:15.532 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:28:15.539 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:28:45.774 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:28:45.780 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:29:15.909 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:29:15.914 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:29:46.226 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:29:46.236 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:30:16.350 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:30:16.357 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:30:46.766 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:30:46.780 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:31:17.024 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:31:17.033 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:31:47.608 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:31:47.619 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:32:17.818 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:32:17.823 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:32:48.667 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:32:48.679 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:33:18.913 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:33:18.920 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:33:49.988 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:33:50.000 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:34:20.202 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:34:20.217 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:34:50.574 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:34:50.589 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:35:20.954 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:35:20.969 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:35:51.212 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:35:51.220 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:36:21.382 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:36:21.389 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:36:51.528 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:36:51.534 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:37:21.756 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:37:21.767 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:37:51.880 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:37:51.888 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:38:22.056 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:38:22.068 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:38:22.155 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-04-20T19:38:22.155+0000 7f4dce800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 46275) UID: 0 2024-04-20T19:38:22.155 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-04-20T19:38:22.155+0000 7f5d77a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 46275) UID: 0 2024-04-20T19:38:22.155 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T19:38:22.155+0000 7fad6ce00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 46275) UID: 0 2024-04-20T19:38:22.156 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-04-20T19:38:22.155+0000 7f1143800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 46275) UID: 0 2024-04-20T19:42:04.325 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:42:04.343 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:42:04.868 INFO:tasks.ceph.mds.c.smithi121.stderr:2024-04-20T19:42:04.869+0000 7f4d9da00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 497929) UID: 0 2024-04-20T19:42:04.868 INFO:tasks.ceph.mds.a.smithi121.stderr:2024-04-20T19:42:04.869+0000 7f95c3c00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 497929) UID: 0 2024-04-20T19:46:52.193 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T19:46:52.190+0000 7f9c11a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 498798) UID: 0 2024-04-20T19:46:52.196 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-04-20T19:46:52.187+0000 7fb3ce400640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 498798) UID: 0 2024-04-20T19:46:52.206 INFO:tasks.ceph.osd.2.smithi121.stderr:2024-04-20T19:46:52.194+0000 7f11dfa00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 498798) UID: 0 2024-04-20T19:46:52.212 INFO:tasks.ceph.osd.3.smithi121.stderr:2024-04-20T19:46:52.192+0000 7fca40a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 498798) UID: 0 2024-04-20T19:58:01.285 INFO:teuthology.orchestra.run.smithi121.stdout:generic/650 3567s 2024-04-20T19:58:01.368 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/650 2024-04-20T19:58:01.368 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T19:58:01.376 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:01.416 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T19:58:01.417 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/651) 2024-04-20T19:58:03.065 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T19:58:03.077 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T19:58:03.078 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:03.567 INFO:teuthology.orchestra.run.smithi121.stdout:generic/651 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T19:58:03.580 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/651 2024-04-20T19:58:03.581 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/651 2024-04-20T19:58:03.581 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T19:58:03.591 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:03.622 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T19:58:03.622 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/652) 2024-04-20T19:58:04.265 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T19:58:04.282 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T19:58:04.282 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:04.697 INFO:teuthology.orchestra.run.smithi121.stdout:generic/652 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T19:58:04.709 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/652 2024-04-20T19:58:04.709 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/652 2024-04-20T19:58:04.709 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T19:58:04.718 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:04.752 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T19:58:04.752 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/653) 2024-04-20T19:58:05.360 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T19:58:05.374 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T19:58:05.374 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:06.224 INFO:teuthology.orchestra.run.smithi121.stdout:generic/653 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T19:58:06.237 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/653 2024-04-20T19:58:06.237 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/653 2024-04-20T19:58:06.237 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T19:58:06.251 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:06.292 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T19:58:06.292 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/654) 2024-04-20T19:58:06.889 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T19:58:06.902 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T19:58:06.902 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:07.393 INFO:teuthology.orchestra.run.smithi121.stdout:generic/654 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T19:58:07.406 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/654 2024-04-20T19:58:07.407 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/654 2024-04-20T19:58:07.407 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T19:58:07.415 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:07.443 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T19:58:07.444 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/655) 2024-04-20T19:58:08.094 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T19:58:08.110 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T19:58:08.110 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:08.579 INFO:teuthology.orchestra.run.smithi121.stdout:generic/655 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T19:58:08.592 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/655 2024-04-20T19:58:08.592 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/655 2024-04-20T19:58:08.592 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T19:58:08.601 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:08.636 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T19:58:08.637 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/656) 2024-04-20T19:58:09.218 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T19:58:09.233 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T19:58:09.233 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:09.863 INFO:teuthology.orchestra.run.smithi121.stdout:generic/656 [not run] vfstest not support by ceph-fuse 2024-04-20T19:58:09.873 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/656 2024-04-20T19:58:09.873 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/656 2024-04-20T19:58:09.873 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T19:58:09.881 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:09.914 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T19:58:09.914 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/657) 2024-04-20T19:58:10.551 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T19:58:10.568 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T19:58:10.569 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:11.015 INFO:teuthology.orchestra.run.smithi121.stdout:generic/657 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T19:58:11.027 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/657 2024-04-20T19:58:11.027 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/657 2024-04-20T19:58:11.027 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T19:58:11.036 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:11.069 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T19:58:11.069 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/658) 2024-04-20T19:58:11.643 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T19:58:11.660 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T19:58:11.660 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:12.094 INFO:teuthology.orchestra.run.smithi121.stdout:generic/658 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T19:58:12.108 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/658 2024-04-20T19:58:12.108 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/658 2024-04-20T19:58:12.108 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T19:58:12.120 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:12.161 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T19:58:12.161 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/659) 2024-04-20T19:58:12.824 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T19:58:12.843 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T19:58:12.843 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:13.257 INFO:teuthology.orchestra.run.smithi121.stdout:generic/659 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T19:58:13.267 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/659 2024-04-20T19:58:13.267 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/659 2024-04-20T19:58:13.268 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T19:58:13.277 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:13.311 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T19:58:13.311 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/660) 2024-04-20T19:58:13.962 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T19:58:13.977 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T19:58:13.978 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:14.389 INFO:teuthology.orchestra.run.smithi121.stdout:generic/660 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T19:58:14.400 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/660 2024-04-20T19:58:14.400 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/660 2024-04-20T19:58:14.400 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T19:58:14.408 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:14.442 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T19:58:14.442 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/661) 2024-04-20T19:58:15.069 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T19:58:15.092 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T19:58:15.092 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:15.494 INFO:teuthology.orchestra.run.smithi121.stdout:generic/661 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T19:58:15.508 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/661 2024-04-20T19:58:15.510 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/661 2024-04-20T19:58:15.510 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T19:58:15.522 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:15.556 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T19:58:15.556 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/662) 2024-04-20T19:58:16.169 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T19:58:16.182 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T19:58:16.182 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:16.718 INFO:teuthology.orchestra.run.smithi121.stdout:generic/662 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T19:58:16.729 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/662 2024-04-20T19:58:16.729 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/662 2024-04-20T19:58:16.729 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T19:58:16.741 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:16.778 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T19:58:16.779 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/663) 2024-04-20T19:58:17.405 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T19:58:17.421 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T19:58:17.421 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:17.853 INFO:teuthology.orchestra.run.smithi121.stdout:generic/663 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T19:58:17.865 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/663 2024-04-20T19:58:17.865 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/663 2024-04-20T19:58:17.865 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T19:58:17.877 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:17.921 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T19:58:17.921 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/664) 2024-04-20T19:58:18.518 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T19:58:18.537 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T19:58:18.538 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:18.976 INFO:teuthology.orchestra.run.smithi121.stdout:generic/664 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T19:58:18.989 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/664 2024-04-20T19:58:18.990 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/664 2024-04-20T19:58:18.990 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T19:58:19.001 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:19.032 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T19:58:19.032 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/665) 2024-04-20T19:58:19.646 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T19:58:19.663 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T19:58:19.663 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:20.103 INFO:teuthology.orchestra.run.smithi121.stdout:generic/665 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T19:58:20.118 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/665 2024-04-20T19:58:20.118 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/665 2024-04-20T19:58:20.118 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T19:58:20.129 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:20.166 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T19:58:20.167 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/666) 2024-04-20T19:58:20.767 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T19:58:20.780 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T19:58:20.780 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:21.218 INFO:teuthology.orchestra.run.smithi121.stdout:generic/666 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T19:58:21.233 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/666 2024-04-20T19:58:21.233 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/666 2024-04-20T19:58:21.233 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T19:58:21.247 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:21.282 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T19:58:21.282 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/667) 2024-04-20T19:58:21.899 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T19:58:21.918 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T19:58:21.918 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:22.348 INFO:teuthology.orchestra.run.smithi121.stdout:generic/667 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T19:58:22.363 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/667 2024-04-20T19:58:22.363 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/667 2024-04-20T19:58:22.363 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T19:58:22.372 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:22.402 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T19:58:22.402 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/668) 2024-04-20T19:58:23.010 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T19:58:23.025 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T19:58:23.025 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:23.457 INFO:teuthology.orchestra.run.smithi121.stdout:generic/668 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T19:58:23.468 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/668 2024-04-20T19:58:23.469 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/668 2024-04-20T19:58:23.469 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T19:58:23.478 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:23.504 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T19:58:23.505 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/669) 2024-04-20T19:58:24.112 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T19:58:24.125 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T19:58:24.125 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:24.556 INFO:teuthology.orchestra.run.smithi121.stdout:generic/669 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T19:58:24.567 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/669 2024-04-20T19:58:24.567 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/669 2024-04-20T19:58:24.567 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T19:58:24.578 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:24.617 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T19:58:24.617 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/670) 2024-04-20T19:58:25.218 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T19:58:25.228 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T19:58:25.228 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:25.585 INFO:teuthology.orchestra.run.smithi121.stdout:generic/670 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T19:58:25.600 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/670 2024-04-20T19:58:25.600 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/670 2024-04-20T19:58:25.601 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T19:58:25.610 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:25.648 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T19:58:25.648 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/671) 2024-04-20T19:58:26.230 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T19:58:26.248 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T19:58:26.248 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:26.724 INFO:teuthology.orchestra.run.smithi121.stdout:generic/671 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T19:58:26.738 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/671 2024-04-20T19:58:26.738 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/671 2024-04-20T19:58:26.738 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T19:58:26.747 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:26.787 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T19:58:26.787 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/672) 2024-04-20T19:58:27.442 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T19:58:27.460 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T19:58:27.460 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:27.907 INFO:teuthology.orchestra.run.smithi121.stdout:generic/672 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T19:58:27.920 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/672 2024-04-20T19:58:27.920 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/672 2024-04-20T19:58:27.920 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T19:58:27.929 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:27.963 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T19:58:27.963 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/673) 2024-04-20T19:58:28.564 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T19:58:28.580 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T19:58:28.580 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:30.191 INFO:teuthology.orchestra.run.smithi121.stdout:generic/673 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T19:58:30.204 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/673 2024-04-20T19:58:30.204 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/673 2024-04-20T19:58:30.204 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T19:58:30.219 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:30.250 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T19:58:30.251 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/674) 2024-04-20T19:58:30.879 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T19:58:30.890 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T19:58:30.890 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:31.873 INFO:teuthology.orchestra.run.smithi121.stdout:generic/674 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T19:58:31.884 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/674 2024-04-20T19:58:31.884 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/674 2024-04-20T19:58:31.885 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T19:58:32.166 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:32.201 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T19:58:32.202 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/675) 2024-04-20T19:58:32.813 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T19:58:32.828 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T19:58:32.828 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:33.382 INFO:teuthology.orchestra.run.smithi121.stdout:generic/675 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T19:58:33.393 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/675 2024-04-20T19:58:33.394 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/675 2024-04-20T19:58:33.394 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T19:58:33.406 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:58:33.443 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T19:58:33.444 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/676) 2024-04-20T19:58:34.070 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T19:58:34.085 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T19:58:34.085 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T19:59:16.305 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:59:16.310 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T19:59:16.421 INFO:tasks.ceph.mds.a.smithi121.stderr:2024-04-20T19:59:16.421+0000 7f95c3c00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 511281) UID: 0 2024-04-20T19:59:16.421 INFO:tasks.ceph.mds.c.smithi121.stderr:2024-04-20T19:59:16.421+0000 7f4d9da00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 511281) UID: 0 2024-04-20T19:59:16.625 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-04-20T19:59:16.626+0000 7f4dce800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 46349) UID: 0 2024-04-20T19:59:16.626 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-04-20T19:59:16.626+0000 7f5d77a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 46349) UID: 0 2024-04-20T19:59:16.626 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-04-20T19:59:16.626+0000 7f1143800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 46349) UID: 0 2024-04-20T19:59:16.627 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T19:59:16.627+0000 7fad6ce00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 46349) UID: 0 2024-04-20T20:07:32.747 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-04-20T20:07:32.745+0000 7fb3ce400640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 511326) UID: 0 2024-04-20T20:07:32.747 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T20:07:32.745+0000 7f9c11a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 511326) UID: 0 2024-04-20T20:07:32.748 INFO:tasks.ceph.osd.2.smithi121.stderr:2024-04-20T20:07:32.745+0000 7f11dfa00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 511326) UID: 0 2024-04-20T20:07:32.748 INFO:tasks.ceph.osd.3.smithi121.stderr:2024-04-20T20:07:32.745+0000 7fca40a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 511326) UID: 0 2024-04-20T20:24:41.453 INFO:teuthology.orchestra.run.smithi121.stdout:generic/676 1567s 2024-04-20T20:24:41.534 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/676 2024-04-20T20:24:41.534 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:24:41.541 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:24:41.578 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:24:41.578 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/677) 2024-04-20T20:24:43.176 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:24:43.192 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:24:43.192 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:24:43.871 INFO:teuthology.orchestra.run.smithi121.stdout:generic/677 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T20:24:43.883 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/677 2024-04-20T20:24:43.883 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/677 2024-04-20T20:24:43.883 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:24:43.890 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:24:43.933 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:24:43.934 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/678) 2024-04-20T20:24:44.628 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:24:44.645 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:24:44.645 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:24:45.305 INFO:teuthology.orchestra.run.smithi121.stdout:generic/678 [not run] kernel does not support IO_URING 2024-04-20T20:24:45.319 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/678 2024-04-20T20:24:45.319 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/678 2024-04-20T20:24:45.319 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:24:45.336 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:24:45.375 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:24:45.376 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/679) 2024-04-20T20:24:46.058 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:24:46.073 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:24:46.073 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:24:46.793 INFO:teuthology.orchestra.run.smithi121.stdout:generic/679 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T20:24:46.805 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/679 2024-04-20T20:24:46.806 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/679 2024-04-20T20:24:46.806 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:24:46.814 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:24:46.847 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:24:46.848 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/680) 2024-04-20T20:24:47.427 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:24:47.449 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:24:47.450 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:24:54.591 INFO:teuthology.orchestra.run.smithi121.stdout:generic/680 7s 2024-04-20T20:24:54.611 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/680 2024-04-20T20:24:54.611 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:24:54.621 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:24:54.667 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:24:54.667 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/681) 2024-04-20T20:24:55.548 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:24:55.561 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:24:55.561 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:24:56.098 INFO:teuthology.orchestra.run.smithi121.stdout:generic/681 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2024-04-20T20:24:56.110 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/681 2024-04-20T20:24:56.111 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/681 2024-04-20T20:24:56.111 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:24:56.120 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:24:56.159 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:24:56.159 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/682) 2024-04-20T20:24:56.809 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:24:56.823 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:24:56.823 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:24:57.494 INFO:teuthology.orchestra.run.smithi121.stdout:generic/682 [not run] disk quotas not supported by this filesystem type: ceph-fuse 2024-04-20T20:24:57.509 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/682 2024-04-20T20:24:57.509 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/682 2024-04-20T20:24:57.509 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:24:57.519 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:24:57.564 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:24:57.564 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/683) 2024-04-20T20:24:58.124 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:24:58.139 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:24:58.139 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:24:59.267 INFO:teuthology.orchestra.run.smithi121.stdout:generic/683 [not run] /tmp/tmp.ru8ajEsoUutest: file alloc unit 4194304 larger than op length 65536 2024-04-20T20:24:59.279 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/683 2024-04-20T20:24:59.279 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/683 2024-04-20T20:24:59.279 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:24:59.287 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:24:59.326 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:24:59.326 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/684) 2024-04-20T20:24:59.925 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:24:59.943 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:24:59.943 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:02.989 INFO:teuthology.orchestra.run.smithi121.stdout:generic/684 [not run] /tmp/tmp.ru8ajEsoUutest: file alloc unit 4194304 larger than op length 65536 2024-04-20T20:25:03.001 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/684 2024-04-20T20:25:03.001 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/684 2024-04-20T20:25:03.001 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:25:03.010 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:03.046 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:25:03.046 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/685) 2024-04-20T20:25:03.661 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:25:03.683 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:25:03.684 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:06.515 INFO:teuthology.orchestra.run.smithi121.stdout:generic/685 [not run] xfs_io fzero failed (old kernel/wrong fs?) 2024-04-20T20:25:06.528 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/685 2024-04-20T20:25:06.528 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/685 2024-04-20T20:25:06.528 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:25:06.541 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:06.578 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:25:06.578 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/686) 2024-04-20T20:25:07.225 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:25:07.242 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:25:07.243 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:10.176 INFO:teuthology.orchestra.run.smithi121.stdout:generic/686 [not run] xfs_io finsert failed (old kernel/wrong fs?) 2024-04-20T20:25:10.191 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/686 2024-04-20T20:25:10.191 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/686 2024-04-20T20:25:10.191 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:25:10.200 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:10.235 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:25:10.235 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/687) 2024-04-20T20:25:10.800 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:25:10.817 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:25:10.817 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:14.011 INFO:teuthology.orchestra.run.smithi121.stdout:generic/687 [not run] xfs_io fcollapse failed (old kernel/wrong fs?) 2024-04-20T20:25:14.021 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/687 2024-04-20T20:25:14.021 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/687 2024-04-20T20:25:14.021 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:25:14.030 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:14.067 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:25:14.067 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/688) 2024-04-20T20:25:14.673 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:25:14.689 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:25:14.689 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:15.408 INFO:teuthology.orchestra.run.smithi121.stdout:generic/688 [not run] /tmp/tmp.ru8ajEsoUutest: file alloc unit 4194304 larger than op length 65536 2024-04-20T20:25:15.420 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/688 2024-04-20T20:25:15.420 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/688 2024-04-20T20:25:15.420 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:25:15.429 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:15.457 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:25:15.457 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/689) 2024-04-20T20:25:16.041 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:25:16.056 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:25:16.057 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:17.624 INFO:teuthology.orchestra.run.smithi121.stdout:generic/689 [not run] vfstest not support by ceph-fuse 2024-04-20T20:25:17.636 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/689 2024-04-20T20:25:17.637 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/689 2024-04-20T20:25:17.637 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:25:17.647 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:17.683 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:25:17.683 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/690) 2024-04-20T20:25:18.309 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:25:18.325 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:25:18.325 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:18.939 INFO:teuthology.orchestra.run.smithi121.stdout:generic/690 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T20:25:18.950 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/690 2024-04-20T20:25:18.950 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/690 2024-04-20T20:25:18.950 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:25:18.961 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:18.994 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:25:18.995 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/691) 2024-04-20T20:25:19.565 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:25:19.578 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:25:19.578 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:19.987 INFO:teuthology.orchestra.run.smithi121.stdout:generic/691 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T20:25:19.998 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/691 2024-04-20T20:25:19.998 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/691 2024-04-20T20:25:19.998 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:25:20.008 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:20.036 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:25:20.037 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/692) 2024-04-20T20:25:20.631 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:25:20.645 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:25:20.645 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:21.066 INFO:teuthology.orchestra.run.smithi121.stdout:generic/692 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T20:25:21.078 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/692 2024-04-20T20:25:21.078 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/692 2024-04-20T20:25:21.078 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:25:21.088 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:21.125 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:25:21.125 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/693) 2024-04-20T20:25:21.696 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:25:21.711 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:25:21.711 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:22.290 INFO:teuthology.orchestra.run.smithi121.stdout:generic/693 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T20:25:22.301 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/693 2024-04-20T20:25:22.301 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/693 2024-04-20T20:25:22.301 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:25:22.310 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:22.343 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:25:22.343 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/694) 2024-04-20T20:25:22.916 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:25:22.930 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:25:22.930 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:24.143 INFO:teuthology.orchestra.run.smithi121.stdout:generic/694 2s 2024-04-20T20:25:24.169 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/694 2024-04-20T20:25:24.169 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:25:24.183 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:24.211 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:25:24.211 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/695) 2024-04-20T20:25:24.830 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:25:24.847 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:25:24.847 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:25.968 INFO:teuthology.orchestra.run.smithi121.stdout:generic/695 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T20:25:25.982 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/695 2024-04-20T20:25:25.983 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/695 2024-04-20T20:25:25.983 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:25:25.995 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:26.027 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:25:26.027 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/696) 2024-04-20T20:25:26.705 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:25:26.720 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:25:26.720 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:27.168 INFO:teuthology.orchestra.run.smithi121.stdout:generic/696 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T20:25:27.182 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/696 2024-04-20T20:25:27.182 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/696 2024-04-20T20:25:27.183 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:25:27.194 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:27.229 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:25:27.229 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/697) 2024-04-20T20:25:27.861 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:25:27.881 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:25:27.881 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:28.445 INFO:teuthology.orchestra.run.smithi121.stdout:generic/697 [not run] ACLs not supported by this filesystem type: ceph-fuse 2024-04-20T20:25:28.456 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/697 2024-04-20T20:25:28.456 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/697 2024-04-20T20:25:28.456 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:25:28.464 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:28.496 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:25:28.497 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/698) 2024-04-20T20:25:29.098 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:25:29.116 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:25:29.116 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:29.541 INFO:teuthology.orchestra.run.smithi121.stdout:generic/698 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T20:25:29.552 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/698 2024-04-20T20:25:29.552 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/698 2024-04-20T20:25:29.552 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:25:29.561 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:29.591 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:25:29.591 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/699) 2024-04-20T20:25:30.207 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:25:30.224 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:25:30.225 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:30.951 INFO:teuthology.orchestra.run.smithi121.stdout:generic/699 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T20:25:30.963 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/699 2024-04-20T20:25:30.963 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/699 2024-04-20T20:25:30.963 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:25:30.973 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:31.010 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:25:31.010 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/700) 2024-04-20T20:25:31.658 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:25:31.674 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:25:31.675 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:32.105 INFO:teuthology.orchestra.run.smithi121.stdout:generic/700 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T20:25:32.122 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/700 2024-04-20T20:25:32.122 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/700 2024-04-20T20:25:32.122 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:25:32.133 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:32.172 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:25:32.172 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/701) 2024-04-20T20:25:32.749 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:25:32.760 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:25:32.760 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:33.725 INFO:teuthology.orchestra.run.smithi121.stdout:generic/701 1s 2024-04-20T20:25:33.747 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/701 2024-04-20T20:25:33.747 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:25:33.757 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:33.809 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:25:33.809 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/702) 2024-04-20T20:25:34.672 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:25:34.685 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:25:34.685 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:35.132 INFO:teuthology.orchestra.run.smithi121.stdout:generic/702 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T20:25:35.144 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/702 2024-04-20T20:25:35.144 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/702 2024-04-20T20:25:35.144 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:25:35.155 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:35.185 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:25:35.185 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/703) 2024-04-20T20:25:35.832 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:25:35.850 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:25:35.850 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:36.414 INFO:teuthology.orchestra.run.smithi121.stdout:generic/703 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T20:25:36.427 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/703 2024-04-20T20:25:36.427 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/703 2024-04-20T20:25:36.427 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:25:36.436 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:36.466 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:25:36.466 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/704) 2024-04-20T20:25:37.093 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:25:37.110 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:25:37.110 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:37.666 INFO:teuthology.orchestra.run.smithi121.stdout:generic/704 [not run] require ceph-fuse to be valid block disk 2024-04-20T20:25:37.678 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/704 2024-04-20T20:25:37.678 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/704 2024-04-20T20:25:37.678 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:25:37.688 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:37.729 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:25:37.729 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/705) 2024-04-20T20:25:38.346 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:25:38.362 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:25:38.362 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:38.778 INFO:teuthology.orchestra.run.smithi121.stdout:generic/705 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T20:25:38.791 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/705 2024-04-20T20:25:38.791 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/705 2024-04-20T20:25:38.791 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:25:38.799 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:38.831 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:25:38.832 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/706) 2024-04-20T20:25:39.453 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:25:39.468 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:25:39.468 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:41.874 INFO:teuthology.orchestra.run.smithi121.stdout:generic/706 2s 2024-04-20T20:25:41.895 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/706 2024-04-20T20:25:41.895 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:25:41.906 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:41.940 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:25:41.941 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/707) 2024-04-20T20:25:42.862 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:25:42.879 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:25:42.879 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:43.429 INFO:teuthology.orchestra.run.smithi121.stdout:generic/707 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T20:25:43.442 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/707 2024-04-20T20:25:43.442 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/707 2024-04-20T20:25:43.442 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:25:43.451 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:43.481 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:25:43.481 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/708) 2024-04-20T20:25:44.080 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:25:44.095 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:25:44.095 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:45.905 INFO:teuthology.orchestra.run.smithi121.stdout:generic/708 1s 2024-04-20T20:25:45.923 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/708 2024-04-20T20:25:45.923 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:25:45.932 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:45.971 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:25:45.972 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/709) 2024-04-20T20:25:46.838 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:25:46.854 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:25:46.854 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:47.542 INFO:teuthology.orchestra.run.smithi121.stdout:generic/709 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-20T20:25:47.555 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/709 2024-04-20T20:25:47.555 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/709 2024-04-20T20:25:47.555 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:25:47.564 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:47.603 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:25:47.603 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/710) 2024-04-20T20:25:48.420 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:25:48.438 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:25:48.438 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:48.977 INFO:teuthology.orchestra.run.smithi121.stdout:generic/710 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-20T20:25:48.995 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/710 2024-04-20T20:25:48.995 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/710 2024-04-20T20:25:48.995 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:25:49.006 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:49.043 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:25:49.043 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/711) 2024-04-20T20:25:49.879 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:25:49.892 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:25:49.892 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:50.395 INFO:teuthology.orchestra.run.smithi121.stdout:generic/711 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-20T20:25:50.404 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/711 2024-04-20T20:25:50.404 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/711 2024-04-20T20:25:50.405 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:25:50.416 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:50.462 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:25:50.462 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/712) 2024-04-20T20:25:51.302 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:25:51.316 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:25:51.316 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:51.843 INFO:teuthology.orchestra.run.smithi121.stdout:generic/712 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-20T20:25:51.853 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/712 2024-04-20T20:25:51.853 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/712 2024-04-20T20:25:51.853 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:25:51.862 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:51.893 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:25:51.894 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/713) 2024-04-20T20:25:52.872 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:25:52.887 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:25:52.888 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:53.705 INFO:teuthology.orchestra.run.smithi121.stdout:generic/713 [not run] xfs_io swapext -v exchrange -s 64k -l 64k not supported on ceph-fuse 2024-04-20T20:25:53.716 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/713 2024-04-20T20:25:53.716 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/713 2024-04-20T20:25:53.716 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:25:53.725 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:53.753 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:25:53.753 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/714) 2024-04-20T20:25:54.663 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:25:54.680 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:25:54.680 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:55.334 INFO:teuthology.orchestra.run.smithi121.stdout:generic/714 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-20T20:25:55.344 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/714 2024-04-20T20:25:55.344 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/714 2024-04-20T20:25:55.344 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:25:55.354 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:55.395 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:25:55.395 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/715) 2024-04-20T20:25:56.275 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:25:56.293 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:25:56.293 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:56.875 INFO:teuthology.orchestra.run.smithi121.stdout:generic/715 [not run] xfs_io swapext -v exchrange -s 64k -l 64k not supported on ceph-fuse 2024-04-20T20:25:56.888 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/715 2024-04-20T20:25:56.888 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/715 2024-04-20T20:25:56.888 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:25:56.901 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:56.944 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:25:56.944 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/716) 2024-04-20T20:25:57.724 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:25:57.741 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:25:57.741 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:58.656 INFO:teuthology.orchestra.run.smithi121.stdout:generic/716 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-20T20:25:58.667 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/716 2024-04-20T20:25:58.667 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/716 2024-04-20T20:25:58.667 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:25:58.677 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:25:58.716 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:25:58.716 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/717) 2024-04-20T20:25:59.504 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:25:59.519 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:25:59.519 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:26:00.068 INFO:teuthology.orchestra.run.smithi121.stdout:generic/717 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-20T20:26:00.081 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/717 2024-04-20T20:26:00.081 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/717 2024-04-20T20:26:00.081 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:26:00.092 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:26:00.128 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:26:00.128 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/718) 2024-04-20T20:26:01.019 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:26:01.041 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:26:01.042 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:26:01.766 INFO:teuthology.orchestra.run.smithi121.stdout:generic/718 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-20T20:26:01.780 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/718 2024-04-20T20:26:01.780 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/718 2024-04-20T20:26:01.780 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:26:01.791 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:26:01.843 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:26:01.843 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/719) 2024-04-20T20:26:02.622 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:26:02.638 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:26:02.638 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:26:03.204 INFO:teuthology.orchestra.run.smithi121.stdout:generic/719 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-20T20:26:03.218 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/719 2024-04-20T20:26:03.218 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/719 2024-04-20T20:26:03.218 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:26:03.226 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:26:03.270 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:26:03.271 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/720) 2024-04-20T20:26:04.065 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:26:04.080 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:26:04.080 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:26:04.667 INFO:teuthology.orchestra.run.smithi121.stdout:generic/720 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-20T20:26:04.682 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/720 2024-04-20T20:26:04.683 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/720 2024-04-20T20:26:04.683 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:26:04.694 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:26:04.737 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:26:04.737 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/721) 2024-04-20T20:26:05.544 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:26:05.562 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:26:05.562 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:26:06.107 INFO:teuthology.orchestra.run.smithi121.stdout:generic/721 [not run] xfs_io startupdate support is missing 2024-04-20T20:26:06.119 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/721 2024-04-20T20:26:06.119 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/721 2024-04-20T20:26:06.119 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:26:06.133 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:26:06.170 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:26:06.171 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/722) 2024-04-20T20:26:06.779 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:26:06.795 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:26:06.795 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:26:07.371 INFO:teuthology.orchestra.run.smithi121.stdout:generic/722 [not run] xfs_io swapext -v exchrange -a not supported on ceph-fuse 2024-04-20T20:26:07.383 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/722 2024-04-20T20:26:07.383 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/722 2024-04-20T20:26:07.383 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:26:07.397 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:26:07.437 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:26:07.437 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/723) 2024-04-20T20:26:08.317 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:26:08.330 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:26:08.331 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:26:08.876 INFO:teuthology.orchestra.run.smithi121.stdout:generic/723 [not run] xfs_io swapext -v exchrange not supported on ceph-fuse 2024-04-20T20:26:08.893 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/723 2024-04-20T20:26:08.893 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/723 2024-04-20T20:26:08.893 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:26:08.905 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:26:08.945 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:26:08.945 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/724) 2024-04-20T20:26:09.800 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:26:09.814 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:26:09.815 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:26:10.402 INFO:teuthology.orchestra.run.smithi121.stdout:generic/724 [not run] xfs_io swapext -v exchrange -a not supported on ceph-fuse 2024-04-20T20:26:10.416 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/724 2024-04-20T20:26:10.416 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/724 2024-04-20T20:26:10.416 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:26:10.427 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:26:10.456 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:26:10.456 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/725) 2024-04-20T20:26:11.287 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:26:11.302 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:26:11.302 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:26:11.970 INFO:teuthology.orchestra.run.smithi121.stdout:generic/725 [not run] xfs_io swapext -v exchrange -a not supported on ceph-fuse 2024-04-20T20:26:11.982 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/725 2024-04-20T20:26:11.982 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/725 2024-04-20T20:26:11.982 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:26:11.993 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:26:12.025 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:26:12.025 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/726) 2024-04-20T20:26:12.921 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:26:12.937 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:26:12.937 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:26:14.773 INFO:teuthology.orchestra.run.smithi121.stdout:generic/726 [not run] xfs_io swapext -v exchrange -a not supported on ceph-fuse 2024-04-20T20:26:14.785 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/726 2024-04-20T20:26:14.786 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/726 2024-04-20T20:26:14.786 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:26:14.797 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:26:14.843 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:26:14.843 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/727) 2024-04-20T20:26:15.697 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:26:15.718 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:26:15.718 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:26:16.412 INFO:teuthology.orchestra.run.smithi121.stdout:generic/727 [not run] xfs_io swapext -v exchrange -a not supported on ceph-fuse 2024-04-20T20:26:16.424 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/727 2024-04-20T20:26:16.424 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/727 2024-04-20T20:26:16.424 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:26:16.434 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:26:16.483 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:26:16.483 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/728) 2024-04-20T20:26:17.297 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:26:17.312 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:26:17.313 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:26:22.064 INFO:teuthology.orchestra.run.smithi121.stdout:generic/728 5s 2024-04-20T20:26:22.083 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/728 2024-04-20T20:26:22.084 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:26:22.097 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:26:22.135 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:26:22.135 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/729) 2024-04-20T20:26:22.729 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:26:22.743 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:26:22.743 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:26:23.742 INFO:teuthology.orchestra.run.smithi121.stdout:generic/729 1s 2024-04-20T20:26:23.759 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/729 2024-04-20T20:26:23.759 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:26:23.771 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:26:23.804 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:26:23.804 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/730) 2024-04-20T20:26:24.682 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:26:24.693 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:26:24.693 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:26:25.186 INFO:teuthology.orchestra.run.smithi121.stdout:generic/730 [not run] require ceph-fuse to be valid block disk 2024-04-20T20:26:25.200 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/730 2024-04-20T20:26:25.200 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/730 2024-04-20T20:26:25.201 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:26:25.209 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:26:25.241 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:26:25.241 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/731) 2024-04-20T20:26:25.880 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:26:25.898 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:26:25.898 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:26:26.375 INFO:teuthology.orchestra.run.smithi121.stdout:generic/731 [not run] require ceph-fuse to be valid block disk 2024-04-20T20:26:26.384 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/731 2024-04-20T20:26:26.384 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/731 2024-04-20T20:26:26.384 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:26:26.393 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:26:26.429 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:26:26.429 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/732) 2024-04-20T20:26:27.064 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:26:27.077 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:26:27.077 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:26:27.519 INFO:teuthology.orchestra.run.smithi121.stdout:generic/732 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T20:26:27.531 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/732 2024-04-20T20:26:27.532 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/732 2024-04-20T20:26:27.532 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:26:27.544 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:26:27.582 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:26:27.583 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/733) 2024-04-20T20:26:28.151 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:26:28.172 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:26:28.172 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:26:28.990 INFO:teuthology.orchestra.run.smithi121.stdout:generic/733 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T20:26:29.000 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/733 2024-04-20T20:26:29.001 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/733 2024-04-20T20:26:29.001 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:26:29.011 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:26:29.049 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:26:29.049 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/734) 2024-04-20T20:26:29.695 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:26:29.717 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:26:29.718 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:26:30.348 INFO:teuthology.orchestra.run.smithi121.stdout:generic/734 [not run] Reflink not supported by test filesystem type: ceph-fuse 2024-04-20T20:26:30.368 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/734 2024-04-20T20:26:30.369 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/734 2024-04-20T20:26:30.369 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:26:30.382 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:26:30.418 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:26:30.418 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/735) 2024-04-20T20:26:31.258 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:26:31.273 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:26:31.273 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:26:34.247 INFO:teuthology.orchestra.run.smithi121.stdout:generic/735 [not run] xfs_io finsert failed (old kernel/wrong fs?) 2024-04-20T20:26:34.266 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/735 2024-04-20T20:26:34.266 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/735 2024-04-20T20:26:34.266 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:26:34.284 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:26:34.328 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:26:34.328 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/736) 2024-04-20T20:26:35.361 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:26:35.375 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:26:35.376 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:27:05.501 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T20:27:05.507 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T20:27:05.647 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-04-20T20:27:05.648+0000 7f4dce800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 46428) UID: 0 2024-04-20T20:27:05.648 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-04-20T20:27:05.649+0000 7f1143800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 46428) UID: 0 2024-04-20T20:27:05.648 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-04-20T20:27:05.648+0000 7f5d77a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 46428) UID: 0 2024-04-20T20:27:05.648 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T20:27:05.649+0000 7fad6ce00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 46428) UID: 0 2024-04-20T20:27:05.969 INFO:tasks.ceph.mds.c.smithi121.stderr:2024-04-20T20:27:05.970+0000 7f4d9da00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 537995) UID: 0 2024-04-20T20:27:05.970 INFO:tasks.ceph.mds.a.smithi121.stderr:2024-04-20T20:27:05.970+0000 7f95c3c00640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 537995) UID: 0 2024-04-20T20:30:09.215 INFO:teuthology.orchestra.run.smithi121.stdout:generic/736 213s 2024-04-20T20:30:10.582 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/736 2024-04-20T20:30:10.582 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:30:10.592 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:30:10.628 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:30:10.628 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/737) 2024-04-20T20:30:11.564 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:30:11.577 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:30:11.577 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:30:12.362 INFO:teuthology.orchestra.run.smithi121.stdout:generic/737 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T20:30:12.375 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/737 2024-04-20T20:30:12.376 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/737 2024-04-20T20:30:12.376 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:30:12.385 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:30:12.414 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:30:12.414 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/738) 2024-04-20T20:30:13.009 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:30:13.025 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:30:13.025 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:30:16.795 INFO:teuthology.orchestra.run.smithi121.stdout:generic/738 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T20:30:16.809 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/738 2024-04-20T20:30:16.809 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/738 2024-04-20T20:30:16.809 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:30:16.818 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:30:16.848 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:30:16.848 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/739) 2024-04-20T20:30:17.432 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:30:17.584 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:30:17.584 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:30:18.526 INFO:teuthology.orchestra.run.smithi121.stdout:generic/739 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T20:30:18.540 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/739 2024-04-20T20:30:18.541 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/739 2024-04-20T20:30:18.541 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:30:18.551 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:30:18.617 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:30:18.617 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/741) 2024-04-20T20:30:19.247 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:30:19.267 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:30:19.267 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:30:19.991 INFO:teuthology.orchestra.run.smithi121.stdout:generic/741 [not run] this test requires a valid $SCRATCH_DEV 2024-04-20T20:30:20.005 INFO:teuthology.orchestra.run.smithi121.stdout:Ran: generic/741 2024-04-20T20:30:20.005 INFO:teuthology.orchestra.run.smithi121.stdout:Not run: generic/741 2024-04-20T20:30:20.005 INFO:teuthology.orchestra.run.smithi121.stdout:Passed all 1 tests 2024-04-20T20:30:20.015 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:30:20.050 INFO:tasks.cephfs.xfstests_dev:Command return value: 0 2024-04-20T20:30:20.050 DEBUG:teuthology.orchestra.run.smithi121:> (cd /tmp/tmp.yONpfXTHudxfstests-dev && exec sudo env DIFF_LENGTH=0 ./check generic/742) 2024-04-20T20:30:20.620 INFO:teuthology.orchestra.run.smithi121.stdout:FSTYP -- ceph-fuse 2024-04-20T20:30:20.636 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM -- Linux/x86_64 smithi121 6.9.0-rc3-g806ec36b2112 #1 SMP PREEMPT_DYNAMIC Fri Apr 19 09:08:57 UTC 2024 2024-04-20T20:30:20.637 INFO:teuthology.orchestra.run.smithi121.stdout: 2024-04-20T20:42:46.703 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T20:42:46.703+0000 7f9c11a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 540496) UID: 0 2024-04-20T20:42:46.703 INFO:tasks.ceph.osd.3.smithi121.stderr:2024-04-20T20:42:46.703+0000 7fca40a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 540496) UID: 0 2024-04-20T20:42:46.704 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-04-20T20:42:46.703+0000 7fb3ce400640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 540496) UID: 0 2024-04-20T20:42:46.704 INFO:tasks.ceph.osd.2.smithi121.stderr:2024-04-20T20:42:46.704+0000 7f11dfa00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 540496) UID: 0 2024-04-20T20:53:05.022 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T20:53:05.029 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T20:53:05.245 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-04-20T20:53:05.245+0000 7f4dce800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 46874) UID: 0 2024-04-20T20:53:05.245 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-04-20T20:53:05.246+0000 7f5d77a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 46874) UID: 0 2024-04-20T20:53:05.246 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-04-20T20:53:05.246+0000 7f1143800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 46874) UID: 0 2024-04-20T20:53:05.246 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T20:53:05.246+0000 7fad6ce00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 46874) UID: 0 2024-04-20T20:53:05.537 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T20:53:05.538+0000 7f9c11a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 540656) UID: 0 2024-04-20T20:53:05.537 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-04-20T20:53:05.538+0000 7fb3ce400640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 540656) UID: 0 2024-04-20T20:53:05.537 INFO:tasks.ceph.osd.2.smithi121.stderr:2024-04-20T20:53:05.538+0000 7f11dfa00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 540656) UID: 0 2024-04-20T20:53:05.542 INFO:tasks.ceph.osd.3.smithi121.stderr:2024-04-20T20:53:05.538+0000 7fca40a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 540656) UID: 0 2024-04-20T21:02:45.231 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:02:45.237 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:02:45.358 INFO:tasks.ceph.osd.1.smithi121.stderr:2024-04-20T21:02:45.358+0000 7f9c11a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 540750) UID: 0 2024-04-20T21:02:45.359 INFO:tasks.ceph.osd.0.smithi121.stderr:2024-04-20T21:02:45.358+0000 7fb3ce400640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 540750) UID: 0 2024-04-20T21:02:45.359 INFO:tasks.ceph.osd.3.smithi121.stderr:2024-04-20T21:02:45.358+0000 7fca40a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 540750) UID: 0 2024-04-20T21:02:45.359 INFO:tasks.ceph.osd.2.smithi121.stderr:2024-04-20T21:02:45.359+0000 7f11dfa00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 540750) UID: 0 2024-04-20T21:07:33.801 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:07:33.808 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:07:33.896 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-04-20T21:07:33.896+0000 7f4dce800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 47049) UID: 0 2024-04-20T21:07:33.897 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-04-20T21:07:33.896+0000 7f1143800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 47049) UID: 0 2024-04-20T21:07:33.897 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T21:07:33.896+0000 7fad6ce00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 47049) UID: 0 2024-04-20T21:07:33.897 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-04-20T21:07:33.896+0000 7f5d77a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 47049) UID: 0 2024-04-20T21:11:39.487 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:11:39.494 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:12:10.335 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:12:10.346 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:12:40.453 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:12:40.460 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:13:10.592 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:13:10.597 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:13:10.694 INFO:tasks.ceph.osd.4.smithi123.stderr:2024-04-20T21:13:10.695+0000 7f4dce800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 47195) UID: 0 2024-04-20T21:13:10.695 INFO:tasks.ceph.osd.7.smithi123.stderr:2024-04-20T21:13:10.695+0000 7f5d77a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 47195) UID: 0 2024-04-20T21:13:10.695 INFO:tasks.ceph.osd.6.smithi123.stderr:2024-04-20T21:13:10.695+0000 7fad6ce00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 47195) UID: 0 2024-04-20T21:13:10.695 INFO:tasks.ceph.osd.5.smithi123.stderr:2024-04-20T21:13:10.695+0000 7f1143800640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 47195) UID: 0 2024-04-20T21:16:43.697 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:16:43.709 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:17:13.874 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:17:13.884 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:17:44.027 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:17:44.036 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:18:14.265 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:18:14.275 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:18:45.253 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:18:45.263 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:19:15.396 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:19:15.407 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:19:45.653 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:19:45.662 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:20:15.824 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:20:15.833 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:20:46.022 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:20:46.032 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:21:16.226 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:21:16.235 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:21:46.400 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:21:46.409 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:22:16.614 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:22:16.622 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:22:46.801 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:22:46.810 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:23:16.999 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:23:17.008 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:23:47.136 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:23:47.145 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:24:17.309 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:24:17.317 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:24:47.457 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:24:47.467 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:25:17.646 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:25:17.656 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:25:47.868 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:25:47.878 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:26:17.999 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:26:18.009 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:26:48.129 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:26:48.139 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:27:18.294 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:27:18.306 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:27:48.448 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:27:48.457 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:28:18.671 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:28:18.680 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:28:48.860 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:28:48.871 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:29:19.056 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:29:19.065 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:29:49.186 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:29:49.196 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:30:19.359 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:30:19.369 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:30:49.497 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:30:49.507 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:31:19.811 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:31:19.820 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:31:49.943 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:31:49.954 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:32:20.091 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:32:20.100 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:32:50.220 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:32:50.231 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:33:20.418 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:33:20.428 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:33:50.565 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:33:50.576 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:34:20.737 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:34:20.747 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:34:50.933 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:34:50.942 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:35:21.123 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:35:21.132 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:35:51.269 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:35:51.278 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:36:21.401 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:36:21.409 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:36:51.573 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:36:51.582 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:37:21.753 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:37:21.765 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:37:51.909 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:37:51.919 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:38:22.818 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:38:22.828 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:38:52.961 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:38:52.971 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:39:23.103 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:39:23.112 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:39:53.306 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:39:53.318 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:40:23.503 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:40:23.512 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:40:53.684 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:40:53.694 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:41:23.890 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:41:23.900 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:41:54.095 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:41:54.104 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:42:24.285 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:42:24.295 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:42:54.456 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:42:54.466 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:43:24.644 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:43:24.654 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:43:54.884 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:43:54.894 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:44:25.090 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:44:25.098 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:44:55.218 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:44:55.229 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:45:25.349 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:45:25.358 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:45:55.488 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:45:55.498 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:46:25.618 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:46:25.629 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:46:55.799 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:46:55.809 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:47:26.014 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:47:26.025 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:47:56.227 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:47:56.237 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:48:26.348 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:48:26.357 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:48:56.496 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:48:56.505 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:49:26.643 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:49:26.654 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:49:56.824 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:49:56.834 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:50:27.030 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:50:27.040 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:50:57.209 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:50:57.219 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:51:27.343 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:51:27.354 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:51:57.487 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:51:57.497 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:52:27.617 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:52:27.625 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:52:57.756 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:52:57.766 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:53:27.945 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:53:27.954 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:53:58.126 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:53:58.135 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:54:28.256 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:54:28.265 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:54:58.495 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:54:58.506 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:55:28.691 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:55:28.700 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:55:58.872 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:55:58.881 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:56:29.087 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:56:29.096 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:57:00.458 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:57:00.468 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:57:30.772 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:57:30.781 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:58:00.902 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:58:00.910 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:58:31.075 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:58:31.085 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:59:01.298 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:59:01.306 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:59:31.494 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T21:59:31.503 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T22:00:01.844 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T22:00:01.854 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T22:00:32.073 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T22:00:32.082 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T22:01:02.288 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T22:01:02.297 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T22:01:32.490 DEBUG:teuthology.orchestra.run.smithi121:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T22:01:32.499 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T22:01:56.088 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2024-04-20T22:01:56.089 DEBUG:teuthology.task.console_log:Killing console logger for smithi121 2024-04-20T22:01:56.089 DEBUG:teuthology.task.console_log:Killing console logger for smithi123 2024-04-20T22:01:56.090 DEBUG:teuthology.exit:Finished running handlers